henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,381 Commits
34 Branches
0 Tags
19 MiB
thrimbletrimmer-solid
HubbeKing-multiplatform-builds
bus-progress-ui
master
bus-post-processing
chrusher/bus_synthesizer
mike/faster-chat-merge
mike/chat-no-end-time
chrusher-playlist-thumbnails
chrusher-thumbnail-frontend
mike/prizebot
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '0f7b181be1'
${ noResults }
Commit Graph

1381 Commits (0f7b181be1a52cc0a6db158aa50473adcca51bfb)
 

Author SHA1 Message Date
Mike Lang 0f7b181be1 Create buscribe database during setup 1 month ago
Mike Lang 9c6340a5e0 nginx: buscribe_api, like thrimshim, should be forwarded to 1 month ago
Mike Lang 1e7e400096 docker-compose: Pass correct database args for buscribe 1 month ago
Mike Lang dde7696877 buscribe: Work with no end time 1 month ago
Mike Lang ff8475dc48 Disable type registration when buscribe connects to the DB 
Because those types don't exist in the buscribe DB.
 1 month ago
Mike Lang b16ceef96a rename buscribe-api to buscribe_api 
to match other services that use _ instead of -
 1 month ago
Mike Lang 834d27c17d Add buscribe to docker-compose 1 month ago
Mike Lang 005ff9dac7 buscribe: Do database schema and user setup 1 month ago
Mike Lang 5f3a02fd14 buscribe-remainder: delete things we've already handled/don't need 1 month ago
Mike Lang d5caace905 Move buscribe/buscribe to buscribe/ and everything else to buscribe-remainder/ 
And make buscribe buildable using ./build
 1 month ago
Mike Lang bcf585776e buscribe: Download models as part of build process 1 month ago
Mike Lang ee5e23b2bb Delete buscribe professor 
The professor system for annotating the data was never fully finished and isn't needed
for our purposes. We can always resurrect it later, but for now it's easier to just get rid of it.
 1 month ago
Mike Lang 510af15418 Serve buscribe-web out of the main nginx container, like thrimbletrimmer 
This involves, as part of the nginx build process, having a build container that runs less
in order to render the css before copying it over.
 1 month ago
Mike Lang 2d40893825 Move buscribe-web to top level 1 month ago
Mike Lang b18688e901 buscribe-api: Change path to be under /buscribe-api instead of /buscribe 1 month ago
Mike Lang 752c5ad610 buscribe-api: Finish removing segments dir stuff 1 month ago
Mike Lang e758418f76 buscribe-api: Move from buscribe/ to top level and add to ./build 1 month ago
Mike Lang 2462a957d2 build: Split and format components array 
No actual changes
 1 month ago
Mike Lang 54a97c2d01 buscribe-api: Remove VTT support 
This is unused, and it was using get_best_segments to work out the actual segment time range,
which is the only thing in buscribe-api that requires actual access to segments.
 1 month ago
Mike Lang 961bc56fd4 buscribe: Use wubloader's version of common instead of an old copy 
None of the apis it uses has changed, so no changes required
except for having the dockerfiles take the full wubloader repo as build context.
 1 month ago
Mike Lang ea0e84f476 Add 'buscribe/' from commit 'fbb6c4dca07ace89eab39f789537bd0f694c77eb' 
git-subtree-dir: buscribe
git-subtree-mainline: e923853ca8
git-subtree-split: fbb6c4dca0
 1 month ago
Mike Lang e923853ca8 sheetsync: Update playlist sheet column order 1 month ago
Mike Lang 1915d98143 thrimbletrimmer: Split error path for submission into a function 
and use it correctly when erroring in transition code
 1 month ago
Mike Lang ca60852b3d thrimbletrimmer: Add transitions to download URL 1 month ago
Mike Lang 8c6141b5ca thrimbletrimmer: Correct video time conversions due to overlap from transitions 1 month ago
Mike Lang c96c7cedd6 thrimbletrimmer: add transitions when submitting 1 month ago
Mike Lang 54695bcad4 thrimbletrimmer: Set existing transition info when loading a video 1 month ago
Mike Lang 72110821d5 thrimbletrimmer: Show/hide transition duration if "cut" is selected 1 month ago
Mike Lang 5d09115a8a thrimbletrimmer: Load transition types from thrimshim and populate dropdown 1 month ago
Mike Lang 87d4520e61 thrimbletrimmer: Add new DOM elements for specifying transitions 
Expects a "transitions" list on the video info that currently isn't there.
Not actually used anywhere yet.
 1 month ago
Mike Lang 23e64116bd thrimbletrimmer: Add some helpers for creating elements 
No functional changes intended.
 1 month ago
Mike Lang 3fb6676fbe pubbot: Add more detail to logging 
Wrap raw pubnub messages in a metadata object containing:
- time
- our hostname
- our pid
- any inferred information

Plus make a message-less log write on startup so we know where there might be gaps.

Also do more error handling so that we don't completely lose messages on error.
 2 months ago
Mike Lang 9b19c12a53 Add pubnub-watching zulip bot 
Which also records all pubnub messages for archival to a file SEGMENTS/pubnub-log.json.
We ran this for most of last year, but not as part of wubloader.
 2 months ago
ZeldaZach c378a1e4ab Add Audit Logging for several endpoints 
- Use transactions for DB commits to avoid audit-less logs

Endpoints Supported:
- Manual Link
- Reset Row
- Update Row
 2 months ago
Mike Lang fd78ff288e fix bugs in postgres schema from earlier changes 2 months ago
Mike Lang 7483794a23 docker-compose: Set up sheetsync for downloading media links 
This involves giving it access to the SEGMENTS/media directory.
 2 months ago
Mike Lang 3e7cb38cf0 sheetsync: Optionally download media linked in image links column 
To enable this, you need to:
- set --media-dir globally for sheetsync
- enable download_media=true for the events sync config
To disable for individual rows (eg. because of known issues), put "[nodownload]" in the notes column.
 2 months ago
Mike Lang 7b590cf574 chat-archiver: Some cleanups to the URL matching regex 
With thanks to Me-Me for review
 2 months ago
Mike Lang 15f86551d4 docker-compose: Backfill of media, with or without chat archiver 2 months ago
Mike Lang 9dfb00f4ab chat_archiver: Logic for checking and downloading media links 2 months ago
Mike Lang 2855ec759d download_media: Add pdf to default allowed content types 
We want to capture linked PDFs in addition to videos and images
 2 months ago
Mike Lang b46c577014 download_media: Add function for checking if a URL has been downloaded before 2 months ago
Mike Lang 352c9e9081 download_media: Get data from potentially malicious URLs and store in the filesystem 
This is suitable for taking arbitary URLs from chat, etc and trying to fetch them.
It downloads them to a filepath that contains a hash of the URL and content.
 2 months ago
Mike Lang 07055e3605 chat-archiver: extract the ensure_emotes greenlet management to a class 2 months ago
Mike Lang 3f6263a037 playlist-manager: More fixes 3 months ago
Mike Lang d8953568b2 playlist_manager: Add --once option 3 months ago
Mike Lang 9d12d37053 playlist_manager: Fixes 3 months ago
Mike Lang 23ad78d592 Record in database when end time is "--" 
We need this so that reverse sync reproduces these values correctly.

To handle this in the database, we have a composite type (dashed: boolean, value: timestamp).
Value is always valid and is equivalent to the old timestamp column,
but must be equal to start_time if dashed is true.

The only place we directly reference this column outside sheetsync is thrimshim, where we
always consider the value only.
 3 months ago
Mike Lang e50adbf2da Fix a bug where transitions past the first are not timed correctly 
The video offset is timed relative to the full video up until that point, not the previous range.
 3 months ago
Mike Lang 69bfa79e24 Make video transition durations floats, not intervals 
Almost all code was already expecting this.
 3 months ago