henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
373 Commits
34 Branches
0 Tags
19 MiB
Commit Graph

373 Commits (80d829b83be87a2c20bce6e52c578f74c8c80579)
 

Author SHA1 Message Date
Mike Lang 80d829b83b full cut: ffmpeg requires a seekable output file 
Most formats like mp4 require ffmpeg to make changes at the start of the file
throughout writing.

Unfortunately, this prevents us from streaming the upload as we cut it.

Instead, we spool to a temporary file until ffmpeg exits,
then upload that all at once.
Mike Lang 3a9543a4b5 Suppress less ffmpeg output when cutting 
The "fatal" level was causing some useful errors to be suppressed.
Mike Lang 3fbbe59b00 cutter Local backend: Fix typo and file extension when full cutting
Mike Lang b0a71fd9f1 cutter: Fix mistake in check_candidate error handling
Mike Lang ba53172cbc cutter: Get video link from backend instead of hard-coding youtube
Mike Lang 12decf015e Fix multiple typos and mistakes with full cuts
Mike Lang ed2395d08e cutter: Fix typo that prevented backends from being configured
Mike Lang bf55b1c75e
Merge pull request from ekimekim/mike/cutter/full-cut 
Resurrect non-experimental cut, now dubbed "full" (vs "fast") cut
Mike Lang c970677a76
full cut: Fix a typo
Mike Lang 1218b694dd cutter: Allow fast or full cuts per location 
On full cut, use encoding settings determined by the backend.
Mike Lang 09887f17ab restreamer: Add option to download full cut instead of fast cut 
Transcoding to mp4 with no specific encoding args as a reasonable default.
Mike Lang d3e1d6b4fc Resurrect non-experimental cut, now dubbed "full" (vs "fast") cut 
In a fast cut, we edit the first and last segments then concatenate them all.
However, this leads to some tiny but perciptible artifacting around the border
of the first and second (and second-last and last) segments.

A full cut is much slower, but re-encodes the video into the desired format
and is more reliable.

We want both options to be available.

With this commit, we only add the option, we don't use it in restreamer or cutter.
Mike Lang 6533eed4d4
Merge pull request from ekimekim/mike/cutter/tags-and-things 
tags, other video presentation tweaks and things
Mike Lang 40c4baef0f youtube upload: Set category and language settings 
configured on a per-location basis.
Mike Lang 5e0d5b9ddc cutter: Add category and sheet_name as video tags 
This gives us tags on categories, and on which day's sheet the event came from.

We use these to make automatic playlists.
Mike Lang cbd0ef3d9e cutter: Add title header, description footer and static tags 
These are pre-canned parts of the video metadata that we want to be configurable.
Mike Lang 48593e2b06 database, sheetsync: Add worksheet name column 'sheet_name' 
This tells us which sheet a row came from
(so we don't need to scan every sheet to find it if we're trying to do
lookups in that direction).

It is also needed in order to tag the videos with the Day number.
Mike Lang 1159a518f0
Merge pull request from ekimekim/mike/cutter/multiple-locations 
cutter: Allow multiple upload locations
Mike Lang 1a177931a6 cutter: Simplify upload backend interface 
Only support iterable of string, not file-like or string.

This is a minor usability loss but we only call this from one place anyway
and it's always an iterable of string.
Mike Lang 2194e47ab5
Merge pull request from ekimekim/mike/cutter/set-error-on-holes 
cutter: Set an error message to warn humans when it discards a candidate due to holes
Mike Lang f8039028c4 local upload backend: Add a testing option to show info upload_video() was called with 
This is useful because it lets us test uploads and tag logic without involving youtube.
Mike Lang d6ed138db8 cutter: Set an error message to warn humans when it discards a candidate due to holes 
This prevents videos being stuck in EDITED with no visible problem when
they contain holes, but is likely to false positive sometimes.

This is fine though, as it's just a human-readable warning and
it will be cleared as soon as any node accepts the row to be cut.
Mike Lang 75acd978de cutter: Implement a Local upload location 
This cuts to local disk, useful for self-hosting and testing.
Mike Lang 8e6c40610a cutter: Allow config to manually disable transcode checking for location 
This deals with the problem where multiple youtube locations that refer
to the same actual account (but with different settings) will all try to check
for when videos are done transcoding, when only one is needed.
Mike Lang ecf58dfbfc cutter: Add ability to configure multiple upload locations 
Cutter now takes a 'config' arg which is a json blob with detail
on each upload location. This is a bit nasty if you're trying to run it manually
but was the easiest way to transfer the config data from docker-compose.jsonnet
to the actual application.
Mike Lang 514a7a4d08 Split cutter and sheetsync creds so they can be different 
This also means that they can't access each others' creds,
which is good for security.
Mike Lang 32d352e2ba cutter: Set video directly to DONE if upload backend doesn't need transcoding 
This will be the case for probably all non-youtube backends.
Mike Lang 39f006fdab cutter: Replace youtube client with generic upload backend and specific youtube implementaton 
Still to go: Actually constructing the correct backend based on given config,
switching behaviour based on needs_transcode.
Mike Lang 15f879682d cutter: Allow job to specify upload location 
This lays the groundwork for being able to cut to many upload locations.
Right now, only a single location can be configured, and only youtube is supported.
Christopher Usher 9ed8facce8 fixed bug in error handling
Christopher Usher a63e18c0a9 added state to upload_errors
Christopher Usher 228468749c changes based on ekim's suggestions
Christopher Usher 5c08540c0a stats for the cutter
Christopher Usher 989ef80d30 starting on adding some stats to the cutter
Christopher Usher f9ce41ef32 metrics for the sheetsync
Christopher Usher 928f9733d2 horrible bug with negative times fixed
Christopher Usher a2e47b98f5 fixes -- in dates and the lack of the preshow
Christopher Usher ccb7f3c684 now use parse_segment_path to get hash from filename
Christopher Usher a6303c38ce fixed parse_segment_path to allow just a filename to be parsed
Christopher Usher 36da1926d0 fixes for ekims suggestions
Christopher Usher 34785d1179 now checking the hashes
Christopher Usher b0562495d2 reject mismatched hashes; more metrics
Christopher Usher 120a5a7de0 started on checking the hash
Ubuntu 875e4271a6 Fixing handling of null times.
MasterGunner 04f7928715 Cleaning up the new dashboard columns.
MasterGunner 52e456e0c1 Draft Button and Dashboard Columns
Christopher Usher 632c5fae2f added a default timeout to database connections
Christopher Usher d1e2957198 minor style changes to docker-compose
Chris Usher 557ddddc31 better logging for the backfiller
MasterGunner 919f993c33
Merge pull request from ekimekim/gunner/security 
Proposal for security model