henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
567 Commits
34 Branches
0 Tags
19 MiB
Commit Graph

567 Commits (3fabb2944febc827360d0cfb1b642305c26359ec)
 

Author SHA1 Message Date
Christopher Usher ac72f775c9 functional
Christopher Usher 223af52265 added coverage to docker-compose
Christopher Usher 722cbd20fa first pass at checking for holes and repeats
Christopher Usher 66f5a06a5c basic segment counting working
Christopher Usher 3618510f35 basic functionality
Christopher Usher 929308f3e7 started on the segment_coverage service
Mike Lang a42e7b48f6 thrimshim: Allow degraded operation even if DB broken 
Any endpoints that don't need a DB conn will still work fine.

Notably, this includes /defaults, which is needed for
thrimbletrimmer to work in a non-specific-row mode.
Mike Lang 17af1c4e89 cutter, sheetsync: Wait for DB to connect on startup 
This is a nicer error than crashing in the depths of some error handler
(which is what will happen if the DB goes unavailable while they're running),
and it's a far more common case (eg. the DB is misconfigured) than having it fail
halfway through.

Neither of these services can do anything meaningful without the DB,
so crashing without it is acceptable behaviour.
Mike Lang 0e437566aa backfiller: Don't crash on DB errors 
We move all connection handling into get_nodes().
This means that problems connecting won't cause further errors
and cause the application to completely crash.

In turn, this means that the behaviour if the database goes down becomes
"continue backfilling from the nodes we know about" instead of crashing.
Mike Lang fc791e03d4 DBManager: Don't test connection on start 
This gives the individual services more freedom in how to handle
a failing connection.
Mike Lang 3c20a9aece nginx: Serve the segments directly from the segments dir 
instead of proxying through restreamer.

This should improve performance when serving the (large) segment files,
and free up restreamer for things like generating the playlist.
Mike Lang 095e391b60
Merge pull request from ekimekim/mike/thrimbletrimmer-things 
A lot of changes to thrimbletrimmer and related stuff
Mike Lang 934c640a04 thrimbletrimmer: Display edit notes below the video 
In nice distracting pink so they're sure to see it.
Mike Lang f7f07a2688 Add /defaults to thrimshim, use it to populate config stuff in thrimbletrimmer 
Thrimbletrimmer needs to know stuff like the bustime start time,
default channel, even if it's not looking at a specific row.
Mike Lang d03120548b thrimbletrimmer: Always respect the channel given by thrimshim
Mike Lang 6cfad08a26 thrimbletrimmer: Restore previously saved advanced options 
and show the advanced pane if any are non-default.
Mike Lang 728ede61c0 thrimbletrimmer: Don't leave page after saving edits
Mike Lang 983e3a61c6 thrimbletrimmer: also pause on spacebar 
Most video players pause on space, so we make sure it works if they try
Mike Lang 775799944a thrimbletrimmer: Don't hide the video player on error 
It means you can't read the error, and it's more confusing.
Mike Lang 7dea1f0cdb thrimbletrimmer: Fix some typos in the new error handling
Mike Lang da3cc24ed1 Pass a list of upload locations from thrimshim to thrimbletrimmer 
with the first one being the default.
Mike Lang b11fe39371 thrimshim: Validate title length, non-empty title/description 
and pass the title prefix / max length through to thrimbletrimmer
Mike Lang ba746ff6e6 Add title and description header/footer in thrimshim instead of cutter 
This accomplishes two things:
1. It allows thrimshim to properly validate length restrictions (not implemented yet)
2. It means that the database has a record of the values actually written for each of these rows,
instead of that information depending on how the cutter was configured at the time.
Mike Lang b0ea790407 thrimbletrimmer: Misc cleanup
Mike Lang 93fc202394 thrimbletrimmer: submit button: clean up, error handling, no-auth mode
Mike Lang cdd286c0ee thrimbletrimmer: manual link and download buttons: cleanup, error handling, no-auth mode
Mike Lang b06721df07 thrimbletrimmer: Update bustime/UTC values when you switch 
so if you change one it changes the other.

Also in general takes those conversions and makes them available as functions for re-use.
Mike Lang 91c67899ee thrimbletrimmer: Read title prefix from thrimshim and display it greyed out 
This gives a visual indicator showing the editor what prefix the title will have.
It also allows us to set the correct max title length.
Mike Lang caa129e192 thrimbletrimmer: allow holes should be off by default
Mike Lang 93f812eb64 thrimbletrimmer: Fix Streams and Hours links to not leave the current page 
The existing approach, at least in my browser (firefox),
was causing the page to be replaced with "[window Object]".
Mike Lang 15cf65c926 nginx: Nicer formatting for thrimbletrimmer part of config
Mike Lang 60c6145836 nginx: Fix invalid config when no services deployed 
ie. for a thrimbletrimmer-only node.
Mike Lang e5c5eb4090
Merge pull request from ekimekim/mike/config-fixes 
A grab-bag of fixes that came out of the recent test on toodles
ekimekim 6ef27fe757 nginx: Always use relative paths 
So that using non-standard ports, etc. doesn't break things
ekimekim 742fcc3a28 Update default worksheet list to correct typo
ekimekim 86f22d38d8 docker-compose: Need to tell sheet-sync to allocate ids 
NOTE: This is only safe if we're only running one.

Later we should make a way to control this in config so one node has it
but others can run non-allocating sheetsyncs.
Mike Lang b2d3faeab2
Merge pull request from ekimekim/mike/cutter/upload-errors 
Refactor error handling in uploads
Mike Lang 596cd92644 cutter: Add more specific error handling to upload backends 
For youtube, know that 4xx's are safe even if finalizing was set.

For local, make all disk errors retryable since it doesn't matter.
Mike Lang 736040435c Refactor error handling in uploads 
Instead of handling each error condition seperately,
we raise an UploadError which includes whether it's retryable.

The advantage of this is that upload backends can also raise an UploadError
to indicate two conditions it currently cannot:
	That an error is unretryable
	That an error is retryable, even if the row was already in finalizing

Under this scheme, errors while cutting become unretryable UploadErrors,
and unhandled exceptions in uploading become retryable UploadErrors if
the row is not yet finalizing only.
Mike Lang e435abf72e
Merge pull request from ekimekim/mike/fixes 
Grab-bag of cutter fixes
Mike Lang 40458d9d7f local backend: Use original version of title in write_info 
instead of safe version
Mike Lang 80d829b83b full cut: ffmpeg requires a seekable output file 
Most formats like mp4 require ffmpeg to make changes at the start of the file
throughout writing.

Unfortunately, this prevents us from streaming the upload as we cut it.

Instead, we spool to a temporary file until ffmpeg exits,
then upload that all at once.
Christopher Usher f4cd3f546e removed comments no longer needed
Christopher Usher 51e4520826 replaced warnings.warn with logger.warn
Christopher Usher 7d85eb7272 warn about and ignore files that don't parse as segments
Mike Lang 3a9543a4b5 Suppress less ffmpeg output when cutting 
The "fatal" level was causing some useful errors to be suppressed.
Mike Lang 3fbbe59b00 cutter Local backend: Fix typo and file extension when full cutting
Mike Lang b0a71fd9f1 cutter: Fix mistake in check_candidate error handling
Mike Lang ba53172cbc cutter: Get video link from backend instead of hard-coding youtube
Mike Lang 12decf015e Fix multiple typos and mistakes with full cuts