henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,558 Commits
34 Branches
0 Tags
19 MiB
thrimbletrimmer-solid
HubbeKing-multiplatform-builds
bus-progress-ui
master
bus-post-processing
chrusher/bus_synthesizer
mike/faster-chat-merge
mike/chat-no-end-time
chrusher-playlist-thumbnails
chrusher-thumbnail-frontend
mike/prizebot
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'master'
${ noResults }
Commit Graph

152 Commits (master)

Author SHA1 Message Date
Mike Lang 8d984110ab thrimshim: Special handling for "archive" events 1 year ago
Mike Lang cbf3269001 thrimshim: Update default template names 1 year ago
Mike Lang 044dfb8084 Pin argh to avoid stupid breaking changes 1 year ago
Mike Lang 30d5ccc483 Fix all old references to github.com/ekimekim/wubloader 1 year ago
Mike Lang c36976dcca Default to rdp template for RDP thumbnails 2 years ago
ElementalAlchemist fafa2fa125 Add missing bracket to playlist strip regex 2 years ago
Mike Lang 6a713993af fix another typo 2 years ago
Mike Lang 8fbbb3f0a1 thrimshim: Fix description footer formatting 
Needs newlines before playlists section, and a line between playlist section and footer.
 2 years ago
Mike Lang 85c4319f2a thrimshim: Fix bug where description playlists header was added with zero playlists 
We only want it to show at all if there's at least one playlist.
 2 years ago
Mike Lang a298087137 playlists in description: round trip the description correctly 2 years ago
Mike Lang 17c5e86250 wip: 2 years ago
Mike Lang 34a33fdeb6 partially implement playlist links in video descriptions 
We make them conceptually "part of the footer" so they're updated only when the video
is otherwise updated (which would generally mean MODIFIED).
 2 years ago
Mike Lang 46ad45bdb9 thrimshim: Don't 500 if MODIFIED video doesn't contain some fields 
Missing fields _should_ be interpreted as "don't update these" but we weren't doing that.
 2 years ago
Mike Lang 57d89b9b39 Fix missing imports 2 years ago
Mike Lang 05816ac51d fix a bug where shift is calculated wrong 
we need to use the hour in PST, which we just hard-code as 8 hours.
If we ever hit DST before the run, we'll go back and do it properly.
 2 years ago
Mike Lang bd519369bd thumbnail fixes 2 years ago
Mike Lang 24c066e3e8 Add thumbnail support to thrimshim 
All thumbnail columns are modifiable, so you can eg. switch from
a templated thumbnail to a custom one, or change the time and template.
 2 years ago
Mike Lang bd948235fb thrimbletrimmer: Add advanced submission option for "public" videos 
Note that the default is true, so we only expand the advanced pane if it's false.
False corresponds to uploading a video as "unlisted".

Also mark it as modifyable.
 2 years ago
Mike Lang f06be1f391 thrimshim changes for modified 
would be ideal to check all args to see if any don't match (and refuse to modify in that case)
but eh too much work to properly normalise.
 2 years ago
Mike Lang 7649a4e840 Improve WSGIServer graceful shutdown handling 
Previously both restreamer and thrimshim had some complex logic for dealing with
graceful shutdown, in different ways, that was still prone to race conditions.

We replace this with a common method that does it properly.

Fixes #226
 3 years ago
Mike Lang a47c29fff4 Link images to github repo by adding a LABEL 
When pushed, this tells github to associate the ghcr.io repo that was pushed to
with the github repo specified (the owner needs to match).

This does a few things.
Most importantly, this automatically gives github actions credentials to push to these
repositories when run in the context of the wubloader repo.
 3 years ago
Mike Lang 192a0869f7 thrimshim: Fix bug in serializing columns 
Converting all datetime values to string no longer works because video_range
isn't a datetime but a list of datetimes. We switch to a more robust approach
of using the json "default" arg to specify how to serialize.
 3 years ago
Mike Lang aab8cf2f0f Set up plumbing for multi-range videos and implement no-transition fast cut videos only 
This is the simplest case as we can just cut each range like we already do,
then concat the results.

We still allow for the full design in the database and cutter, but error out if transitions
is ever anything but hard cuts or if it's a full cut.

We also update the restreamer to allow accepting ranges, however for usability we still allow
the old "just one start and end" args.

Note this changes the thrimshim API to give and take the new "video_ranges" and "video_transitions" columns.
 3 years ago
Mike Lang 2865088692 thrimshim: Further validation of titles/descriptions 
Youtube doesn't allow < or > characters, or extremely long descriptions.

Fixes #215, #219
 3 years ago
Mike Lang 62bd6539ea Unpin gevent as that was a workaround for a py2 issue 3 years ago
Mike Lang f2a8007bf7 Fix build dependency issues 3 years ago
Mike Lang d42d850153 thrimshim: misc cleanup 3 years ago
HubbeKing 6d790a1b36 Do a first naive pass for py3 compatibility 
Check that open() calls for reading and writing use binary modes
Use alpine version with py3-pip package
Use python3 in Dockerfile CMD
Remove sys.setdefaultencoding() "hack"
Simplify ensure_directory() in common.common package
 3 years ago
Mike Lang f0546e2ee3 Pin gevent to 1.5a2 to avoid https://github.com/gevent/gevent/issues/1711 3 years ago
Mike Lang f8c877775d Add an alternate manual upload mode specifically for youtube 
Adds a built-in "youtube-manual" location which is like "manual" except that it only works
with youtube URLs and populates the video_id column.

The intent is so that we can have playlist_manager manage videos we upload manually,
while still being able to distinguish between that and other manual links that shouldn't
be included (eg. links to third party youtube videos).

This is set when setting a manual link in thrimbletrimmer with a new checkbox, default off.
 4 years ago
Mike Lang 6d789ad42f thrimshim: Handle None values in comparing columns 4 years ago
Mike Lang cb75953e91 thrimshim: Fix a bug preventing submissions 
When comparing old and new video tags, it errors because it's a list, not string.

We change it to apply the transforms to all tags in the list, and also ignore changes in list ordering.
 4 years ago
Christopher Usher 41d85b8614 Remove unicode 4 years ago
Christopher Usher 84f7ef71c1 More helpful UI if the database has changed plus ignore changes that are 
just white space or capitalisation
 4 years ago
Mike Lang 48ef416dfb restreamer, thrimshim: Allow /metrics/* in addition to /metrics 
This allows the metrics proxying from nginx to work even if the path is not rewritten,
which is hard to do in k8s.
 4 years ago
Christopher Usher c19dce6e4e I forgot to add tags 4 years ago
Christopher Usher 27c6b70976 fixes in response to review 4 years ago
Christopher Usher b21a520d3a Checking for changes in the sheet now working in the thrimbletrimmer 4 years ago
Christopher Usher 1216c8e8a5 thrimshim side of checking for changes seems to be working 4 years ago
Christopher Usher 8889648a58 Check whether sheet columns have changed during editing in thrimshim. 4 years ago
Mike Lang b53fcd65a0 Add dependencies required to install psycopg2 from source 
We can't install the binaries as they don't support musl
 4 years ago
HubbeKing 86f7823348 Replace calls to gevent.signal() with gevent.signal_handler() 
gevent.signal() was removed in gevent 1.5a4, see http://www.gevent.org/api/gevent.signal.html
Removed on Feb 5th, see https://github.com/gevent/gevent/pull/1530
 4 years ago
Mike Lang a53786dc2d Add file and make as build dependencies 
gevent now requires these to build. I'm not sure when this changed.
 4 years ago
Mike Lang 7e0c415f83 thrimshim: Add video tags column as required 4 years ago
Mike Lang 4bbcc8bc06 Revert "Merge pull request #155 from ekimekim/mike/manual-uploads" 
This reverts commit 99de586353, reversing
changes made to 4b04f70b6f.

We don't need this feature and it complicates things and adds bugs.
 5 years ago
Mike Lang 9ccc7e4e8d thrimshim: Allow manual_link to set things from UPLOAD_PENDING to TRANSCODING 5 years ago
Mike Lang b2a07ef114
Merge pull request #140 from ekimekim/mike/build-improvements 
Refactor dockerfiles for more shared layers
 5 years ago
Mike Lang 731ef9e2d0 Refactor dockerfiles for more shared layers 
By carefully ensuring most of our dockerfiles are identical in their first few layers,
we only need to build those layers once instead of every time.

In particular, we move installing gevent to before installing common,
so that even when common changes gevent doesn't need to be reinstalled.

This is important because gevent takes ages to install.

Also fixes segment_coverage, which wasn't being installed.
 5 years ago
Mike Lang 3dfe0e8722 Allow thrimshim to safely cancel a job while it is cutting 
This differs from the existing reset row by only suceeding if the upload is not
in finalizing.

We also make some changes to cutter to handle this situation gracefully.
 5 years ago
Mike Lang a42e7b48f6 thrimshim: Allow degraded operation even if DB broken 
Any endpoints that don't need a DB conn will still work fine.

Notably, this includes /defaults, which is needed for
thrimbletrimmer to work in a non-specific-row mode.
 5 years ago
Mike Lang f7f07a2688 Add /defaults to thrimshim, use it to populate config stuff in thrimbletrimmer 
Thrimbletrimmer needs to know stuff like the bustime start time,
default channel, even if it's not looking at a specific row.
 5 years ago
Mike Lang da3cc24ed1 Pass a list of upload locations from thrimshim to thrimbletrimmer 
with the first one being the default.
 5 years ago
Mike Lang b11fe39371 thrimshim: Validate title length, non-empty title/description 
and pass the title prefix / max length through to thrimbletrimmer
 5 years ago
Mike Lang ba746ff6e6 Add title and description header/footer in thrimshim instead of cutter 
This accomplishes two things:
1. It allows thrimshim to properly validate length restrictions (not implemented yet)
2. It means that the database has a record of the values actually written for each of these rows,
instead of that information depending on how the cutter was configured at the time.
 5 years ago
mg 936a2387c8 Change to how default channel is handled. 5 years ago
mg ca11e59511 Adding channel/start time info to Thrimshim to pass to Thrimbletrimmer. 5 years ago
mg 6edd224395 Merging Thrimbletrimmer into Wubloader with latest changes. 5 years ago
Christopher Usher fec8cff185
Merge branch 'master' into chrusher-database-replication 5 years ago
Christopher Usher 027c2900e2 fixes in response to ekim's comments 5 years ago
Christopher Usher 92f6ed5891 function name fix 5 years ago
Christopher Usher 86477fae13 fixes for ekim's comments 5 years ago
Christopher Usher 611a4bd793 added option not to authenticate thrimshim 5 years ago
Christopher Usher 36fe22bf99 authentication implemented and tested as much as possible without authentication in thrimbletrimmer 5 years ago
Christopher Usher 057bd031a2 decorator to avoid duplication authentication code 5 years ago
Christopher Usher 0cf84ef510 code style clean up 5 years ago
mg 64824973ba Adding issuer validation. 5 years ago
mg 96d2566ab2 Adding dependencies and retrieving account info. 5 years ago
Christopher Usher aea0fe1766 Setting up Auth Test function 5 years ago
Christopher Usher d07cb5de44 Placeholder text 5 years ago
Christopher Usher 23e3cfce20 Added editor, edit_time and upload_time to thrimshim and cutter updates of the 
database
 5 years ago
Christopher Usher 1dbe585837 retry database connection if it fails 5 years ago
mg e49ec90d3e UI Improvements 5 years ago
Christopher Usher d55d6fb7bc added option not to authenticate thrimshim 5 years ago
Christopher Usher c8e44af20d authentication implemented and tested as much as possible without authentication in thrimbletrimmer 5 years ago
Christopher Usher 377a6697b0 decorator to avoid duplication authentication code 5 years ago
Christopher Usher 4779d88e37 code style clean up 5 years ago
mg f4d0fbf42e Adding issuer validation. 5 years ago
mg f3b1b991e5 Adding dependencies and retrieving account info. 5 years ago
Christopher Usher d578da90d0 Setting up Auth Test function 5 years ago
Christopher Usher 365d7b7852 Placeholder text 5 years ago
Christopher Usher ea42cab8fd Added editor, edit_time and upload_time to thrimshim and cutter updates of the 
database
 5 years ago
Christopher Usher 76bc629720 moved flask monitoring to its own module 5 years ago
Christopher Usher 73541f852f logging and monitoring for thrimshim 5 years ago
Christopher Usher 6858c2e2de starting on logging and monitoring 5 years ago
MasterGunner 2e953eddde Cleanup from Ekim's comments, removed auth placeholder until I know what I'm doing. 5 years ago
MasterGunner 6a171130e8 Updated Get All Rows route. 5 years ago
MasterGunner 736f0e0fe4 Adding get_all_row and auth function stubs 5 years ago
Christopher Usher 9b28765ff2 Bug fixes to get the database connection working 6 years ago
Mike Lang 33be5a92cb Update docker-compose file for thrimshim and cutter 
Also update thrimshim to take db connect args as positional,
for consistency with cutter.
 6 years ago
Christopher Usher afdad26a51 More fixes 6 years ago
Christopher Usher c3c75b1e4f manually setting a video link now also its own endpoint 6 years ago
Christopher Usher b7a57d4766 reset is now it's own method 6 years ago
Christopher Usher fe5b10f86b Fixes the state transitions of the trimbleshim to allow video links to be changed or removed 6 years ago
Christopher Usher f43c699e05 updated thrimshim to handle all non-null edit columns 6 years ago
Christopher Usher d23de10b3e a few small fixes to ekim's comments 6 years ago
Christopher Usher 1d09e28b1e fixes to ekimekim's suggestions 6 years ago
Christopher Usher c81d538a79 thrimshim seems to be working 6 years ago
Christopher Usher e4fc878577 logic of the post 6 years ago
Christopher Usher 4c5b6e4cda GET working 6 years ago
Christopher Usher 5faa70dfc2 getting thrimshim to build and run is a minor success 6 years ago