henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,144 Commits
34 Branches
0 Tags
19 MiB
thrimbletrimmer-solid
HubbeKing-multiplatform-builds
bus-progress-ui
master
bus-post-processing
chrusher/bus_synthesizer
mike/faster-chat-merge
mike/chat-no-end-time
chrusher-playlist-thumbnails
chrusher-thumbnail-frontend
mike/prizebot
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'a34af372d0'
${ noResults }
Commit Graph

111 Commits (a34af372d00dee93951ffaf5944d99452d8cb324)

Author SHA1 Message Date
Mike Lang 3606fadaa8 Pin gevent version to work around build issues 
Seeing the following error on latest versions of gevent:

 Traceback (most recent call last):
   File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
     return _run_code(code, main_globals, None,
   File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
     exec(code, run_globals)
   File "/usr/lib/python3.9/site-packages/zulip_bots/schedulebot.py", line 2, in <module>
     import gevent.monkey
   File "/usr/lib/python3.9/site-packages/gevent/__init__.py", line 72, in <module>
     from gevent._hub_local import get_hub
   File "/usr/lib/python3.9/site-packages/gevent/_hub_local.py", line 150, in <module>
     import_c_accel(globals(), 'gevent.__hub_local')
   File "/usr/lib/python3.9/site-packages/gevent/_util.py", line 148, in import_c_accel
     mod = importlib.import_module(cname)
   File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
     return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'gevent._gevent_c_hub_local'
 1 year ago
Mike Lang 0ed808688e cutter: Handle case where existing video has no last written thumb 1 year ago
Mike Lang 4d962ba187 Fix typo in cutter 1 year ago
Mike Lang e689626815 Add a small time range around the timestamp when extracting a frame 
This should hopefully result in frames on the edge of timestamps being extracted
from a combination of the neighboring segment and the naive one,
so that we don't get errors extracting a frame.
 1 year ago
Mike Lang 78c053000e Upgrade pip in order to make wheels work 1 year ago
Mike Lang 4d9d444cc3 cuter: local upload backends: Use _ instead of - as replacement char 
This is apparently nicer to deal with in URLs.
 1 year ago
Mike Lang 80c9be0baf cutter: Get archive cut working 1 year ago
Mike Lang 4c70878a53 cutter: Fix bug when no thumbnail set 1 year ago
Mike Lang 044dfb8084 Pin argh to avoid stupid breaking changes 1 year ago
Mike Lang cedebff1ce Add LocalArchive upload backend 1 year ago
Mike Lang 5e7904dab3 wip: archive cut 1 year ago
Mike Lang 3ea0532838 wip: 1 year ago
Mike Lang ad10ad6ebf cutter: Remove no_transcode_check option 
It's misleading and never useful, as the modern TranscodeChecker only checks
for videos with its exact upload location.
 1 year ago
Mike Lang a201a2d4c9 cutter: Add support for smart cuts 1 year ago
Mike Lang 30d5ccc483 Fix all old references to github.com/ekimekim/wubloader 1 year ago
Mike Lang 39b48c8b3a Don't attempt to upload a video with a >2MB thumbnail 2 years ago
Mike Lang b155c53b74 cutter: Correctly set video id and link on failed thumbnail upload 2 years ago
Mike Lang 2fee3a6e9d cutter: Don't set error on concurrent video update 
If two cutters modify a video at the same time, youtube may respond with a 409 to one of them.
We want to treat that as a retryable error, and ideally let another cutter finish it instead.
 2 years ago
Mike Lang 8b8b2a347e Fix typo in respecting upload location 2 years ago
Mike Lang 7fc9d3d2df cutter: Transcode and Update checkers should only affect videos for that upload location 2 years ago
Mike Lang ab985cf1b0 Fix syntax error 2 years ago
Mike Lang cfe93e31dd Fix a bug where modified videos are missing static tags 2 years ago
Mike Lang bd519369bd thumbnail fixes 2 years ago
Mike Lang 31c3f0b8b1 OS-level dependencies for Pillow 2 years ago
Mike Lang 45c46df8bb Add thumbnail templating code 2 years ago
Mike Lang d3e21ae9b0 Implement thumbnails in cutter 2 years ago
Mike Lang 46468409cb Add "public" column to control if videos are unlisted 
This column is modifyable.
This replaces the old "hidden" argument to youtube upload backends.
 2 years ago
Mike Lang c4a1d72240 Add new state to postgres and implement cutter logic 2 years ago
Mike Lang 101468cbca Fix a bug when reporting errors claiming candidates 
The SQL args were the wrong way around, so we tried to set error to the id.
Thankfully this failed as the id queried is not a valid uuid.

Fix by making args named like every other usage of query()
 3 years ago
Mike Lang 5e60344522 cutter: Fix errors due to unsupported cut type not being handled correctly 
Previously, determining the correct cut type and setting up the cut iterator
wasn't an operation we expected to be able to fail.
It happened outside all error handling blocks.

However, due to the multi-range work, this can now fail if we are requesting
an unsupported combination of transitions and cut type.

In order to correctly handle this like other cut errors, ie. set to UNEDITED
and set error column, we move this logic into the upload_wrapper right before we iterate
through the resulting cut.
 3 years ago
Mike Lang a47c29fff4 Link images to github repo by adding a LABEL 
When pushed, this tells github to associate the ghcr.io repo that was pushed to
with the github repo specified (the owner needs to match).

This does a few things.
Most importantly, this automatically gives github actions credentials to push to these
repositories when run in the context of the wubloader repo.
 3 years ago
Mike Lang aab8cf2f0f Set up plumbing for multi-range videos and implement no-transition fast cut videos only 
This is the simplest case as we can just cut each range like we already do,
then concat the results.

We still allow for the full design in the database and cutter, but error out if transitions
is ever anything but hard cuts or if it's a full cut.

We also update the restreamer to allow accepting ranges, however for usability we still allow
the old "just one start and end" args.

Note this changes the thrimshim API to give and take the new "video_ranges" and "video_transitions" columns.
 3 years ago
Mike Lang 62bd6539ea Unpin gevent as that was a workaround for a py2 issue 3 years ago
Mike Lang 21856c68aa Fix all instances of file.write() for py3 
In python 3, file.write() may do a partial write and returns the number of characters written.
In order to not lose data, we need to wrap every instance of file.write() with our new
common.writeall() wrapper that loops until the data is actually written.
 3 years ago
Mike Lang a56f6859bb more py3 fixes 3 years ago
Mike Lang f2a8007bf7 Fix build dependency issues 3 years ago
Mike Lang 50231a5154 py3 fixes for cutter 3 years ago
HubbeKing 6d790a1b36 Do a first naive pass for py3 compatibility 
Check that open() calls for reading and writing use binary modes
Use alpine version with py3-pip package
Use python3 in Dockerfile CMD
Remove sys.setdefaultencoding() "hack"
Simplify ensure_directory() in common.common package
 3 years ago
Mike Lang f0546e2ee3 Pin gevent to 1.5a2 to avoid https://github.com/gevent/gevent/issues/1711 3 years ago
Mike Lang b53fcd65a0 Add dependencies required to install psycopg2 from source 
We can't install the binaries as they don't support musl
 4 years ago
HubbeKing 86f7823348 Replace calls to gevent.signal() with gevent.signal_handler() 
gevent.signal() was removed in gevent 1.5a4, see http://www.gevent.org/api/gevent.signal.html
Removed on Feb 5th, see https://github.com/gevent/gevent/pull/1530
 4 years ago
Mike Lang a53786dc2d Add file and make as build dependencies 
gevent now requires these to build. I'm not sure when this changed.
 4 years ago
Mike Lang b9cd76b1a2 Add non-static implict tags in sheetsync 
In order for the upcoming playlist manager to be able to use the DB `tags` column to know
what tags a video has, all the tags it needs need to be present.

Previously, this was a problem because the day and category tags only get added at the cutter
and so wouldn't be listed.

This moves them so they are added when parsing the row in sheetsync.
It also adds the poster moment tag if poster moment is checked.

Note that fully static tags that go on all videos are still only added in cutter,
but the playlist manager doesn't need to care about those (since by definition
they will match every video).
 4 years ago
Mike Lang a30d595fee cutter: Use video's custom tags when uploading 4 years ago
Mike Lang 4a2fe7a6ed cutter: Explicitly set mime type of uploads correctly 5 years ago
Mike Lang fde2758275 cutter: Fix bug where uploader was cleared on non-retryable error 
Instead of on retryable error
 5 years ago
Mike Lang a002619c4c youtube upload: Explicitly set mime type 5 years ago
Mike Lang a438d86f80 cutter: Fix multiple problems with logging errors 5 years ago
Mike Lang 4bbcc8bc06 Revert "Merge pull request #155 from ekimekim/mike/manual-uploads" 
This reverts commit 99de586353, reversing
changes made to 4b04f70b6f.

We don't need this feature and it complicates things and adds bugs.
 5 years ago
Mike Lang c580671da2 Create upload pending state 5 years ago