henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,051 Commits
24 Branches
0 Tags
15 MiB
master
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '18b71e6f5f'
${ noResults }
Commit Graph

1051 Commits (18b71e6f5f2dafb034890ab65d7b0c1184099173)
 

Author SHA1 Message Date
Mike Lang 18b71e6f5f downloader: Add youtube support 
Adds very simple youtube stream support where we only ever use the "best" quality,
which we call "source" for consistency with twitch.

We use yt-dlp to do the heavy lifting of getting the playlist url out of youtube.
 11 months ago
Mike Lang 05349a62df downloader: Prepend any HLS "map" data to every segment 
In some formats, most notably DASH, there is a "initialization data" that is required
in order to play the segment. The data is common to all segments so it is served as a seperate URL
under EXT-X-MAP. However, redundant copies of this data are benign and it's very small, so
we just put it in front of EVERY segment so that we can play every one independently (but
concatenating them still works).

We use a very simple cache to avoid downloading it again for every segment.
 11 months ago
Mike Lang e4252544be downloader: Add ability to download from an arbitrary playlist URL 
This is a very limited feature, it can't handle multiple qualities.
You must use "source" only and it takes the first available rendition.
 11 months ago
Mike Lang 7a75442243 downloader: rename twitch.py to providers.py 
Since it will now contain non-twitch providers also.
 11 months ago
Mike Lang bc08d97e56 downloader: Add framework to allow alternate "providers" besides twitch 
This abstracts out the process of obtaining media playlists so that we can support non-twitch
streaming services.
 11 months ago
Mike Lang 30c1877b36 Fix bugs with chat_archiver 
* Order of arguments matters - option can't be between other positional args and *args
* Messed up the count structure
 11 months ago
Mike Lang 66603bf648 docker-compose: Update some defaults 
These more closely match our actual run usage
 11 months ago
Mike Lang c3d5405ebc docker-compose: archive sheet support 11 months ago
Mike Lang b93597c274 sheetsync: Support archive sheet 11 months ago
Mike Lang 8d984110ab thrimshim: Special handling for "archive" events 11 months ago
Mike Lang 4d9d444cc3 cuter: local upload backends: Use _ instead of - as replacement char 
This is apparently nicer to deal with in URLs.
 11 months ago
Mike Lang 80c9be0baf cutter: Get archive cut working 11 months ago
Mike Lang 4c70878a53 cutter: Fix bug when no thumbnail set 11 months ago
Mike Lang 0164f9def8 schedulebot: Don't set groups before the run starts 
This is mainly a problem with groups_by_shift which will be set
even for negative hours that aren't on the schedule.

This causes notification spam as people move in and out of groups.
 11 months ago
Mike Lang 5c7061efd2 docker-compose: Have chat-archiver archive all watched channels 11 months ago
Mike Lang 260293d40d chat_archiver: Allow passing multiple channels on CLI 11 months ago
Mike Lang 91910c0972 chat_archiver: Fix misconfiguration where MAX_SERVER_LAG < MAX_DELAY 
This leads to delayed JOIN/PARTs not being put in their proper batch
as it's already been closed. In fact, since each message is re-opening a batch from
more than MAX_SERVER_LAG seconds ago, each message becomes one batch.
 11 months ago
Mike Lang 200d2df9ba chat_archiver: Add code support for archiving multiple channels at once 
* Join every channel on connect
* Move the "wait for initial ROOMSTATE" logic into the main loop and make it per-channel
* Make batch keys (channel, time) instead of just time

For now the CLI doesn't actually allow you to run chat_archiver in this mode,
it always calls the implementation with a 1-element list of channels.
 11 months ago
Mike Lang b050b71036 chat_archiver: Improve logging and monitoring by using unique client name more 11 months ago
Mike Lang 879fe2c70b thrimbletrimmer: Change default download type to smart 
The new smart cut type avoids artifacting/playing issues that happen with fast cuts or multi-range rough cuts.

However it is new and experimental so we keep the old options available as backups.
 11 months ago
Mike Lang 78d0f227e8 backfiller: Include channel name being backfilled in logs 11 months ago
Mike Lang 4e51c3d6b7 backfiller: Update node list from database more often 
5min is a long time when we want to pull a node from circulation,
and it's a very lightweight operation. So let's say 30 seconds.
 11 months ago
Mike Lang 51a2ef35cc schedulebot: Extend group-by-shift to allow multiple group members 11 months ago
ElementalAlchemist 8382e90659 Highlight the chapter description field as an error if it contains angle brackets 11 months ago
ElementalAlchemist f5c171f5ca Fix formatting 11 months ago
Mike Lang cbf3269001 thrimshim: Update default template names 11 months ago
Mike Lang d13a014bab schedulebot: Accept start timestamp with a trailing Z 11 months ago
Mike Lang 044dfb8084 Pin argh to avoid stupid breaking changes 11 months ago
Mike Lang 5a8bf99d12 schedulebot: Add ability to hard-code a group membership by shift 
This feature is intended for implementing a "current wub wrangler" group, which:
a) Isn't on the schedule
b) Can't be put on the schedule as it would overlap with other duties
c) We don't want to announce in "starting/ending shift" messages.
 11 months ago
Mike Lang 91d99bb22b Also build zulip_bots in CI 11 months ago
Mike Lang e146ed97db docker-compose: Add ability to give downloader an auth token 11 months ago
Mike Lang d636817b36 downloader: Add optional ability to authenticate when getting master playlist 
Authenticating to a particular twitch account can give benefits, most notably not being served ads.
 11 months ago
Mike Lang cedebff1ce Add LocalArchive upload backend 11 months ago
Mike Lang 5e7904dab3 wip: archive cut 11 months ago
Mike Lang 3ea0532838 wip: 11 months ago
Mike Lang c0e5f32459 Fix bad normalize function for fast_cut_range 11 months ago
Mike Lang ad10ad6ebf cutter: Remove no_transcode_check option 
It's misleading and never useful, as the modern TranscodeChecker only checks
for videos with its exact upload location.
 11 months ago
Mike Lang a201a2d4c9 cutter: Add support for smart cuts 11 months ago
Mike Lang dce0b3a5fc Update docker-compose defaults 
with values that are common (not for a particular year or node) but aren't secrets.
 11 months ago
Mike Lang ab3a89e6a6 Pin grafana version 
This is required due to grafana changes invalidating our old dashboards.
We don't have time to fix them right now.
 11 months ago
Mike Lang f324ef23cf backfiller: Fix critical issues with backfilling extras 11 months ago
Mike Lang fff01876a9 docker-compose: Backfill emotes directory 12 months ago
Mike Lang 9f523c65cd backfiller: Allow backfilling of non-segment directories 
These are referred to as "extras" and all files in all subdirectories are backfilled
if not present.
 12 months ago
Mike Lang 4fb039d7d9 restreamer: Expose an endpoint for listing files that aren't segments 12 months ago
Mike Lang 76c9208be5 Move chat_archiver atomic_write() to common for re-use 12 months ago
Mike Lang 8220c87258 Fix zulip_bots bugs to get them working in package 1 year ago
Mike Lang 491945cc24 docker-compose: Run zulip bots 1 year ago
Mike Lang de521c4da9 schedulebot: Take timestamp string instead of epoch 1 year ago
Mike Lang 627eda5e74 schedulebot: Remove memes 1 year ago
Mike Lang 05d5a190fc Allow conf_file to be the JSON data directly 
Easier for containers
 1 year ago