henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,259 Commits
34 Branches
0 Tags
19 MiB
thrimbletrimmer-solid
HubbeKing-multiplatform-builds
bus-progress-ui
master
bus-post-processing
chrusher/bus_synthesizer
mike/faster-chat-merge
mike/chat-no-end-time
chrusher-playlist-thumbnails
chrusher-thumbnail-frontend
mike/prizebot
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'fd78ff288e'
${ noResults }
Commit Graph

1259 Commits (fd78ff288e3b8d7fb1bd8db96c4df28c3f9b9e8a)
 

Author SHA1 Message Date
Mike Lang fccec1ace0 downloader: Fix a bug where hashes are miscalculated 
due to not including the map data in the hash calculation.

This is only relevant for streams with map data, which does not include twitch or youtube URLs.
 1 year ago
Mike Lang 590e056f7a docker-compose: Add support for backfill only, chat only channels 
And don't try to archive twitch chat for non-twitch channels
 1 year ago
Mike Lang 3e836b935b grafana: Disable analytics 1 year ago
Mike Lang 3df15b5784 grafana: Make it HTTPS 
Note this introduces two files the user must provide.
We set up the makefile to fail if they don't exist.
 1 year ago
Mike Lang 1e6b8e576e grafana: Use config file instead of env vars 1 year ago
Mike Lang 5f42b8419e downloader: Hard-code source quality for non-twitch providers 
since they can't do any other quality, but we still want to be able to set other qualities
for twitch streams.

Really qualities should be per-channel but I'm being lazy.
 1 year ago
Mike Lang 304e7b0fd9 downloader: Install yt-dlp in docker container 
for youtube support
 1 year ago
Mike Lang 0192e5246a docker-compose: Add support for new channel format 1 year ago
Mike Lang 18b71e6f5f downloader: Add youtube support 
Adds very simple youtube stream support where we only ever use the "best" quality,
which we call "source" for consistency with twitch.

We use yt-dlp to do the heavy lifting of getting the playlist url out of youtube.
 1 year ago
Mike Lang 05349a62df downloader: Prepend any HLS "map" data to every segment 
In some formats, most notably DASH, there is a "initialization data" that is required
in order to play the segment. The data is common to all segments so it is served as a seperate URL
under EXT-X-MAP. However, redundant copies of this data are benign and it's very small, so
we just put it in front of EVERY segment so that we can play every one independently (but
concatenating them still works).

We use a very simple cache to avoid downloading it again for every segment.
 1 year ago
Mike Lang e4252544be downloader: Add ability to download from an arbitrary playlist URL 
This is a very limited feature, it can't handle multiple qualities.
You must use "source" only and it takes the first available rendition.
 1 year ago
Mike Lang 7a75442243 downloader: rename twitch.py to providers.py 
Since it will now contain non-twitch providers also.
 1 year ago
Mike Lang bc08d97e56 downloader: Add framework to allow alternate "providers" besides twitch 
This abstracts out the process of obtaining media playlists so that we can support non-twitch
streaming services.
 1 year ago
Mike Lang 30c1877b36 Fix bugs with chat_archiver 
* Order of arguments matters - option can't be between other positional args and *args
* Messed up the count structure
 1 year ago
Mike Lang 66603bf648 docker-compose: Update some defaults 
These more closely match our actual run usage
 1 year ago
Mike Lang c3d5405ebc docker-compose: archive sheet support 1 year ago
Mike Lang b93597c274 sheetsync: Support archive sheet 1 year ago
Mike Lang 8d984110ab thrimshim: Special handling for "archive" events 1 year ago
Mike Lang 4d9d444cc3 cuter: local upload backends: Use _ instead of - as replacement char 
This is apparently nicer to deal with in URLs.
 1 year ago
Mike Lang 80c9be0baf cutter: Get archive cut working 1 year ago
Mike Lang 4c70878a53 cutter: Fix bug when no thumbnail set 1 year ago
Mike Lang 0164f9def8 schedulebot: Don't set groups before the run starts 
This is mainly a problem with groups_by_shift which will be set
even for negative hours that aren't on the schedule.

This causes notification spam as people move in and out of groups.
 1 year ago
Mike Lang 5c7061efd2 docker-compose: Have chat-archiver archive all watched channels 1 year ago
Mike Lang 260293d40d chat_archiver: Allow passing multiple channels on CLI 1 year ago
Mike Lang 91910c0972 chat_archiver: Fix misconfiguration where MAX_SERVER_LAG < MAX_DELAY 
This leads to delayed JOIN/PARTs not being put in their proper batch
as it's already been closed. In fact, since each message is re-opening a batch from
more than MAX_SERVER_LAG seconds ago, each message becomes one batch.
 1 year ago
Mike Lang 200d2df9ba chat_archiver: Add code support for archiving multiple channels at once 
* Join every channel on connect
* Move the "wait for initial ROOMSTATE" logic into the main loop and make it per-channel
* Make batch keys (channel, time) instead of just time

For now the CLI doesn't actually allow you to run chat_archiver in this mode,
it always calls the implementation with a 1-element list of channels.
 1 year ago
Mike Lang b050b71036 chat_archiver: Improve logging and monitoring by using unique client name more 1 year ago
Mike Lang 879fe2c70b thrimbletrimmer: Change default download type to smart 
The new smart cut type avoids artifacting/playing issues that happen with fast cuts or multi-range rough cuts.

However it is new and experimental so we keep the old options available as backups.
 1 year ago
Mike Lang 78d0f227e8 backfiller: Include channel name being backfilled in logs 1 year ago
Mike Lang 4e51c3d6b7 backfiller: Update node list from database more often 
5min is a long time when we want to pull a node from circulation,
and it's a very lightweight operation. So let's say 30 seconds.
 1 year ago
Mike Lang 51a2ef35cc schedulebot: Extend group-by-shift to allow multiple group members 1 year ago
ElementalAlchemist 8382e90659 Highlight the chapter description field as an error if it contains angle brackets 1 year ago
ElementalAlchemist f5c171f5ca Fix formatting 1 year ago
Mike Lang cbf3269001 thrimshim: Update default template names 1 year ago
Mike Lang d13a014bab schedulebot: Accept start timestamp with a trailing Z 1 year ago
Mike Lang 044dfb8084 Pin argh to avoid stupid breaking changes 1 year ago
Mike Lang 5a8bf99d12 schedulebot: Add ability to hard-code a group membership by shift 
This feature is intended for implementing a "current wub wrangler" group, which:
a) Isn't on the schedule
b) Can't be put on the schedule as it would overlap with other duties
c) We don't want to announce in "starting/ending shift" messages.
 1 year ago
Mike Lang 91d99bb22b Also build zulip_bots in CI 1 year ago
Mike Lang e146ed97db docker-compose: Add ability to give downloader an auth token 1 year ago
Mike Lang d636817b36 downloader: Add optional ability to authenticate when getting master playlist 
Authenticating to a particular twitch account can give benefits, most notably not being served ads.
 1 year ago
Mike Lang cedebff1ce Add LocalArchive upload backend 1 year ago
Mike Lang 5e7904dab3 wip: archive cut 1 year ago
Mike Lang 3ea0532838 wip: 1 year ago
Mike Lang c0e5f32459 Fix bad normalize function for fast_cut_range 1 year ago
Mike Lang ad10ad6ebf cutter: Remove no_transcode_check option 
It's misleading and never useful, as the modern TranscodeChecker only checks
for videos with its exact upload location.
 1 year ago
Mike Lang a201a2d4c9 cutter: Add support for smart cuts 1 year ago
Mike Lang dce0b3a5fc Update docker-compose defaults 
with values that are common (not for a particular year or node) but aren't secrets.
 1 year ago
Mike Lang ab3a89e6a6 Pin grafana version 
This is required due to grafana changes invalidating our old dashboards.
We don't have time to fix them right now.
 1 year ago
Mike Lang f324ef23cf backfiller: Fix critical issues with backfilling extras 1 year ago
Mike Lang fff01876a9 docker-compose: Backfill emotes directory 1 year ago