henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,057 Commits
34 Branches
0 Tags
19 MiB
Commit Graph

1057 Commits (3e836b935be1996919aa09ced9542729e163f07b)
 

Author SHA1 Message Date
Mike Lang 3e836b935b grafana: Disable analytics
Mike Lang 3df15b5784 grafana: Make it HTTPS 
Note this introduces two files the user must provide.
We set up the makefile to fail if they don't exist.
Mike Lang 1e6b8e576e grafana: Use config file instead of env vars
Mike Lang 5f42b8419e downloader: Hard-code source quality for non-twitch providers 
since they can't do any other quality, but we still want to be able to set other qualities
for twitch streams.

Really qualities should be per-channel but I'm being lazy.
Mike Lang 304e7b0fd9 downloader: Install yt-dlp in docker container 
for youtube support
Mike Lang 0192e5246a docker-compose: Add support for new channel format
Mike Lang 18b71e6f5f downloader: Add youtube support 
Adds very simple youtube stream support where we only ever use the "best" quality,
which we call "source" for consistency with twitch.

We use yt-dlp to do the heavy lifting of getting the playlist url out of youtube.
Mike Lang 05349a62df downloader: Prepend any HLS "map" data to every segment 
In some formats, most notably DASH, there is a "initialization data" that is required
in order to play the segment. The data is common to all segments so it is served as a seperate URL
under EXT-X-MAP. However, redundant copies of this data are benign and it's very small, so
we just put it in front of EVERY segment so that we can play every one independently (but
concatenating them still works).

We use a very simple cache to avoid downloading it again for every segment.
Mike Lang e4252544be downloader: Add ability to download from an arbitrary playlist URL 
This is a very limited feature, it can't handle multiple qualities.
You must use "source" only and it takes the first available rendition.
Mike Lang 7a75442243 downloader: rename twitch.py to providers.py 
Since it will now contain non-twitch providers also.
Mike Lang bc08d97e56 downloader: Add framework to allow alternate "providers" besides twitch 
This abstracts out the process of obtaining media playlists so that we can support non-twitch
streaming services.
Mike Lang 30c1877b36 Fix bugs with chat_archiver 
* Order of arguments matters - option can't be between other positional args and *args
* Messed up the count structure
Mike Lang 66603bf648 docker-compose: Update some defaults 
These more closely match our actual run usage
Mike Lang c3d5405ebc docker-compose: archive sheet support
Mike Lang b93597c274 sheetsync: Support archive sheet
Mike Lang 8d984110ab thrimshim: Special handling for "archive" events
Mike Lang 4d9d444cc3 cuter: local upload backends: Use _ instead of - as replacement char 
This is apparently nicer to deal with in URLs.
Mike Lang 80c9be0baf cutter: Get archive cut working
Mike Lang 4c70878a53 cutter: Fix bug when no thumbnail set
Mike Lang 0164f9def8 schedulebot: Don't set groups before the run starts 
This is mainly a problem with groups_by_shift which will be set
even for negative hours that aren't on the schedule.

This causes notification spam as people move in and out of groups.
Mike Lang 5c7061efd2 docker-compose: Have chat-archiver archive all watched channels
Mike Lang 260293d40d chat_archiver: Allow passing multiple channels on CLI
Mike Lang 91910c0972 chat_archiver: Fix misconfiguration where MAX_SERVER_LAG < MAX_DELAY 
This leads to delayed JOIN/PARTs not being put in their proper batch
as it's already been closed. In fact, since each message is re-opening a batch from
more than MAX_SERVER_LAG seconds ago, each message becomes one batch.
Mike Lang 200d2df9ba chat_archiver: Add code support for archiving multiple channels at once 
* Join every channel on connect
* Move the "wait for initial ROOMSTATE" logic into the main loop and make it per-channel
* Make batch keys (channel, time) instead of just time

For now the CLI doesn't actually allow you to run chat_archiver in this mode,
it always calls the implementation with a 1-element list of channels.
Mike Lang b050b71036 chat_archiver: Improve logging and monitoring by using unique client name more
Mike Lang 879fe2c70b thrimbletrimmer: Change default download type to smart 
The new smart cut type avoids artifacting/playing issues that happen with fast cuts or multi-range rough cuts.

However it is new and experimental so we keep the old options available as backups.
Mike Lang 78d0f227e8 backfiller: Include channel name being backfilled in logs
Mike Lang 4e51c3d6b7 backfiller: Update node list from database more often 
5min is a long time when we want to pull a node from circulation,
and it's a very lightweight operation. So let's say 30 seconds.
Mike Lang 51a2ef35cc schedulebot: Extend group-by-shift to allow multiple group members
ElementalAlchemist 8382e90659 Highlight the chapter description field as an error if it contains angle brackets
ElementalAlchemist f5c171f5ca Fix formatting
Mike Lang cbf3269001 thrimshim: Update default template names
Mike Lang d13a014bab schedulebot: Accept start timestamp with a trailing Z
Mike Lang 044dfb8084 Pin argh to avoid stupid breaking changes
Mike Lang 5a8bf99d12 schedulebot: Add ability to hard-code a group membership by shift 
This feature is intended for implementing a "current wub wrangler" group, which:
a) Isn't on the schedule
b) Can't be put on the schedule as it would overlap with other duties
c) We don't want to announce in "starting/ending shift" messages.
Mike Lang 91d99bb22b Also build zulip_bots in CI
Mike Lang e146ed97db docker-compose: Add ability to give downloader an auth token
Mike Lang d636817b36 downloader: Add optional ability to authenticate when getting master playlist 
Authenticating to a particular twitch account can give benefits, most notably not being served ads.
Mike Lang cedebff1ce Add LocalArchive upload backend
Mike Lang 5e7904dab3 wip: archive cut
Mike Lang 3ea0532838 wip:
Mike Lang c0e5f32459 Fix bad normalize function for fast_cut_range
Mike Lang ad10ad6ebf cutter: Remove no_transcode_check option 
It's misleading and never useful, as the modern TranscodeChecker only checks
for videos with its exact upload location.
Mike Lang a201a2d4c9 cutter: Add support for smart cuts
Mike Lang dce0b3a5fc Update docker-compose defaults 
with values that are common (not for a particular year or node) but aren't secrets.
Mike Lang ab3a89e6a6 Pin grafana version 
This is required due to grafana changes invalidating our old dashboards.
We don't have time to fix them right now.
Mike Lang f324ef23cf backfiller: Fix critical issues with backfilling extras
Mike Lang fff01876a9 docker-compose: Backfill emotes directory
Mike Lang 9f523c65cd backfiller: Allow backfilling of non-segment directories 
These are referred to as "extras" and all files in all subdirectories are backfilled
if not present.
Mike Lang 4fb039d7d9 restreamer: Expose an endpoint for listing files that aren't segments