henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,553 Commits
34 Branches
0 Tags
19 MiB
Commit Graph

129 Commits (6062f0a8ec4093793c76af9f625af29e643a0f6b)

Author SHA1 Message Date
Mike Lang e794b0cf7d prizebot fixes
Mike Lang 3b1c837bfa Import prizebot
Mike Lang 7ca45e89b9 fix pubbot config
Mike Lang 69d3892101 pubbot: Get channel ids from config and handle prizes in the new website
Christopher Usher d6f976ac0f Bug fixes and improvements in response to comments on the PR
Christopher Usher 8d6d37e462 added shift definitions to docker-compose
Mike Lang fbbe37ca9e blogbot: Save blog contents by hash when seen 
For every unique hash of the blog html, it will save a copy.
Blogs are saved as JSON files in {segments_dir}/blogs
Mike Lang 18905a3c17 buscribe: Tell nginx to actually serve those pages
Mike Lang e91d6b65ac blogbot: add to docker-compose
Mike Lang b866b5d21c playlist manager: run more often
Mike Lang 9ad0f5f512 Pass db connect to restreamer
Mike Lang 48c91b7828 docker-compose: Set buscribe tag seperately 
It's big and slow and most of the time doesn't need updating.
Mike Lang 4cd0ef1327 schedulebot: Take creds as a file 
Avoids needing to put the creds in docker-compose.jsonnet.
The existing "import" trick didn't work due to the way we run jsonnet in docker.
Mike Lang 8533c01d7f docker-compose changes for passing schedule sheet to schedulebot
Mike Lang 0dfc8ea127 bus_analyzer: fix docker-compose
Mike Lang 0e229dc80b lower backfill_max_hours_ago so we aren't backfilling 6 months of chat logs
Mike Lang 1e7e400096 docker-compose: Pass correct database args for buscribe
Mike Lang 834d27c17d Add buscribe to docker-compose
Mike Lang 9b19c12a53 Add pubnub-watching zulip bot 
Which also records all pubnub messages for archival to a file SEGMENTS/pubnub-log.json.
We ran this for most of last year, but not as part of wubloader.
Mike Lang 7483794a23 docker-compose: Set up sheetsync for downloading media links 
This involves giving it access to the SEGMENTS/media directory.
Mike Lang 15f86551d4 docker-compose: Backfill of media, with or without chat archiver
Mike Lang 9dfb00f4ab chat_archiver: Logic for checking and downloading media links
Mike Lang 48725ac739 docker-compose: fix typos
Mike Lang 96181fd875 Support archive sync in sheetsync again
Mike Lang 73eaac16f3 Update sheetsync CLI interface to allow any combination of types and backends
Mike Lang eebfa5885b sheetsync: pass in event id instead of event name
Mike Lang db3a779292 docker-compose: Allow setting up sheetsync for sheets or streamlog
Mike Lang 986a1db964 sheetsync: Change how options are specified to allow multiple backends / syncs
Mike Lang 9dec3cb729 Allow the youtube-manual upload backend to modify videos 
So we can modify youtube videos we manually uploaded.
If the video isn't actually ours, it will error when the modify is attempted which is ok.
Christopher Usher e66540087a Fixes suggested by ekim
Christopher Usher a6e990db68 Donation graph now works in a container
Mike Lang ce665047ac Add bus_analyzer to docker-compose
Mike Lang 590e056f7a docker-compose: Add support for backfill only, chat only channels 
And don't try to archive twitch chat for non-twitch channels
Mike Lang 0192e5246a docker-compose: Add support for new channel format
Mike Lang bc08d97e56 downloader: Add framework to allow alternate "providers" besides twitch 
This abstracts out the process of obtaining media playlists so that we can support non-twitch
streaming services.
Mike Lang 30c1877b36 Fix bugs with chat_archiver 
* Order of arguments matters - option can't be between other positional args and *args
* Messed up the count structure
Mike Lang 66603bf648 docker-compose: Update some defaults 
These more closely match our actual run usage
Mike Lang c3d5405ebc docker-compose: archive sheet support
Mike Lang 5c7061efd2 docker-compose: Have chat-archiver archive all watched channels
Mike Lang 260293d40d chat_archiver: Allow passing multiple channels on CLI
Mike Lang 5a8bf99d12 schedulebot: Add ability to hard-code a group membership by shift 
This feature is intended for implementing a "current wub wrangler" group, which:
a) Isn't on the schedule
b) Can't be put on the schedule as it would overlap with other duties
c) We don't want to announce in "starting/ending shift" messages.
Mike Lang e146ed97db docker-compose: Add ability to give downloader an auth token
Mike Lang dce0b3a5fc Update docker-compose defaults 
with values that are common (not for a particular year or node) but aren't secrets.
Mike Lang fff01876a9 docker-compose: Backfill emotes directory
Mike Lang 491945cc24 docker-compose: Run zulip bots
Mike Lang 30d5ccc483 Fix all old references to github.com/ekimekim/wubloader
Mike Lang 8e314eea94 Collect metrics for chat_archiver on port 8008
Mike Lang 46468409cb Add "public" column to control if videos are unlisted 
This column is modifyable.
This replaces the old "hidden" argument to youtube upload backends.
Mike Lang c50224415c more backfiller chat fixes 
fixup: more backfiller fixes

Enable backfilling of chat logs
Mike Lang 4cfc362f76 chat-archiver: pass in node name 
instead of using container hostname