Mike Lang
1e7e400096
docker-compose: Pass correct database args for buscribe
1 month ago
Mike Lang
834d27c17d
Add buscribe to docker-compose
1 month ago
Mike Lang
9b19c12a53
Add pubnub-watching zulip bot
...
Which also records all pubnub messages for archival to a file SEGMENTS/pubnub-log.json.
We ran this for most of last year, but not as part of wubloader.
2 months ago
Mike Lang
7483794a23
docker-compose: Set up sheetsync for downloading media links
...
This involves giving it access to the SEGMENTS/media directory.
2 months ago
Mike Lang
15f86551d4
docker-compose: Backfill of media, with or without chat archiver
2 months ago
Mike Lang
9dfb00f4ab
chat_archiver: Logic for checking and downloading media links
2 months ago
Mike Lang
48725ac739
docker-compose: fix typos
3 months ago
Mike Lang
96181fd875
Support archive sync in sheetsync again
3 months ago
Mike Lang
73eaac16f3
Update sheetsync CLI interface to allow any combination of types and backends
3 months ago
Mike Lang
eebfa5885b
sheetsync: pass in event id instead of event name
3 months ago
Mike Lang
db3a779292
docker-compose: Allow setting up sheetsync for sheets or streamlog
3 months ago
Mike Lang
986a1db964
sheetsync: Change how options are specified to allow multiple backends / syncs
3 months ago
Mike Lang
9dec3cb729
Allow the youtube-manual upload backend to modify videos
...
So we can modify youtube videos we manually uploaded.
If the video isn't actually ours, it will error when the modify is attempted which is ok.
3 months ago
Christopher Usher
e66540087a
Fixes suggested by ekim
1 year ago
Christopher Usher
a6e990db68
Donation graph now works in a container
1 year ago
Mike Lang
ce665047ac
Add bus_analyzer to docker-compose
1 year ago
Mike Lang
590e056f7a
docker-compose: Add support for backfill only, chat only channels
...
And don't try to archive twitch chat for non-twitch channels
1 year ago
Mike Lang
0192e5246a
docker-compose: Add support for new channel format
1 year ago
Mike Lang
bc08d97e56
downloader: Add framework to allow alternate "providers" besides twitch
...
This abstracts out the process of obtaining media playlists so that we can support non-twitch
streaming services.
1 year ago
Mike Lang
30c1877b36
Fix bugs with chat_archiver
...
* Order of arguments matters - option can't be between other positional args and *args
* Messed up the count structure
1 year ago
Mike Lang
66603bf648
docker-compose: Update some defaults
...
These more closely match our actual run usage
1 year ago
Mike Lang
c3d5405ebc
docker-compose: archive sheet support
1 year ago
Mike Lang
5c7061efd2
docker-compose: Have chat-archiver archive all watched channels
1 year ago
Mike Lang
260293d40d
chat_archiver: Allow passing multiple channels on CLI
1 year ago
Mike Lang
5a8bf99d12
schedulebot: Add ability to hard-code a group membership by shift
...
This feature is intended for implementing a "current wub wrangler" group, which:
a) Isn't on the schedule
b) Can't be put on the schedule as it would overlap with other duties
c) We don't want to announce in "starting/ending shift" messages.
1 year ago
Mike Lang
e146ed97db
docker-compose: Add ability to give downloader an auth token
1 year ago
Mike Lang
dce0b3a5fc
Update docker-compose defaults
...
with values that are common (not for a particular year or node) but aren't secrets.
1 year ago
Mike Lang
fff01876a9
docker-compose: Backfill emotes directory
1 year ago
Mike Lang
491945cc24
docker-compose: Run zulip bots
1 year ago
Mike Lang
30d5ccc483
Fix all old references to github.com/ekimekim/wubloader
1 year ago
Mike Lang
8e314eea94
Collect metrics for chat_archiver on port 8008
2 years ago
Mike Lang
46468409cb
Add "public" column to control if videos are unlisted
...
This column is modifyable.
This replaces the old "hidden" argument to youtube upload backends.
2 years ago
Mike Lang
c50224415c
more backfiller chat fixes
...
fixup: more backfiller fixes
Enable backfilling of chat logs
2 years ago
Mike Lang
4cfc362f76
chat-archiver: pass in node name
...
instead of using container hostname
2 years ago
Mike Lang
315c9c8297
Integrate chat archiver as a proper component
2 years ago
Mike Lang
0756539b85
chat-archiver: Early work and basic archival
2 years ago
Mike Lang
467edf3d19
Read dynamic playlist manager config from sheet
...
The sheetsync loads playlist ids and tags into a new table `playlists`.
playlist manager reads this table and merges it with the playlists given on the command line.
3 years ago
ElementalAlchemist
2b4e7a23e7
Update the default edit_url to make sense with the new Thrimbletrimmer structure
3 years ago
ElementalAlchemist
e4204d9b9a
Add Thrimbletrimmer dev option
3 years ago
Mike Lang
415932e289
docker-compose: fix bug where database_tag isn't respected
3 years ago
Mike Lang
3e768aa2a9
switch from quay.io to ghcr.io for image registry
3 years ago
Mike Lang
5f81c383bc
Add day 8
3 years ago
Mike Lang
317d45f16d
Manually specify upload locations that should be added to playlists
...
This lets us exclude some youtube locations (like unlisted),
and include non-youtube ones (like manual-youtube)
3 years ago
Mike Lang
463843668d
segment-coverage: Render a monitoring page
3 years ago
Mike Lang
e9b2831b71
docker-compose: Fix bug preventing playlist manager metrics from being routed
4 years ago
Mike Lang
2327c7c9a8
docker-compose: Fix typos in configuring playlist manager
4 years ago
Mike Lang
6e5e4b41df
docker-compose/k8s: Limit segment-coverage start/end times
...
Instead of defaulting to oldest/newest.
4 years ago
Mike Lang
324d180c5e
docker-compose: Default to only backfilling things from this year
...
By setting a 6-month limit
4 years ago
Mike Lang
b029250c1c
Disable stacksampler by default
...
It causes problems due to the sheer number of unique metrics emitted, which makes
the prometheus endpoint be very expensive / fail a lot.
The data is not useful enough to justify the cost.
4 years ago
Mike Lang
a89bfcd271
Add playlist manager to all the various other places
4 years ago