henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,033 Commits
34 Branches
0 Tags
19 MiB
Commit Graph

25 Commits (b050b71036b1297c9d7aba15b2d7cac41d3904fa)

Author SHA1 Message Date
Mike Lang b050b71036 chat_archiver: Improve logging and monitoring by using unique client name more
Mike Lang 044dfb8084 Pin argh to avoid stupid breaking changes
Mike Lang 76c9208be5 Move chat_archiver atomic_write() to common for re-use
Mike Lang 30d5ccc483 Fix all old references to github.com/ekimekim/wubloader
Mike Lang ad4827237f Fix bug in checking if message has tags
Mike Lang 681da9a76e Fix a bug where we try to fetch emote "" for messages with no emotes
Mike Lang 3b6ce86c46 chat archiver: Add cli tool for downloading emotes
Mike Lang e74d655ce5 chat_archiver: Download each seen emote 
so we have a permanent record, in case they're deleted or changed later
Mike Lang 8e314eea94 Collect metrics for chat_archiver on port 8008
Mike Lang 08257386e2 Add restreamer endpoint for viewing chat messages
Mike Lang 9320251de7 Some extra documentation on chat_archiver
Mike Lang d8a9b5ddf0 chat_archiver: Always sort json object keys to ensure canonical output
Mike Lang 651658e507 JOINs and PARTs have been observed with up to 30sec difference 
it turns out to be completely undocumented what the max delay is. so let's assume 45s.
anything > 60s might cause problems due to matching messages being more than 1 batch apart.
Mike Lang 05a989f67d chat-archiver: fixes
Mike Lang c1c1c11bce chat_archiver: Add prometheus metrics
Mike Lang c25d79e7c2 chat-archiver: Merge all files every minute
Mike Lang 4cfc362f76 chat-archiver: pass in node name 
instead of using container hostname
Mike Lang a48beab576 chat-archiver: update girc for py3 support and fixes
Mike Lang 315c9c8297 Integrate chat archiver as a proper component
Mike Lang f8b3ace148 Backfill chat archives under the "chat" quality
Mike Lang 05ddd39504 chat_archiver: Split files into directories by hour 
matching how we handle video files
Mike Lang 1d626738bd chat_archiver: Start a new client on RECONNECT
Mike Lang 96cc212bf0 chat_archiver: fixes, implement merge_all
Mike Lang d32cbbb7e1 chat-archiver: File merging and other fixes
Mike Lang 0756539b85 chat-archiver: Early work and basic archival