Commit Graph

22 Commits (fa1603e99a89e042d7f6eee05a195599b7b2a1e0)

Author SHA1 Message Date
Mike Lang 30d5ccc483 Fix all old references to github.com/ekimekim/wubloader 1 year ago
Mike Lang ad4827237f Fix bug in checking if message has tags 2 years ago
Mike Lang 681da9a76e Fix a bug where we try to fetch emote "" for messages with no emotes 2 years ago
Mike Lang 3b6ce86c46 chat archiver: Add cli tool for downloading emotes 2 years ago
Mike Lang e74d655ce5 chat_archiver: Download each seen emote
so we have a permanent record, in case they're deleted or changed later
2 years ago
Mike Lang 8e314eea94 Collect metrics for chat_archiver on port 8008 2 years ago
Mike Lang 08257386e2 Add restreamer endpoint for viewing chat messages 2 years ago
Mike Lang 9320251de7 Some extra documentation on chat_archiver 2 years ago
Mike Lang d8a9b5ddf0 chat_archiver: Always sort json object keys to ensure canonical output 2 years ago
Mike Lang 651658e507 JOINs and PARTs have been observed with up to 30sec difference
it turns out to be completely undocumented what the max delay is. so let's assume 45s.
anything > 60s might cause problems due to matching messages being more than 1 batch apart.
2 years ago
Mike Lang 05a989f67d chat-archiver: fixes 2 years ago
Mike Lang c1c1c11bce chat_archiver: Add prometheus metrics 2 years ago
Mike Lang c25d79e7c2 chat-archiver: Merge all files every minute 2 years ago
Mike Lang 4cfc362f76 chat-archiver: pass in node name
instead of using container hostname
2 years ago
Mike Lang a48beab576 chat-archiver: update girc for py3 support and fixes 2 years ago
Mike Lang 315c9c8297 Integrate chat archiver as a proper component 2 years ago
Mike Lang f8b3ace148 Backfill chat archives under the "chat" quality 2 years ago
Mike Lang 05ddd39504 chat_archiver: Split files into directories by hour
matching how we handle video files
2 years ago
Mike Lang 1d626738bd chat_archiver: Start a new client on RECONNECT 2 years ago
Mike Lang 96cc212bf0 chat_archiver: fixes, implement merge_all 2 years ago
Mike Lang d32cbbb7e1 chat-archiver: File merging and other fixes 2 years ago
Mike Lang 0756539b85 chat-archiver: Early work and basic archival 2 years ago