1024 Commits (044dfb80845f723de2f273b44d8e9464b11d3b55)

 

All Branches   

Search

Author	SHA1	Message	Date
Mike Lang	4f2c5c48ec	build chat_archiver in CI	2 years ago
Mike Lang	08257386e2	Add restreamer endpoint for viewing chat messages	2 years ago
Mike Lang	9320251de7	Some extra documentation on chat_archiver	2 years ago
Mike Lang	d8a9b5ddf0	chat_archiver: Always sort json object keys to ensure canonical output	2 years ago
Mike Lang	651658e507	JOINs and PARTs have been observed with up to 30sec difference ... it turns out to be completely undocumented what the max delay is. so let's assume 45s. anything > 60s might cause problems due to matching messages being more than 1 batch apart.	2 years ago
Mike Lang	c50224415c	more backfiller chat fixes ... fixup: more backfiller fixes Enable backfilling of chat logs	2 years ago
Mike Lang	05a989f67d	chat-archiver: fixes	2 years ago
Mike Lang	c1c1c11bce	chat_archiver: Add prometheus metrics	2 years ago
Mike Lang	c25d79e7c2	chat-archiver: Merge all files every minute	2 years ago
Mike Lang	4cfc362f76	chat-archiver: pass in node name ... instead of using container hostname	2 years ago
Mike Lang	a48beab576	chat-archiver: update girc for py3 support and fixes	2 years ago
Mike Lang	315c9c8297	Integrate chat archiver as a proper component	2 years ago
Mike Lang	f8b3ace148	Backfill chat archives under the "chat" quality	2 years ago
Mike Lang	05ddd39504	chat_archiver: Split files into directories by hour ... matching how we handle video files	2 years ago
Mike Lang	1d626738bd	chat_archiver: Start a new client on RECONNECT	2 years ago
Mike Lang	96cc212bf0	chat_archiver: fixes, implement merge_all	2 years ago
Mike Lang	d32cbbb7e1	chat-archiver: File merging and other fixes	2 years ago
Mike Lang	0756539b85	chat-archiver: Early work and basic archival	2 years ago
Mike Lang	6b9d8ab785	Fix submit changes button - should be hidden by default	2 years ago
Mike Lang	ac6612fb85	Add ability to submit video as MODIFIED to thrimbletrimmer	2 years ago
Mike Lang	f06be1f391	thrimshim changes for modified ... would be ideal to check all args to see if any don't match (and refuse to modify in that case) but eh too much work to properly normalise.	2 years ago
Mike Lang	c4a1d72240	Add new state to postgres and implement cutter logic	2 years ago
Mike Lang	1add3c5c22	Implement tombstoning to allow for segment deletion ... Rarely, we find ourselves needing to explicitly delete some data, eg. something that shouldn't have been public and should be removed from all records. It would also be nice if we could "clean up" bad versions of the same segment, which occasionally come up when downloaders have issues. With our distributed segment database, this is actually rather difficult as deleting the data from any one server would cause it to be restored from the others. It was only possible by stopping all backfill, deleting the data on all servers, then starting backfill again. Here we introduce a more practical approach. An operator creates an empty flag file with the same name as the segment to be deleted, but with a `.tombstone` extension. eg. to delete a file `/segments/desertbus/source/2019-11-13T02/45:51.608000-2.0-full-7IS92rssMzoSBQDIevHStbTNy-URRV3Vw-jzZ6pwOZM.ts`, you would create a tombstone `/segments/desertbus/source/2019-11-13T02/45:51.608000-2.0-full-7IS92rssMzoSBQDIevHStbTNy-URRV3Vw-jzZ6pwOZM.tombstone`. These tombstone files do two important things: * They hide the segment from being listed, which both means: * It can't be restreamed or put into a video * It can't be backfilled to other nodes * The tombstone files themselves do get backfilled to other nodes, so you only need to mark them on one server. Once the tombstone has propagated to all nodes, the segment file can be deleted independently on each one. We chose not to have a tombstone automatically trigger a segment deletion for safety reasons.	2 years ago
Mike Lang	44d0c0269a	cache results of common.segments.best_segments_by_start ... The restreamer spends most of its time iterating through segments (parsing them, determining the best one for each start time) to serve large time ranges. Since this only depends on the list of filenames read from disk, we can cache it for a given hour as long as that list is identical. This is a little trickier than it sounds because best_segments_by_start is an iterator and in most cases it won't be fully consumed. So we introduce a `CachedIterator` abstraction that will both remember the previously yielded values, and keep track of the live iterator so it can be resumed again if a previous invocation only partially consumed it. This also has the nice side effect of merging simultaneous operations - if two requests come in for the same hour at the same time, they'll share one iterator and both consume the results as they come in.	3 years ago
Mike Lang	871925aef5	restreamer: Disallow implicit time ranges over 12h ... Fixes #286	3 years ago
Mike Lang	9f9ef66a85	Add endpoint to get a given frame of video	3 years ago
Mike Lang	63d8b1d504	playlist manager: serialize insert requests ... We've seen cases where videos are not inserted even though the API call succeeded. Our suspicion is that two concurrent insert calls for the same video are causing a race. We try to avoid this by putting a lock around insert calls	3 years ago
ElementalAlchemist	a942af9cb4	When padding video in editor, maintain video playback position in the same spot when possible	3 years ago
ElementalAlchemist	4bac4a46d0	Add stream name to restreamer URL	3 years ago
ElementalAlchemist	210ae6f92a	Support linking to specific times in restreamer	3 years ago
ElementalAlchemist	8e8ace6f3c	Properly set initial video quality	3 years ago
ElementalAlchemist	e515c448eb	Fix loading chapter marker setting when loading a saved draft edit	3 years ago
ElementalAlchemist	1ffb9ae2e1	Fix saving quality levels on change	3 years ago
ElementalAlchemist	965a3dfbd3	Fix range removal	3 years ago
ElementalAlchemist	59567f9404	Fix issue where pressing space for video pause/unpause can scroll the page	3 years ago
ElementalAlchemist	91cc331128	Move the error selecting times on restreamer to be common to both pages	3 years ago
ElementalAlchemist	ac79206ba0	Don't reload when canceling or resetting	3 years ago
ElementalAlchemist	1ff1bcf0ab	Add keyboard shortcuts to mute, set playback rate to slowest speed, and set playback rate to fastest speed	3 years ago
ElementalAlchemist	d635a7941c	Add support for generating chapter markers in the video description	3 years ago
ElementalAlchemist	e2fb245da2	Correctly handle padding of range times when padding video load time over holes	3 years ago
ElementalAlchemist	0709082af7	Show a confirmation dialog when closing the page while a submission is still pending	3 years ago
ElementalAlchemist	c1e20de813	Fix loading videos with no end time	3 years ago
ElementalAlchemist	2698b7fd74	Fix handling of lack of end time for a row	3 years ago
ElementalAlchemist	13035c3e97	Add help for new keyboard controls	3 years ago
ElementalAlchemist	e96b77992f	Add one-second jump keyboard shortcuts	3 years ago
Mike Lang	ac44298299	restreamer: Disallow very long waveform images ... These can take a LOT of server resources (RAM, IO), so we'd rather just fail out on very long edits where the resulting image would be useless anyway.	3 years ago
ElementalAlchemist	f798b689a4	Add keyboard shortcut for setting playback speed to 1x	3 years ago
ElementalAlchemist	481b934eb8	Fix losing much of video human time when converting a time containing hours	3 years ago
Mike Lang	101468cbca	Fix a bug when reporting errors claiming candidates ... The SQL args were the wrong way around, so we tried to set error to the id. Thankfully this failed as the id queried is not a valid uuid. Fix by making args named like every other usage of query()	3 years ago
ElementalAlchemist	b1b8ef8973	Add fullscreen functionality to Thrimbletrimmer video player (#250) ... * Add fullscreen functionality to Thrimbletrimmer video player * Fix issue where clicking on fullscreen video pauses and unpauses instead of doing just one	3 years ago