henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,002 Commits
34 Branches
0 Tags
19 MiB
thrimbletrimmer-solid
HubbeKing-multiplatform-builds
bus-progress-ui
master
bus-post-processing
chrusher/bus_synthesizer
mike/faster-chat-merge
mike/chat-no-end-time
chrusher-playlist-thumbnails
chrusher-thumbnail-frontend
mike/prizebot
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '05d5a190fc'
${ noResults }
Commit Graph

1002 Commits (05d5a190fc550ba51c1d8d7a0d730a719422f18c)
 

Author SHA1 Message Date
Mike Lang 1add3c5c22 Implement tombstoning to allow for segment deletion 
Rarely, we find ourselves needing to explicitly delete some data, eg. something that shouldn't
have been public and should be removed from all records.

It would also be nice if we could "clean up" bad versions of the same segment,
which occasionally come up when downloaders have issues.

With our distributed segment database, this is actually rather difficult as deleting the data
from any one server would cause it to be restored from the others. It was only possible
by stopping all backfill, deleting the data on all servers, then starting backfill again.

Here we introduce a more practical approach. An operator creates an empty flag file
with the same name as the segment to be deleted, but with a `.tombstone` extension.
eg. to delete a file `/segments/desertbus/source/2019-11-13T02/45:51.608000-2.0-full-7IS92rssMzoSBQDIevHStbTNy-URRV3Vw-jzZ6pwOZM.ts`,
you would create a tombstone `/segments/desertbus/source/2019-11-13T02/45:51.608000-2.0-full-7IS92rssMzoSBQDIevHStbTNy-URRV3Vw-jzZ6pwOZM.tombstone`.

These tombstone files do two important things:
* They hide the segment from being listed, which both means:
  * It can't be restreamed or put into a video
  * It can't be backfilled to other nodes
* The tombstone files themselves do get backfilled to other nodes, so you only need to mark them on one server.

Once the tombstone has propagated to all nodes, the segment file can be deleted independently on each one.

We chose not to have a tombstone automatically trigger a segment deletion for safety reasons.
 2 years ago
Mike Lang 44d0c0269a cache results of common.segments.best_segments_by_start 
The restreamer spends most of its time iterating through segments (parsing them, determining the best one for each start time)
to serve large time ranges. Since this only depends on the list of filenames read from disk,
we can cache it for a given hour as long as that list is identical.

This is a little trickier than it sounds because best_segments_by_start is an iterator
and in most cases it won't be fully consumed. So we introduce a `CachedIterator` abstraction
that will both remember the previously yielded values, and keep track of the live iterator
so it can be resumed again if a previous invocation only partially consumed it.

This also has the nice side effect of merging simultaneous operations - if two requests come in
for the same hour at the same time, they'll share one iterator and both consume the results
as they come in.
 3 years ago
Mike Lang 871925aef5 restreamer: Disallow implicit time ranges over 12h 
Fixes #286
 3 years ago
Mike Lang 9f9ef66a85 Add endpoint to get a given frame of video 3 years ago
Mike Lang 63d8b1d504 playlist manager: serialize insert requests 
We've seen cases where videos are not inserted even though the API call succeeded.
Our suspicion is that two concurrent insert calls for the same video are causing a race.

We try to avoid this by putting a lock around insert calls
 3 years ago
ElementalAlchemist a942af9cb4 When padding video in editor, maintain video playback position in the same spot when possible 3 years ago
ElementalAlchemist 4bac4a46d0 Add stream name to restreamer URL 3 years ago
ElementalAlchemist 210ae6f92a Support linking to specific times in restreamer 3 years ago
ElementalAlchemist 8e8ace6f3c Properly set initial video quality 3 years ago
ElementalAlchemist e515c448eb Fix loading chapter marker setting when loading a saved draft edit 3 years ago
ElementalAlchemist 1ffb9ae2e1 Fix saving quality levels on change 3 years ago
ElementalAlchemist 965a3dfbd3 Fix range removal 3 years ago
ElementalAlchemist 59567f9404 Fix issue where pressing space for video pause/unpause can scroll the page 3 years ago
ElementalAlchemist 91cc331128 Move the error selecting times on restreamer to be common to both pages 3 years ago
ElementalAlchemist ac79206ba0 Don't reload when canceling or resetting 3 years ago
ElementalAlchemist 1ff1bcf0ab Add keyboard shortcuts to mute, set playback rate to slowest speed, and set playback rate to fastest speed 3 years ago
ElementalAlchemist d635a7941c Add support for generating chapter markers in the video description 3 years ago
ElementalAlchemist e2fb245da2 Correctly handle padding of range times when padding video load time over holes 3 years ago
ElementalAlchemist 0709082af7 Show a confirmation dialog when closing the page while a submission is still pending 3 years ago
ElementalAlchemist c1e20de813 Fix loading videos with no end time 3 years ago
ElementalAlchemist 2698b7fd74 Fix handling of lack of end time for a row 3 years ago
ElementalAlchemist 13035c3e97 Add help for new keyboard controls 3 years ago
ElementalAlchemist e96b77992f Add one-second jump keyboard shortcuts 3 years ago
Mike Lang ac44298299 restreamer: Disallow very long waveform images 
These can take a LOT of server resources (RAM, IO), so we'd rather just fail out
on very long edits where the resulting image would be useless anyway.
 3 years ago
ElementalAlchemist f798b689a4 Add keyboard shortcut for setting playback speed to 1x 3 years ago
ElementalAlchemist 481b934eb8 Fix losing much of video human time when converting a time containing hours 3 years ago
Mike Lang 101468cbca Fix a bug when reporting errors claiming candidates 
The SQL args were the wrong way around, so we tried to set error to the id.
Thankfully this failed as the id queried is not a valid uuid.

Fix by making args named like every other usage of query()
 3 years ago
ElementalAlchemist b1b8ef8973
Add fullscreen functionality to Thrimbletrimmer video player (#250) 
* Add fullscreen functionality to Thrimbletrimmer video player

* Fix issue where clicking on fullscreen video pauses and unpauses instead of doing just one
 3 years ago
Mike Lang d1ba4bc4eb Downgrade overlapping segments from warning to info 
They were causing too much log noise
 3 years ago
ElementalAlchemist a8f9611d93 Update quality indicator when keyboard shortcuts are used to change the video quality 3 years ago
ElementalAlchemist 78b9b5c68d Maintain playback rate across video reloading 3 years ago
ElementalAlchemist 75e0ba56ae Fix error when loading a new playlist (when fragments aren't yet loaded by hls.js) 3 years ago
ElementalAlchemist f7c9eb5e5b Fix incorrect variable reference 3 years ago
ElementalAlchemist c44da044a8 Make the waveform clearer/more useful at larger resolutions by requesting a larger waveform image 3 years ago
Mike Lang 953eb8b2a6 Fix typo in postgres schema 3 years ago
Mike Lang 467edf3d19 Read dynamic playlist manager config from sheet 
The sheetsync loads playlist ids and tags into a new table `playlists`.
playlist manager reads this table and merges it with the playlists given on the command line.
 3 years ago
ElementalAlchemist 7e5705996e Further improve video controls styling for aesthetics 3 years ago
ElementalAlchemist d20f37266f Add a visible error when trying to load a time range with no video content 3 years ago
ElementalAlchemist 4fb8548aba Make muting work 3 years ago
ElementalAlchemist e6aa395c95 Improve appearance of video controls slightly 3 years ago
ElementalAlchemist 4e35fed5f2 Handle when the video stops playing due to load issues when seeking 3 years ago
ElementalAlchemist 497c975e3e Implement custom video controls for the new player (so we can better control styling) 3 years ago
ElementalAlchemist 56699d5737 Put quality name in the place where clients read it (or at least HLS.js does) 3 years ago
ElementalAlchemist ed2ff1b9a5 Make HLS.js not combine different quality levels into one 3 years ago
ElementalAlchemist e2487f61c0 Fix interpretation of times located in a hole in the video segments 3 years ago
ElementalAlchemist 5a9f4502c2 Set default time zone for Luxon instead of setting the time zone on each DateTime object 3 years ago
ElementalAlchemist 0340f06170 Initial transition from VideoJS to HLS.js as the video player 3 years ago
ElementalAlchemist 11bf89305a Fix video overflowing player area 3 years ago
Hubbe 9750cbd728 Set ghcr.io username to user triggering the build 
This should hopefully make pushes work in PRs? I'm grasping at straws at this point.
 3 years ago
HubbeKing daa1b2741b
Fix default k8s.jsonnet PVC capacity 3 years ago