henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
50 Commits
37 Branches
0 Tags
20 MiB
mlang/suspicious-skew
master
mike/encode-worker
mike/playlist-debug
mike/faster-chat-merge
HubbeKing-multiplatform-builds
thrimbletrimmer-solid
bus-progress-ui
bus-post-processing
chrusher/bus_synthesizer
mike/chat-no-end-time
chrusher-playlist-thumbnails
chrusher-thumbnail-frontend
mike/prizebot
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'a9b4b32a11'
${ noResults }
Commit Graph

50 Commits (a9b4b32a119b21bd5a882e8c060a2c27603cf2ea)
 

Author SHA1 Message Date
Christopher Usher a9b4b32a11 fixed white space and the like 7 years ago
Christopher Usher e1ba794be5 Forgot to try to remove temporary file 7 years ago
Christopher Usher 828b23e12a moved rename, ensure_directory and jitter to common 
Move a few useful functions in downloader used in the backfiller to common
 7 years ago
Christopher Usher d2dfcdc8f7 modifications to the backfiller in response to ekimekim's comments 7 years ago
Christopher Usher 9ce479a2a0 hopefully more robust 7 years ago
Christopher Usher 8497b293c9 Moving things around to make the backfiller a bit more like a proper package 7 years ago
Christopher Usher f4d5a4e0e0 Getting close to something I can show ekimekim 7 years ago
Christopher Usher 3404c67278 Chipping away at backfiller 
I'm going to have to learn to write better commit messages
 7 years ago
Christopher Usher 29e230462d much closer to being functional 7 years ago
Christopher Usher 5c368a21fc Merge branch 'master' into chrusher/backfiller 7 years ago
Christopher Usher 1aa2f827b0 a few ideas extra 7 years ago
Christopher Usher 9782a3ebd1
Merge pull request #6 from ekimekim/mike/restreamer/improvements 
restreamer: Multiple improvements and general "finishing"
 7 years ago
Mike Lang b4e627f382 restreamer: When generating playlists, include discontinuities, timestamps and endlist 
This fills out the incomplete playlist generation functionality to handle holes
and communicate extra information. See comments for details.
 7 years ago
Mike Lang 201959888a restreamer: More accurate target duration in playlist 7 years ago
Mike Lang e34f04cf57 restreamer: Harden generate_media_playlist to handle weird inputs and defaults 7 years ago
Mike Lang 6fa74608fb common: Improve some docs to note types of things that are ambiguous 7 years ago
Mike Lang 8f5a98a906 restreamer: Don't offer a variant on the master playlist if it's outside requested time range 
This prevents clients from picking a variant that they then can't play any content for.
In general we expect the same content to be available on all variants being captured,
but if the set of captured variants changes we still want to handle that gracefully.
 7 years ago
Mike Lang 3bbe1ed32d Prefer longer duration on multiple segments 7 years ago
Christopher Usher 4981c6521b
Merge pull request #5 from ekimekim/mike/restreamer/initial 
Initial work on restreamer
 7 years ago
Christopher Usher ed69e19cdc Merge branch 'master' into chrusher/backfiller 7 years ago
Mike Lang 5942091d1a restreamer: Cleanup around argument processing 7 years ago
Mike Lang a1fa60828d Basic media playlist generation, missing special cases 7 years ago
Mike Lang 75c9793eac Remove central config file as it's more trouble than it's worth 
Simpler and easier for testing to stick to configuration via CLI args.
We'll worry about deployment later.
 7 years ago
Mike Lang 031dd60897 downloader: Fix some typos around the max age calculation 7 years ago
Mike Lang 9e115f8a42 restreamer: Also add ability to list known hours so we know where to start replicating from 7 years ago
Mike Lang bab2d15d6e Initial implementation of the restreamer 
Supports serving segments, listing segments for an hour, and generating playlists so it can stream.
 7 years ago
Mike Lang ee8f8f6571 restreamer: Initial skeleton 7 years ago
Mike Lang 0df8288013 common: Implement code for parsing paths and picking the best sequence of segments 
This is needed by both the restreamer and the cutter, hence its inclusion in common.

The algorithm is pretty simple - it takes the 'best' segment per start time by full first,
then length of partial. All the other complexity is mainly just around detecting and reporting holes,
and being inclusive of start/end points.
 7 years ago
Christopher Usher 67dbdf90ac start of a rough prototype of the backfiller 7 years ago
Christopher Usher 15fe6256a4
Merge pull request #3 from ekimekim/pull-format 
Fixed negative times in format_bustime
 7 years ago
Christopher Usher 8f462f5926 Fixed format_bustime docsting 7 years ago
Christopher Usher 4c22edf2e6 Fixed negative times in format_bustime 7 years ago
Mike Lang afb96cbfc4
Merge pull request #1 from ekimekim/mike/downloader/initial 
Re-write downloader to be resilient to failures as much as possible
 7 years ago
Mike Lang 6377db2aa2 downloader: Bug fixes and improvements 
* Fix bug where soft timeout is not cancelled if an exception occurs
* Various logging tweaks
* Prevent master playlist wait time from going negative
* Stop gracefully if stream worker detects end of stream
* Don't treat master playlist 404 as an error, it just means the stream isn't up
 7 years ago
Mike Lang 6e0dcd5e22 downloader: Fix bugs and missing bits in initial implementation 
* Set a reasonable log format
* Make soft timeouts not always fire
* Change soft_hard_timeout signature slightly for ease-of-use
* Make renames not fail if file already exists
* Misc typos
 7 years ago
Mike Lang f193bd0f54 Re-write downloader to be resilient to failures as much as possible 
This makes the code crazy complicated and messy, but means we can be persistent about
not giving up, while still retrying at the same time, and trying multiple urls at once
until we find one that works.

See docstrings for a full discussion on some of the failures we're trying to work around.
 7 years ago
Mike Lang 1a7ae8525b Fix readme internal link 7 years ago
Mike Lang 10241b6190 Downloader: Implement a very basic proof of concept version 
Missing a LOT of tankiness, ways to configure, conversion to bustime, etc.
 7 years ago
Mike Lang 8993773a22 downloader.twitch: Deals with twitch specifics of playlist management 7 years ago
Mike Lang 961712b919 downloader: Import hls_playlist from streamlink 
This is a useful library and we might as well use it.
Copying it over and slightly modifying it to work was easier than importing all of streamlink.

The original version may be found at 30043408c7/src/streamlink/stream/hls_playlist.py
 7 years ago
Mike Lang 30612f00ad downloader: basic startup path 7 years ago
Mike Lang 1b21694c27 Add a simple build script to build docker images and a basic dockerfile 7 years ago
Mike Lang 71b8f8a1b9 gitignore: ignore config file 7 years ago
Mike Lang 439b623599 Add skeleton of downloader service 7 years ago
Mike Lang d7641aecf5 common: Fix bugs and issues with bustime utils 7 years ago
Mike Lang 048277b003 common: Basic config and bustime code 7 years ago
Mike Lang a361ab7a63 Add a common package for common bits in multiple components 7 years ago
Mike Lang 4105c90e9f Add a basic README 7 years ago
Mike Lang 9fb9d0f3e2 Add initial design doc 
This represents the system as envisaged at the start of the project.
It's a snapshot of a living google doc.
 7 years ago
Mike Lang cc82877022 Initial Commit 7 years ago