Christopher Usher
0b524a72cb
docstings and a few minor feature additions to the backfiller
6 years ago
Christopher Usher
a59f6e1569
ignore tempuary files
6 years ago
Christopher Usher
3b0342b872
added options to limit range of hours backfilled and to randomise hours backfilled
6 years ago
Christopher Usher
fec0975d18
fixed white space and the like
6 years ago
Christopher Usher
afd948576d
Forgot to try to remove temporary file
6 years ago
Christopher Usher
3cdfaad664
moved rename, ensure_directory and jitter to common
...
Move a few useful functions in downloader used in the backfiller to common
6 years ago
Christopher Usher
7d26997b1f
modifications to the backfiller in response to ekimekim's comments
6 years ago
Christopher Usher
ba52bf7f5d
hopefully more robust
6 years ago
Christopher Usher
50bcb84c0c
Moving things around to make the backfiller a bit more like a proper package
6 years ago
Christopher Usher
494725fe34
Getting close to something I can show ekimekim
6 years ago
Christopher Usher
5615c1bdb0
Chipping away at backfiller
...
I'm going to have to learn to write better commit messages
6 years ago
Christopher Usher
2fb17fff59
much closer to being functional
6 years ago
Christopher Usher
05fed36ac8
a few ideas extra
6 years ago
Christopher Usher
0e7ba25b76
start of a rough prototype of the backfiller
6 years ago
Mike Lang
97d77e19d6
restreamer: Add CORS headers to all responses
...
TBH I'm not sure why this is needed (i'm completely clueless about browser stuff),
but apparently thrimbletrimmer needs it.
6 years ago
Mike Lang
941b9b017e
build script: Add ability to push to remote repository after building
6 years ago
Mike Lang
afe19ca33e
restreamer: Implement graceful stop on SIGTERM
6 years ago
Mike Lang
7ffa90c7e6
restreamer: Make docker image work, fix missing dependencies
...
setup.py and Dockerfile were both totally out of whack
6 years ago
Mike Lang
1dce14bf77
downloader: Fix and improve the stop mechanism, stop on SIGTERM
...
Allows for graceful shutdown
6 years ago
Mike Lang
7b10429846
downloader: Dockerfile fixes to make it work
6 years ago
Mike Lang
6c3501db6f
downloader: Fix dateutil lib, which is actually called python-dateutil
6 years ago
Mike Lang
7257fb9b73
downloader: Include channel name in path, instead of assuming it's already in base_dir
...
Previously, downloader would put files under BASE_DIR/VARIANT/HOUR/FILE.ts
now, it will put files under BASE_DIR/STREAM/VARIANT/HOUR/FILE.ts
This brings downloader in line with restreamer's concept of base_dir
6 years ago
Christopher Usher
84097f4bbb
Merge pull request #11 from ekimekim/chrusher/comment
...
Added a comment to highlight recursion
6 years ago
Christopher Usher
efe30c1942
Added a comment to highlight recursion
6 years ago
Christopher Usher
62b184e333
Merge pull request #10 from ekimekim/add-license-1
...
Licence under MIT
6 years ago
Mike Lang
d0caf79768
Licence under MIT
...
Closes #9
6 years ago
Christopher Usher
9782a3ebd1
Merge pull request #6 from ekimekim/mike/restreamer/improvements
...
restreamer: Multiple improvements and general "finishing"
6 years ago
Mike Lang
b4e627f382
restreamer: When generating playlists, include discontinuities, timestamps and endlist
...
This fills out the incomplete playlist generation functionality to handle holes
and communicate extra information. See comments for details.
6 years ago
Mike Lang
201959888a
restreamer: More accurate target duration in playlist
6 years ago
Mike Lang
e34f04cf57
restreamer: Harden generate_media_playlist to handle weird inputs and defaults
6 years ago
Mike Lang
6fa74608fb
common: Improve some docs to note types of things that are ambiguous
6 years ago
Mike Lang
8f5a98a906
restreamer: Don't offer a variant on the master playlist if it's outside requested time range
...
This prevents clients from picking a variant that they then can't play any content for.
In general we expect the same content to be available on all variants being captured,
but if the set of captured variants changes we still want to handle that gracefully.
6 years ago
Mike Lang
3bbe1ed32d
Prefer longer duration on multiple segments
6 years ago
Christopher Usher
4981c6521b
Merge pull request #5 from ekimekim/mike/restreamer/initial
...
Initial work on restreamer
6 years ago
Mike Lang
5942091d1a
restreamer: Cleanup around argument processing
6 years ago
Mike Lang
a1fa60828d
Basic media playlist generation, missing special cases
6 years ago
Mike Lang
75c9793eac
Remove central config file as it's more trouble than it's worth
...
Simpler and easier for testing to stick to configuration via CLI args.
We'll worry about deployment later.
6 years ago
Mike Lang
031dd60897
downloader: Fix some typos around the max age calculation
6 years ago
Mike Lang
9e115f8a42
restreamer: Also add ability to list known hours so we know where to start replicating from
6 years ago
Mike Lang
bab2d15d6e
Initial implementation of the restreamer
...
Supports serving segments, listing segments for an hour, and generating playlists so it can stream.
6 years ago
Mike Lang
ee8f8f6571
restreamer: Initial skeleton
6 years ago
Mike Lang
0df8288013
common: Implement code for parsing paths and picking the best sequence of segments
...
This is needed by both the restreamer and the cutter, hence its inclusion in common.
The algorithm is pretty simple - it takes the 'best' segment per start time by full first,
then length of partial. All the other complexity is mainly just around detecting and reporting holes,
and being inclusive of start/end points.
6 years ago
Christopher Usher
15fe6256a4
Merge pull request #3 from ekimekim/pull-format
...
Fixed negative times in format_bustime
6 years ago
Christopher Usher
8f462f5926
Fixed format_bustime docsting
6 years ago
Christopher Usher
4c22edf2e6
Fixed negative times in format_bustime
6 years ago
Mike Lang
afb96cbfc4
Merge pull request #1 from ekimekim/mike/downloader/initial
...
Re-write downloader to be resilient to failures as much as possible
6 years ago
Mike Lang
6377db2aa2
downloader: Bug fixes and improvements
...
* Fix bug where soft timeout is not cancelled if an exception occurs
* Various logging tweaks
* Prevent master playlist wait time from going negative
* Stop gracefully if stream worker detects end of stream
* Don't treat master playlist 404 as an error, it just means the stream isn't up
6 years ago
Mike Lang
6e0dcd5e22
downloader: Fix bugs and missing bits in initial implementation
...
* Set a reasonable log format
* Make soft timeouts not always fire
* Change soft_hard_timeout signature slightly for ease-of-use
* Make renames not fail if file already exists
* Misc typos
6 years ago
Mike Lang
f193bd0f54
Re-write downloader to be resilient to failures as much as possible
...
This makes the code crazy complicated and messy, but means we can be persistent about
not giving up, while still retrying at the same time, and trying multiple urls at once
until we find one that works.
See docstrings for a full discussion on some of the failures we're trying to work around.
6 years ago
Mike Lang
1a7ae8525b
Fix readme internal link
6 years ago