Commit Graph

67 Commits (d8a9b5ddf0d8ed77a62f5dbbe8ff5785375915e0)

Author SHA1 Message Date
Mike Lang 7525b7c135 restreamer: Add basic prometheus stats to all endpoints
I had to go to some effort to get nice labelling,
which also meant none of the existing libs for this were any good,
but this works well enough.

Exposes the metrics on /metrics.
6 years ago
Mike Lang 17972b87aa Allow setting of log level via WUBLOADER_LOG_LEVEL env var
By using an env var, it is universal and happens prior to arg parsing,
at the same point we do other logging setup.
6 years ago
Mike Lang c8cc4a68a0 cutter: Fix bugs that meant things wouldn't actually be cut
The calculations were backwards, so instead of cutting a video by, say, 2 seconds,
it would cut by -2 seconds, which was clamped to 0. So it would never actually cut,
it would always use the closest segment.

Also, once we were actually cutting, we hit an issue where ffmpeg would finish and close
its input early, because we'd reached the end of the cut video, but not all input had been written yet.
This resulted in an EPIPE error (write to closed pipe) in the input feeder. We now ignore that.
6 years ago
Mike Lang 6bf709287a cutter: Introduce an alternate cutting approach that is much faster
This cutter works by only cutting the first and last segments to size,
then concatting them with the other segments, so we only ever process a few seconds
of video instead of the entire video duration.
However, to make this work, care must be taken that the cut segments use the same codecs
as the other segments.

The reason it's experimental is that we are not yet confident in its ability
to cut accurately and without sync issues. We have seen some minor issues when trying to play
back the raw output files, but youtube's re-encoding has consistently smoothed out those issues
and they seem to be highly player-specific. Vigorous testing is needed.

Also note that both methods right now (cat then cut, and cut then cat) only work if all the segments
are cattable, that is they all use the same codecs, have the same resolution, etc.
If a stream were to change its encoding settings, and we were cutting over that change,
both approaches would not work. We should add checks for that scenario (which can only happen
over a stream drop), and if so fallback to a slow method using ffmpeg's concat filter,
which will work even for disparate codecs, though reconciling mismatched resolutions or frame rates
may require further work.
6 years ago
Mike Lang 18aadd6b82 restreamer: Also have an endpoint for generating cut videos on demand
This is mainly just for testing until we get the database and proper cutter up,
but it might prove useful to have in the long run too.

This code will probably end up being totally rewritten,
as it uses the most naive form of cutting and reencoding,
and it has a whole bunch of http-serving specifics intertwined with the cutting logic.
6 years ago
Mike Lang 97d77e19d6 restreamer: Add CORS headers to all responses
TBH I'm not sure why this is needed (i'm completely clueless about browser stuff),
but apparently thrimbletrimmer needs it.
6 years ago
Mike Lang afe19ca33e restreamer: Implement graceful stop on SIGTERM 6 years ago
Mike Lang 7ffa90c7e6 restreamer: Make docker image work, fix missing dependencies
setup.py and Dockerfile were both totally out of whack
6 years ago
Mike Lang b4e627f382 restreamer: When generating playlists, include discontinuities, timestamps and endlist
This fills out the incomplete playlist generation functionality to handle holes
and communicate extra information. See comments for details.
6 years ago
Mike Lang 201959888a restreamer: More accurate target duration in playlist 6 years ago
Mike Lang e34f04cf57 restreamer: Harden generate_media_playlist to handle weird inputs and defaults 6 years ago
Mike Lang 8f5a98a906 restreamer: Don't offer a variant on the master playlist if it's outside requested time range
This prevents clients from picking a variant that they then can't play any content for.
In general we expect the same content to be available on all variants being captured,
but if the set of captured variants changes we still want to handle that gracefully.
6 years ago
Mike Lang 5942091d1a restreamer: Cleanup around argument processing 6 years ago
Mike Lang a1fa60828d Basic media playlist generation, missing special cases 6 years ago
Mike Lang 9e115f8a42 restreamer: Also add ability to list known hours so we know where to start replicating from 6 years ago
Mike Lang bab2d15d6e Initial implementation of the restreamer
Supports serving segments, listing segments for an hour, and generating playlists so it can stream.
6 years ago
Mike Lang ee8f8f6571 restreamer: Initial skeleton 6 years ago