henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
541 Commits
34 Branches
0 Tags
19 MiB
thrimbletrimmer-solid
HubbeKing-multiplatform-builds
bus-progress-ui
master
bus-post-processing
chrusher/bus_synthesizer
mike/faster-chat-merge
mike/chat-no-end-time
chrusher-playlist-thumbnails
chrusher-thumbnail-frontend
mike/prizebot
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'd231078048'
${ noResults }
Commit Graph

541 Commits (d2310780481527fa3acf09148618478a4f20f117)
 

Author SHA1 Message Date
Mike Lang d231078048 Add ability to explicitly pull and re-use layers from other commits when building 
This is intended mainly for travis CI, because by default it doesn't cache any layers
between builds.

By pulling likely-reusable builds (all parents of the current commit),
we take a fixed cost slowdown but in many cases should see a dramatic speed increase
overall, since we won't need to re-build anything that hasn't changed.

This isn't needed for local builds, where docker will do this on its own
with any previously-built images.
 5 years ago
Mike Lang cff5c38691 Add new dashboard 5 years ago
Mike Lang 2efe1d6218 Fix a bad logging line when handling errors 5 years ago
Mike Lang 59ee5cf5c0 Only log at INFO about multiple versions of a segment 
Since these tend to happen around stream endings, etc,
we don't want them to be crazy noisy and cause us to disregard real problems.

We can use the segment coverage to see in metrics if there are overlaps.
 5 years ago
Mike Lang 4be8faf82e
Merge pull request #151 from ekimekim/mike/sheetsync/track-row-stats 
sheetsync: Record counts of rows in the DB, segmented by various columns
 5 years ago
Mike Lang 89a9e5554c sheetsync: Record counts of rows in the DB, segmented by various columns 
This lets us view a number of useful graphs in dashboards, eg. rows by state,
errored rows, rows by day, rows by category, meltdowns per day, fraction of
events that are poster moments by category.

Sheetsync was the natural place to do this since it was already periodically scanning
the entire events table.
 5 years ago
Mike Lang 72172024be overview dashboard: Stop reporting stream delay after stream stops 
It just goes up forever and isn't helpful.
 5 years ago
Mike Lang 77f23d775a overview dashboard: Show offending instance in error log rate graph 5 years ago
Mike Lang 249e32583b get_best_segments: Don't error if the only segments that exist for time are temp 5 years ago
Mike Lang 605fedeb52 cutter: Log when upload errors occur 
This got lost in the shuffle at some point.
 5 years ago
Mike Lang 0f4f04a998
Merge pull request #150 from ekimekim/mike/disable-flamegraph 
Allow disabling of stacksampling with an env var
 5 years ago
Mike Lang 3eb0ed2350 thrimbletrimmer: Fix bug where "Draft Saved" is shown after a failed submission 5 years ago
Mike Lang 8bcc38d386 docker-compose: Add generic option to pass env vars to all services 5 years ago
Mike Lang 6b602592f5 Allow disabling of stacksampling with an env var 
This gives an easy way to do so across all services without adding new options.

Reasons to do so might be to avoid overheads or because your prometheus metrics grow too large.
 5 years ago
Mike Lang 2accfa9080
Merge pull request #149 from ekimekim/mike/monitoring/stuff 
monitoring fixes and improvements
 5 years ago
Mike Lang e5a7c8adfa monitoring: Add "role" concept 
This lets us know if a service is MEANT to be running or not.
 5 years ago
Mike Lang 21a46a66bb monitoring: Set instance to friendly name for each node we're monitoring 
So that you get eg. "charm" instead of "IP:PORT"
 5 years ago
Mike Lang 51adeeab19 monitoring: Fix problems with the prometheus container 5 years ago
Mike Lang b84d4de085 Add segment_coverage service to be monitored 5 years ago
Mike Lang 6f0eff0258
Merge pull request #148 from ekimekim/mike/nginx/prevent-dns-caching 
nginx: Prevent nginx from caching dns results
 5 years ago
Mike Lang 8ae600f2e5 nginx: Prevent nginx from caching dns results 
Otherwise if the containers get restarted and change ip, nginx hits the wrong ip.

We do this via a hack where we make all references indirect through a variable.
Since nginx only resolves this at request time, it always does a dns request.
 5 years ago
Mike Lang 5ed8cf0a33
Merge pull request #147 from ekimekim/chrusher-downloader-label-fix 
fixed outdated "stream", "variant" in metric
 5 years ago
Mike Lang 7b6f691d63
Merge pull request #146 from ekimekim/mike/unicode/hack 
Automatically set default encoding to utf-8 when common is imported
 5 years ago
Christopher Usher abb9193705 fixed outdated "stream", "variant" in metric 5 years ago
Mike Lang 4d3aa94a71 Automatically set default encoding to utf-8 when common is imported 
To be clear, this is an awful hack.

It means that any implicit str/unicode coersion will use the utf-8 encoding,
which is basically always what you want.

However, it is possible that some badly-written libraries might be relying
on the default encoding being ascii, and will do weird things as a result.

Finally, it's especially hacky to be doing this as part of importing a library.
Normally you're meant to do this as part of a sitecustomize.py in your python system directory,
and the function is deleted before passing control to normal code (this is why we need
to reload() to get it back).
 5 years ago
Mike Lang f452aa3c32
Merge pull request #143 from ekimekim/mike/sheetsync/api-usage 
sheetsync improvements
 5 years ago
Mike Lang 1c0f3a627b sheetsync: Log what worksheets got synced 
it's kinda important
 5 years ago
Mike Lang 8b25f8be95 sheetsync: Inject an error into the error column if we fail to parse an input column 5 years ago
Mike Lang 8dc7b80de9 sheetsync: Improve timing of main loop 
Instead of always waiting 5 seconds between runs,
wait until 5 seconds after the previous run started.

This ensures we actually run every 5sec and not every 5sec + how long it takes to run
 5 years ago
Mike Lang cda8078f64 sheetsync: Only check the most recently changed two sheets most times 
Only check the other sheets every 4th time (20sec instead of 5sec).

This elminiates a huge source of unnessecary reads, which prevents us from going over
our API limit.
 5 years ago
Christopher Usher dfe9d61428 Update database install instructions 
Turns out postgres ignores the column order when reading from CSV so
update documenation to give the correct order.
 5 years ago
Mike Lang 48d4dddb1c
Merge pull request #142 from ekimekim/mike/fixes 
Misc fixes and improvements
 5 years ago
Mike Lang 07b93f779c replace readlink -f with realpath in build scripts 
readlink -f doesn't work on OS X
 5 years ago
Mike Lang 520f7821a6 thrimbletrimmer: Streaming page: remove links to editor/dashboard 
A little security through obscurity.

Well, more like just not leading people to stuff that's not helpful to them.
Once we have auth working it shouldn't matter. Worst they can do is download expensive cuts.
 5 years ago
Mike Lang e4706232ad cutter: Fix bug in handling cancellation before finalize 
it would always think it's finalized
 5 years ago
Mike Lang 4c0e20983c Local upload backend: Always use .ts extension 
Now that we're doing that for both fast and full cuts.
 5 years ago
Mike Lang 1cb4d41334 cutter: Set upload time when marking video as done without transcoding 5 years ago
Mike Lang a7ac264201 cutter: Fix local upload backend, which was returning video id as a uuid 
instead of a string
 5 years ago
Mike Lang 5921066ee0 docker-compose: Pass node name into cutter 
Otherwise it doesn't have a stable node name (hostname inside containers
is random) and this results in orphaned videos on unclean shutdown.
 5 years ago
Mike Lang da58021c6f cutter: Fix typo where encoding settings had wrong name 5 years ago
Mike Lang 8739a7a24a thrimbletrimmer: Popup on save draft 
so we know it succeeded
 5 years ago
Mike Lang 2342b66db0 thrimbletrimmer: Don't open advanced pane if upload_location is null 5 years ago
Mike Lang e95957c54f thrimbletrimmer: Widen trim time textboxes to fit timestamps 
This might be because I'm not using webfonts or some malarkey.
 5 years ago
Mike Lang 4f6f4cad8b sheetsync: Fix typos with metrics 5 years ago
Mike Lang 2554b72c3a Fix slightly wrong default sheet name 5 years ago
Mike Lang 6724027e5a segment_coverage: Reduce missing channel/quality to warning 5 years ago
Mike Lang 4d5157cdb5 Fix a mistake with allowing reuse of name in @timed() 5 years ago
Mike Lang 1ef15780df cutter: Fix lack of info in error handling 5 years ago
Mike Lang 71333cf826 backfiller: Only run one manager, not one per channel 
Then treat backfilling each channel just like backfilling each quality.

This is conceptually simpler (only one kind of thing, a (channel, quality))
and has better behaviour when a node is down (we only have one lot of error handling around it).

It also means we aren't asking the database for the same info once per channel,
and cuts down on logging noise.
 5 years ago
Mike Lang a6cd07077a docker-compose: Wrap DSN values in quotes so spaces are allowed 5 years ago