henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,051 Commits
24 Branches
0 Tags
15 MiB
master
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from '18b71e6f5f'
${ noResults }
Commit Graph

1051 Commits (18b71e6f5f2dafb034890ab65d7b0c1184099173)
 

Author SHA1 Message Date
Mike Lang 3de44d6731 Add ability to render waveforms in restreamer 3 years ago
Mike Lang 4db8c8f61c thrimbletrimmer: undo bad refactor 3 years ago
Mike Lang 2865088692 thrimshim: Further validation of titles/descriptions 
Youtube doesn't allow < or > characters, or extremely long descriptions.

Fixes #215, #219
 3 years ago
Mike Lang 7599681b6d yet another py3 map() issue 
"hey i know lets make everything return an iterable but not update anything else to accept them"
 3 years ago
Mike Lang 62bd6539ea Unpin gevent as that was a workaround for a py2 issue 3 years ago
Mike Lang 714a16bac4 thrimbletrimmer: Use template literals and other cleanup 
Now that we've fixed the function declarations, prettier has also removed the semicolons on them
 3 years ago
Mike Lang a69bee0548 thrimbletrimmer: Replace var with let/const and other related bits 3 years ago
Mike Lang 241d302220 thrimbletrimmer: Use "function NAME(...)" instead of "NAME = function (...)" 
It's less confusing and more like other languages
 3 years ago
Mike Lang ce73f7b0ea Auto-format JS files with prettier 
Prettier is a tool for formatting JS files.
We add a minimal configuration to suit our purposes, and run it on all our JS files.

The main things this corrects is mixed tab and space indent issues (replacing fully with tabs),
overly long lines, inconsistent quote usage (prefers double-quotes instead), and missing semicolons.
 3 years ago
Christopher Usher 6c97bd462e fixed integer division issues introduced by port to Python 3 3 years ago
Mike Lang e63aa53019 Remove left-over usage of encode_strings 
More py3 breakage
 3 years ago
Mike Lang 21856c68aa Fix all instances of file.write() for py3 
In python 3, file.write() may do a partial write and returns the number of characters written.
In order to not lose data, we need to wrap every instance of file.write() with our new
common.writeall() wrapper that loops until the data is actually written.
 3 years ago
Mike Lang a56f6859bb more py3 fixes 3 years ago
Mike Lang f2a8007bf7 Fix build dependency issues 3 years ago
Mike Lang 19f70b1d06 py3 fixes for segment_coverage 3 years ago
Mike Lang ac917409e4 Fix formatting issue in playlist manager 3 years ago
Mike Lang 7d4eb3c8db py3 fixes for backfiller 3 years ago
Mike Lang 8f24c2eae1 py3 fixes for sheetsync 3 years ago
Mike Lang d42d850153 thrimshim: misc cleanup 3 years ago
Mike Lang 9fad66c6be py3 fixes for restreamer 3 years ago
Mike Lang 6a98addac8 py3 fixes for downloader 3 years ago
Mike Lang 50231a5154 py3 fixes for cutter 3 years ago
Mike Lang 3e69000058 py3 fixes for common 3 years ago
Mike Lang d03ae49eec Remove defunct "smart cut" method 
This was an alternate way of doing a cut that turned out to work exactly the same as a fast cut,
just with a more complex implementation.
 3 years ago
HubbeKing 6d790a1b36 Do a first naive pass for py3 compatibility 
Check that open() calls for reading and writing use binary modes
Use alpine version with py3-pip package
Use python3 in Dockerfile CMD
Remove sys.setdefaultencoding() "hack"
Simplify ensure_directory() in common.common package
 3 years ago
Mike Lang f0546e2ee3 Pin gevent to 1.5a2 to avoid https://github.com/gevent/gevent/issues/1711 3 years ago
Mike Lang fe4299e926 Change the generate_videos endpoint to make mkvs according to an hours spec passed in 3 years ago
Mike Lang 32138bbd43 downloader: Update to work with twitch's new access token API 
Twitch removed their old access token endpoint and now use a GraphQL endpoint.
The old endpoint would just always return 404, which we sadly interpreted as "stream not up".

Thankfully streamlink has already done the reverse engineering work so I was able to
update it to work again fairly easily, it's just a bit more convoluted.
 4 years ago
Hubbe 36a5387e26 Change wording on edit submitted alert() 
An edit has been submitted for the video. The video hasn't been submitted yet, thrimbletrimmer just informs other components how it wants the edit to be.
 4 years ago
Mike Lang 74a38bfaeb thrimbletrimmer: Instead of redirecting on submit, alert but stay 4 years ago
Mike Lang 337111a9ab thrimbletrimmer: Always alert() on errors 4 years ago
Hubbe 27265f3bca Set video.js volume to 50% by default 
When the video.js player is ready, set its volume to 50%
 4 years ago
Mike Lang f8c877775d Add an alternate manual upload mode specifically for youtube 
Adds a built-in "youtube-manual" location which is like "manual" except that it only works
with youtube URLs and populates the video_id column.

The intent is so that we can have playlist_manager manage videos we upload manually,
while still being able to distinguish between that and other manual links that shouldn't
be included (eg. links to third party youtube videos).

This is set when setting a manual link in thrimbletrimmer with a new checkbox, default off.
 4 years ago
Christopher Usher ad03a087ab Added tags to the thrimbletrimmer dashboard 4 years ago
Mike Lang 6d789ad42f thrimshim: Handle None values in comparing columns 4 years ago
Mike Lang cb75953e91 thrimshim: Fix a bug preventing submissions 
When comparing old and new video tags, it errors because it's a list, not string.

We change it to apply the transforms to all tags in the list, and also ignore changes in list ordering.
 4 years ago
Mike Lang 9d8c47377f segment parsing: Hand-roll microsecond parsing 
float() is inaccurate and Decimal() is very slow (~3x the cpu usage)
so instead we right-pad with 0s (eg. so 1.2345 -> 1.234500) then convert to int microsec directly.
 4 years ago
Christopher Usher 41d85b8614 Remove unicode 4 years ago
Christopher Usher 84f7ef71c1 More helpful UI if the database has changed plus ignore changes that are 
just white space or capitalisation
 4 years ago
Mike Lang 31d241eecc thrimbletrimmer: missing tags arg when submitting 4 years ago
Mike Lang 66669cd4e4 common: When parsing segment timestamps, use decimal instead of float 
Floating point error leads to 1us differences in parsed times,
which causes false positives in the overlapping segments check.

By using a Decimal, we get the exact digits from the filepath.
 4 years ago
Mike Lang 389d3e08d7 Fix typo in making playlist manager case insensitive 4 years ago
Mike Lang 3a19ba744d segment_coverage: Raise default check interval to 5min 4 years ago
Mike Lang 5235c3281a segment_coverage: Allow setting of check interval via cli flag 4 years ago
Mike Lang 15c357509f segment_coverage: Fix a problem where metrics would fail 
Because the checking process is entirely CPU-bound, it does not give any other
greenlets a chance to run while it is processing. This prevents us from responding
to metrics queries, and prometheus then times out.

By stopping to handle all other traffic in between each hour processed, we ensure metrics
remain responsive while processing.
 4 years ago
Mike Lang 13a228070a common.segments: Speed up segment parsing by rolling our own time parsing 
strptime is very slow. In terms of pure get_best_segments() speed, this change
more than doubles the throughput.

In particular for segment_coverage, this halves the run time for each check.
 4 years ago
Mike Lang e9b2831b71 docker-compose: Fix bug preventing playlist manager metrics from being routed 4 years ago
Mike Lang efe185bc0c Fix copy-paste error in restreamer metrics endpoint 4 years ago
Mike Lang fa1f305fda postgres: Changes to work with version 12 
In postgres 12, recovery.conf is removed in favor of normal config options
plus a signal file.

See https://www.2ndquadrant.com/en/blog/replication-configuration-changes-in-postgresql-12/
for a good rundown.
 4 years ago
Mike Lang a044b30465 postgres: Fix replica not working when there are spaces in the password 4 years ago