henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
781 Commits
34 Branches
0 Tags
19 MiB
thrimbletrimmer-solid
HubbeKing-multiplatform-builds
bus-progress-ui
master
bus-post-processing
chrusher/bus_synthesizer
mike/faster-chat-merge
mike/chat-no-end-time
chrusher-playlist-thumbnails
chrusher-thumbnail-frontend
mike/prizebot
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'cc49e703dc'
${ noResults }
Commit Graph

781 Commits (cc49e703dcd876093a6bf2665f54c37adf11ea04)
 

Author SHA1 Message Date
Mike Lang 9413b99b91 Implement playlist manager 4 years ago
Mike Lang 9562032c24 Create new playlist manager service 
Playlist manager adds youtube videos to youtube playlists based on database tags.
 4 years ago
HubbeKing 86f7823348 Replace calls to gevent.signal() with gevent.signal_handler() 
gevent.signal() was removed in gevent 1.5a4, see http://www.gevent.org/api/gevent.signal.html
Removed on Feb 5th, see https://github.com/gevent/gevent/pull/1530
 4 years ago
Mike Lang a53786dc2d Add file and make as build dependencies 
gevent now requires these to build. I'm not sure when this changed.
 4 years ago
Mike Lang 5ccb5afff1 Track number of ignored ads 4 years ago
Mike Lang 50770651ce Detect new style of twitch in-stream ads 
New ad segments always have a title like "Amazon|...".
This is the same fix as used by streamlink at time of writing.
 4 years ago
Mike Lang b9cd76b1a2 Add non-static implict tags in sheetsync 
In order for the upcoming playlist manager to be able to use the DB `tags` column to know
what tags a video has, all the tags it needs need to be present.

Previously, this was a problem because the day and category tags only get added at the cutter
and so wouldn't be listed.

This moves them so they are added when parsing the row in sheetsync.
It also adds the poster moment tag if poster moment is checked.

Note that fully static tags that go on all videos are still only added in cutter,
but the playlist manager doesn't need to care about those (since by definition
they will match every video).
 4 years ago
Mike Lang a30d595fee cutter: Use video's custom tags when uploading 4 years ago
Mike Lang c26a87565f Add video tags to thrimbletrimmer 
Tags default to tags given on the sheet, but can be modified by the editor.
Tags are represented as a comma-seperated string, and are round-tripped on loss of focus
as a way to validate.
 4 years ago
Mike Lang 7e0c415f83 thrimshim: Add video tags column as required 4 years ago
Mike Lang 29571fb60b Add tags column to sheetsync 
New tags column shunts all columns after it right by 1.

Note we parse tags by splitting on commas then discarding whitespace.
If this would create an empty string tag, it is ignored.
Example: "foo, bar baz,a,,bc " -> ["foo", "bar baz", "a", "bc"]
 4 years ago
Mike Lang 68b75fa891 Add tags and video_tags database columns 
tags is a sheet input which provides a default list of tags for the editor.
video_tags is set upon the video being edited and is used by the cutter to set video tags on supported upload locations.
 4 years ago
Hubbe 521b193584
Merge pull request #178 from ekimekim/k8s-ingress-tls 
Add TLS settings to k8s.jsonnet
 4 years ago
HubbeKing 9eca34c7a9 Fix suggested style changes 4 years ago
HubbeKing 4ca425dd12 Simplify ingress_secret_name setting and add ingress_labels setting 
using std.objectHas() removes the need to have a separate boolean to check.
 4 years ago
HubbeKing cf7670d008 More sensible default secretName for ingress TLS 
The given secretName has to be in the same namespace as the ingress itself.
 4 years ago
HubbeKing 1dc62c193f Add TLS settings to k8s.jsonnet 
Use jsonnet computed field names to optionally add TLS configuration to generated Ingress object

In this way, one can easily let the kubernetes ingress handle TLS, with or without a secretName

Additional configuration would be required to tie into cert-manager for automated cert generation
 4 years ago
Mike Lang d21b225a70
Merge pull request #177 from ekimekim/mike/fix-thrim-bug 
Fix a bug when we fail to load a playlist that prevents loading other playlists
 4 years ago
Mike Lang a37d9a0c54 Fix a bug when we fail to load a playlist that prevents loading other playlists 4 years ago
Mike Lang 46d96bf900
Merge pull request #173 from ekimekim/k8s 
Add alternate deploy stack for kubernetes
 4 years ago
Mike Lang 239fc5d9ad
Merge pull request #176 from ekimekim/mike/update-clock 
Update clock.html for 2020 and display negative time properly
 4 years ago
Mike Lang d2d457a45f Add k8s option to README 4 years ago
Mike Lang b248408842 Update clock.html for 2020 and display negative time properly 4 years ago
Mike Lang d6dbf5512d
Merge pull request #175 from ekimekim/sheet-updates 
sheetsync: Move column indexes to match updated sheet
 4 years ago
Mike Lang 7600ce67a9
Merge pull request #174 from ekimekim/mike/generate-videos 
Add restreamer call to generate full concat'd videos
 4 years ago
Mike Lang b85296a81e sheetsync: Move column indexes to match updated sheet 
New tags column shunts all columns after it right by 1.
We will later want to parse that, but for now we ignore it.
 4 years ago
Mike Lang 61d934f333 docker-compose.jsonnet: Use seperate tag for database 
The database changes rarely, and is disruptive to re-deploy,
so we track its image version seperately so it only needs to be upgraded
when there's actually a change.
 4 years ago
Mike Lang 1b12c05e0e make smart cut work, only to discover it doesn't actually have any advantage over fast 4 years ago
Mike Lang b47eb3865f Fix segment_coverage -> segment-coverage 
because _ is illegal in k8s names.

However the image name does contain a _ so we replace - with _ when constructing the image name.
 4 years ago
HubbeKing 9a95dc44eb Fix k8s.jsonnet to compile to something kubernetes accepts 
Add Service definitions to output
Correct apiVersion for Ingress definition
Output a v1 List, as kubectl doesn't parse JSON arrays
 4 years ago
Mike Lang 28faff5c58 Add a k8s version of the docker-compose file 
Note this version is very simplified compared to the docker-compose
and has some major limitations:
* It relies on hostPath and a nodeSelector to put all the components on a shared storage node
* It only supports use as a replication node (downloader, restreamer, backfiller, segment_coverage)
* It uses the k8s Ingress instead of the built-in nginx for http routing.
 4 years ago
Mike Lang 4d21f447ad Add restreamer call to generate full concat'd videos 
This makes the timelapse easier.
 5 years ago
Mike Lang dea143cc0a
Merge pull request #163 from ekimekim/mike/api-ping 
api_ping: A simple util for hitting google apis to prevent inactivity
 5 years ago
Mike Lang 3489b0abaf api_ping: A simple util for hitting google apis to prevent inactivity 5 years ago
Mike Lang 2dbd1132fe common.googleapis: Fix a bug in retrying failed access token get 
Seems that this was never fixed when the code was moved.
 5 years ago
Mike Lang 47584821f9 thrimbletrimmer: Handle null trim in submit 
mainly for save draft
 5 years ago
Mike Lang 3086ab73f6 Add thrimbletrimmer clock 
To provide a true bustime even when they're bussing with the overlay
 5 years ago
Mike Lang 7dcd844e16 add logging to help debug smart cut 5 years ago
Mike Lang c294fa82b8 smart cut: Fix output format 5 years ago
Mike Lang c6172ce37f smart cut: More typos 5 years ago
Mike Lang 82346a55ca smart cut: Fix int in ffmpeg args 5 years ago
Mike Lang b39e844c1e restreamer: Fix missing import of smart cut 5 years ago
Mike Lang 58ff90cefe Further fix for when getRealTimeForPlayerTime is null 5 years ago
Mike Lang 58a2a0402d thrimbletrimmer: Handle getting trim time when no valid trim time is set 
Just return null instead
 5 years ago
Christopher Usher dbad2b4082 fixed download link for stream page 5 years ago
Christopher Usher d56801014b added support for suspect segments to segment_coverage 5 years ago
Mike Lang 21d5548980 Add new segment type "suspect" 
We've noticed that when nodes have connection problems, they get full segments
with different hashes. Inspection of these segments shows that
they all have identical data up to a point.

Segments that fetched normally will then have the remainder of the data.
Segments that had issues will have a slightly corrupted end.
The data is still valid, and no errors are raised. It just doesn't have all the data.

We noticed that these corrupted segments all were cut off exactly 60sec after their requests
began. We believe this is a server-side timeout on the request that returns whatever data
it has, then closes the container file cleanly before returning successfully.

We detect segments that take > 59 seconds to recieve, and label them as "suspect".

Suspect segments are treated identically to partial segments, except they are always preferred
over partials.
 5 years ago
Mike Lang bb05e37ae4 segments: Use longest segment in bytes if duration is the same 
We occasionally see corrupted segments that are slightly shorter in size
but report the same metadata as the full segments. Prefer the largest version
as it's likely the least corrupt.
 5 years ago
Mike Lang b516917e62 Add new "smart" cut technique 5 years ago
Mike Lang bb3814f9f7 overview.jsonnet: Use a template variable to allow restricting to certain nodes 5 years ago