henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,518 Commits
31 Branches
0 Tags
18 MiB
chrusher/time-of-day-fixes
master
HubbeKing-multiplatform-builds
mike/faster-chat-merge
mike/chat-no-end-time
chrusher-playlist-thumbnails
chrusher-thumbnail-frontend
mike/prizebot
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'master'
${ noResults }
Commit Graph

127 Commits (master)

Author SHA1 Message Date
Mike Lang 467edf3d19 Read dynamic playlist manager config from sheet 
The sheetsync loads playlist ids and tags into a new table `playlists`.
playlist manager reads this table and merges it with the playlists given on the command line.
 3 years ago
ElementalAlchemist 2b4e7a23e7 Update the default edit_url to make sense with the new Thrimbletrimmer structure 3 years ago
ElementalAlchemist e4204d9b9a Add Thrimbletrimmer dev option 3 years ago
Mike Lang 415932e289 docker-compose: fix bug where database_tag isn't respected 3 years ago
Mike Lang 3e768aa2a9 switch from quay.io to ghcr.io for image registry 3 years ago
Mike Lang 5f81c383bc Add day 8 3 years ago
Mike Lang 317d45f16d Manually specify upload locations that should be added to playlists 
This lets us exclude some youtube locations (like unlisted),
and include non-youtube ones (like manual-youtube)
 3 years ago
Mike Lang 463843668d segment-coverage: Render a monitoring page 3 years ago
Mike Lang e9b2831b71 docker-compose: Fix bug preventing playlist manager metrics from being routed 4 years ago
Mike Lang 2327c7c9a8 docker-compose: Fix typos in configuring playlist manager 4 years ago
Mike Lang 6e5e4b41df docker-compose/k8s: Limit segment-coverage start/end times 
Instead of defaulting to oldest/newest.
 4 years ago
Mike Lang 324d180c5e docker-compose: Default to only backfilling things from this year 
By setting a 6-month limit
 4 years ago
Mike Lang b029250c1c Disable stacksampler by default 
It causes problems due to the sheer number of unique metrics emitted, which makes
the prometheus endpoint be very expensive / fail a lot.

The data is not useful enough to justify the cost.
 4 years ago
Mike Lang a89bfcd271 Add playlist manager to all the various other places 4 years ago
Mike Lang 61d934f333 docker-compose.jsonnet: Use seperate tag for database 
The database changes rarely, and is disruptive to re-deploy,
so we track its image version seperately so it only needs to be upgraded
when there's actually a change.
 4 years ago
Christopher Usher 1325ccf280 added a read only user to database setup script 5 years ago
Mike Lang 64766bcf35 Add SSL to nginx if certs are given 5 years ago
Mike Lang 8bcc38d386 docker-compose: Add generic option to pass env vars to all services 5 years ago
Mike Lang 5921066ee0 docker-compose: Pass node name into cutter 
Otherwise it doesn't have a stable node name (hostname inside containers
is random) and this results in orphaned videos on unclean shutdown.
 5 years ago
Mike Lang 2554b72c3a Fix slightly wrong default sheet name 5 years ago
Mike Lang a6cd07077a docker-compose: Wrap DSN values in quotes so spaces are allowed 5 years ago
Mike Lang b2a07ef114
Merge pull request #140 from ekimekim/mike/build-improvements 
Refactor dockerfiles for more shared layers
 5 years ago
Mike Lang 751472217b Default to using the current commit, instead of latest 
This is generally what you want, and will behave better than using latest.
 5 years ago
Mike Lang b2cd0fa6f7 docker-compose: Support channel importance markers 
Only pass them to downloader, strip them otherwise.
 5 years ago
Christopher Usher 09d7feebfb changed list comprehension to concatenation 5 years ago
Christopher Usher 02fd265407 allow for multiple channels in docker-compose 5 years ago
Mike Lang eaf69fac1a
Merge pull request #118 from ekimekim/mike/nginx/serve-segments 
nginx: Serve the segments directly from the segments dir
 5 years ago
Christopher Usher 223af52265 added coverage to docker-compose 5 years ago
Mike Lang 3c20a9aece nginx: Serve the segments directly from the segments dir 
instead of proxying through restreamer.

This should improve performance when serving the (large) segment files,
and free up restreamer for things like generating the playlist.
 5 years ago
Mike Lang da3cc24ed1 Pass a list of upload locations from thrimshim to thrimbletrimmer 
with the first one being the default.
 5 years ago
Mike Lang ba746ff6e6 Add title and description header/footer in thrimshim instead of cutter 
This accomplishes two things:
1. It allows thrimshim to properly validate length restrictions (not implemented yet)
2. It means that the database has a record of the values actually written for each of these rows,
instead of that information depending on how the cutter was configured at the time.
 5 years ago
ekimekim 742fcc3a28 Update default worksheet list to correct typo 5 years ago
ekimekim 86f22d38d8 docker-compose: Need to tell sheet-sync to allocate ids 
NOTE: This is only safe if we're only running one.

Later we should make a way to control this in config so one node has it
but others can run non-allocating sheetsyncs.
 5 years ago
Mike Lang cbd0ef3d9e cutter: Add title header, description footer and static tags 
These are pre-canned parts of the video metadata that we want to be configurable.
 5 years ago
Mike Lang 8e6c40610a cutter: Allow config to manually disable transcode checking for location 
This deals with the problem where multiple youtube locations that refer
to the same actual account (but with different settings) will all try to check
for when videos are done transcoding, when only one is needed.
 5 years ago
Mike Lang ecf58dfbfc cutter: Add ability to configure multiple upload locations 
Cutter now takes a 'config' arg which is a json blob with detail
on each upload location. This is a bit nasty if you're trying to run it manually
but was the easiest way to transfer the config data from docker-compose.jsonnet
to the actual application.
 5 years ago
Mike Lang 514a7a4d08 Split cutter and sheetsync creds so they can be different 
This also means that they can't access each others' creds,
which is good for security.
 5 years ago
Christopher Usher a2e47b98f5 fixes -- in dates and the lack of the preshow 5 years ago
Christopher Usher d1e2957198 minor style changes to docker-compose 5 years ago
Christopher Usher 9a13372976 docker-compose fix 5 years ago
Christopher Usher 44f0e0defb changed it back so only the name is checked 5 years ago
Christopher Usher 7591428168 added localhost backfiller option 5 years ago
mg ca11e59511 Adding channel/start time info to Thrimshim to pass to Thrimbletrimmer. 5 years ago
mg 24e4e0ba28 Fixed how thrimbletrimmer is enabled/disabled. 5 years ago
mg 5c08054cdd Fixing default services enabled. 5 years ago
mg ae3d82026e Removed extrenuous .bak files, ensured newline at the end of files, and changed thrimbletrimmer to be separate from services. 5 years ago
mg 6edd224395 Merging Thrimbletrimmer into Wubloader with latest changes. 5 years ago
Christopher Usher af086e26e5 moved schema into database startup script 5 years ago
Christopher Usher fec8cff185
Merge branch 'master' into chrusher-database-replication 5 years ago
Christopher Usher 027c2900e2 fixes in response to ekim's comments 5 years ago
Christopher Usher 86477fae13 fixes for ekim's comments 5 years ago
Christopher Usher 689101b1ee not authenticating no longer default in docker-compose 5 years ago
Christopher Usher 611a4bd793 added option not to authenticate thrimshim 5 years ago
Christopher Usher 57962c8104 allow nodes and editors to be read from csv files on start up 5 years ago
Christopher Usher 0eefdb94e0 docker compose updates; option not to replicate 5 years ago
mg 0cca85b4aa Starting thrimbletrimmer integration. 5 years ago
Christopher Usher 3f2c30849f not authenticating no longer default in docker-compose 5 years ago
Christopher Usher d55d6fb7bc added option not to authenticate thrimshim 5 years ago
Christopher Usher 4d67adb277 editing a comment 5 years ago
Christopher Usher 7a576c7a1f comments in docker-compose 5 years ago
Christopher Usher 14d5c41167 some small edits to docker compose 5 years ago
Christopher Usher 732c56d502 typo in a comment 5 years ago
Christopher Usher 720684a388 refactoring to have consistent terminology 5 years ago
Mike Lang 3fa3c73d0e Fix nginx when some services are disabled 
nginx tries to resolve everything at startup, which doesn't work
if some of the services aren't present.

we instead generate the config file from a passed in env var, so that only
enabled services are present.
 5 years ago
Mike Lang 6071a2f18d docker_compose: Add a local postgres instance as an optional service 
The node hosting the database can then easily run it as part of the stack.
 5 years ago
Mike Lang 63eb324ba5 Add nginx service that provides a frontend to all the other services 
This allows us to run all the different services and expose all their metrics,
all on one port.
 5 years ago
Mike Lang a7a54db726 docker-compose: Restructure for some finer control 
Allow enabling/disabling at top of file
Allow no port to be exposed for any service
 5 years ago
Mike Lang 6c47f32d16 docker-compose.jsonnet: Add sheet sync 5 years ago
Mike Lang 33be5a92cb Update docker-compose file for thrimshim and cutter 
Also update thrimshim to take db connect args as positional,
for consistency with cutter.
 5 years ago
Christopher Usher 728adb7c1d improvements suggested by ekim 6 years ago
Christopher Usher baae0f1ac1 bug fix in arg list 6 years ago
Mike Lang 901cda4814 Enable backdoor in all services, and add telnet to containers 6 years ago
Mike Lang 7f9a1dbe45 downloader: Remove implicit source quality arg 
This brings it in line with backfiller, is more flexible and less surprising
 6 years ago
Mike Lang 89d6b3a6be docker-compose: Add list of peers to backfill from 6 years ago
Mike Lang bdcb217d20 docker-compose: Expose metrics ports for other services 6 years ago
Christopher Usher 225288980a Added the backfiller to docker-compose 6 years ago
Mike Lang 78a9a4e525 Set up a docker compose file to run all images 
For ease-of-use, we use a jsonnet file to generate the yaml.
Jsonnet is a language for generating JSON documents.

In this case it's useful to us because it lets us have comments,
references to settings defined at the top, and some basic logic
like converting qualities from a list of strings to a comma-seperated string.

To avoid requiring jsonnet to be installed, we use the official jsonnet docker image
in the generate script.
 6 years ago