henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
406 Commits
34 Branches
0 Tags
19 MiB
Commit Graph

48 Commits (095e391b6051fdd23d05733053e57b7578561b1c)

Author SHA1 Message Date
Mike Lang da3cc24ed1 Pass a list of upload locations from thrimshim to thrimbletrimmer 
with the first one being the default.
Mike Lang ba746ff6e6 Add title and description header/footer in thrimshim instead of cutter 
This accomplishes two things:
1. It allows thrimshim to properly validate length restrictions (not implemented yet)
2. It means that the database has a record of the values actually written for each of these rows,
instead of that information depending on how the cutter was configured at the time.
ekimekim 742fcc3a28 Update default worksheet list to correct typo
ekimekim 86f22d38d8 docker-compose: Need to tell sheet-sync to allocate ids 
NOTE: This is only safe if we're only running one.

Later we should make a way to control this in config so one node has it
but others can run non-allocating sheetsyncs.
Mike Lang cbd0ef3d9e cutter: Add title header, description footer and static tags 
These are pre-canned parts of the video metadata that we want to be configurable.
Mike Lang 8e6c40610a cutter: Allow config to manually disable transcode checking for location 
This deals with the problem where multiple youtube locations that refer
to the same actual account (but with different settings) will all try to check
for when videos are done transcoding, when only one is needed.
Mike Lang ecf58dfbfc cutter: Add ability to configure multiple upload locations 
Cutter now takes a 'config' arg which is a json blob with detail
on each upload location. This is a bit nasty if you're trying to run it manually
but was the easiest way to transfer the config data from docker-compose.jsonnet
to the actual application.
Mike Lang 514a7a4d08 Split cutter and sheetsync creds so they can be different 
This also means that they can't access each others' creds,
which is good for security.
Christopher Usher a2e47b98f5 fixes -- in dates and the lack of the preshow
Christopher Usher d1e2957198 minor style changes to docker-compose
Christopher Usher 9a13372976 docker-compose fix
Christopher Usher 44f0e0defb changed it back so only the name is checked
Christopher Usher 7591428168 added localhost backfiller option
mg ca11e59511 Adding channel/start time info to Thrimshim to pass to Thrimbletrimmer.
mg 24e4e0ba28 Fixed how thrimbletrimmer is enabled/disabled.
mg 5c08054cdd Fixing default services enabled.
mg ae3d82026e Removed extrenuous .bak files, ensured newline at the end of files, and changed thrimbletrimmer to be separate from services.
mg 6edd224395 Merging Thrimbletrimmer into Wubloader with latest changes.
Christopher Usher af086e26e5 moved schema into database startup script
Christopher Usher fec8cff185
Merge branch 'master' into chrusher-database-replication
Christopher Usher 027c2900e2 fixes in response to ekim's comments
Christopher Usher 86477fae13 fixes for ekim's comments
Christopher Usher 689101b1ee not authenticating no longer default in docker-compose
Christopher Usher 611a4bd793 added option not to authenticate thrimshim
Christopher Usher 57962c8104 allow nodes and editors to be read from csv files on start up
Christopher Usher 0eefdb94e0 docker compose updates; option not to replicate
mg 0cca85b4aa Starting thrimbletrimmer integration.
Christopher Usher 3f2c30849f not authenticating no longer default in docker-compose
Christopher Usher d55d6fb7bc added option not to authenticate thrimshim
Christopher Usher 4d67adb277 editing a comment
Christopher Usher 7a576c7a1f comments in docker-compose
Christopher Usher 14d5c41167 some small edits to docker compose
Christopher Usher 732c56d502 typo in a comment
Christopher Usher 720684a388 refactoring to have consistent terminology
Mike Lang 3fa3c73d0e Fix nginx when some services are disabled 
nginx tries to resolve everything at startup, which doesn't work
if some of the services aren't present.

we instead generate the config file from a passed in env var, so that only
enabled services are present.
Mike Lang 6071a2f18d docker_compose: Add a local postgres instance as an optional service 
The node hosting the database can then easily run it as part of the stack.
Mike Lang 63eb324ba5 Add nginx service that provides a frontend to all the other services 
This allows us to run all the different services and expose all their metrics,
all on one port.
Mike Lang a7a54db726 docker-compose: Restructure for some finer control 
Allow enabling/disabling at top of file
Allow no port to be exposed for any service
Mike Lang 6c47f32d16 docker-compose.jsonnet: Add sheet sync
Mike Lang 33be5a92cb Update docker-compose file for thrimshim and cutter 
Also update thrimshim to take db connect args as positional,
for consistency with cutter.
Christopher Usher 728adb7c1d improvements suggested by ekim
Christopher Usher baae0f1ac1 bug fix in arg list
Mike Lang 901cda4814 Enable backdoor in all services, and add telnet to containers
Mike Lang 7f9a1dbe45 downloader: Remove implicit source quality arg 
This brings it in line with backfiller, is more flexible and less surprising
Mike Lang 89d6b3a6be docker-compose: Add list of peers to backfill from
Mike Lang bdcb217d20 docker-compose: Expose metrics ports for other services
Christopher Usher 225288980a Added the backfiller to docker-compose
Mike Lang 78a9a4e525 Set up a docker compose file to run all images 
For ease-of-use, we use a jsonnet file to generate the yaml.
Jsonnet is a language for generating JSON documents.

In this case it's useful to us because it lets us have comments,
references to settings defined at the top, and some basic logic
like converting qualities from a list of strings to a comma-seperated string.

To avoid requiring jsonnet to be installed, we use the official jsonnet docker image
in the generate script.