Christopher Usher
377a6697b0
decorator to avoid duplication authentication code
5 years ago
Christopher Usher
4779d88e37
code style clean up
5 years ago
mg
f4d0fbf42e
Adding issuer validation.
5 years ago
mg
f3b1b991e5
Adding dependencies and retrieving account info.
5 years ago
Christopher Usher
d578da90d0
Setting up Auth Test function
5 years ago
Christopher Usher
365d7b7852
Placeholder text
5 years ago
Christopher Usher
ea42cab8fd
Added editor, edit_time and upload_time to thrimshim and cutter updates of the
...
database
5 years ago
Christopher Usher
fb889cd078
database changes to keep track of editors and edit times
5 years ago
Christopher Usher
84270c02ec
logging fix
5 years ago
Christopher Usher
4d67adb277
editing a comment
5 years ago
Christopher Usher
67100c4126
comments
5 years ago
Christopher Usher
7a576c7a1f
comments in docker-compose
5 years ago
Christopher Usher
fdb5d20db7
fix to database logging
5 years ago
Christopher Usher
497845f2da
typos in comments
5 years ago
Christopher Usher
14d5c41167
some small edits to docker compose
5 years ago
Christopher Usher
54aa919131
Update readme
5 years ago
Christopher Usher
72ab291a13
Update INSTALL.md
5 years ago
Christopher Usher
b4ce8fdbb6
Update INSTALL.md
5 years ago
Christopher Usher
56c9a68c82
Draft of an installation guide for the wubloader
5 years ago
Christopher Usher
76bc629720
moved flask monitoring to its own module
5 years ago
Christopher Usher
73541f852f
logging and monitoring for thrimshim
5 years ago
Christopher Usher
6c633df3ee
move restreamer.stats to common.stats
5 years ago
Christopher Usher
6858c2e2de
starting on logging and monitoring
5 years ago
Mike Lang
1721fbd92e
fix dashboards for channel/quality naming
5 years ago
Mike Lang
04ef0d3823
fix a few remaining usages of StreamWorker.stream instead of .quality
5 years ago
Christopher Usher
361e577474
fixes based on ekimekims suggestions
5 years ago
Christopher Usher
732c56d502
typo in a comment
5 years ago
Christopher Usher
3564643613
refactoring downloader
5 years ago
Christopher Usher
b959853593
refactored to channel and quality
5 years ago
Christopher Usher
720684a388
refactoring to have consistent terminology
5 years ago
Christopher Usher
6d38250674
starting to refactor stream to channel and variant to quality
5 years ago
Mike Lang
a2b21966b9
Merge pull request #65 from ekimekim/mike/dashboards
...
Add grafana dashboards as jsonnet code
5 years ago
Mike Lang
f7b591e78b
sheetsync: Log more information on HTTPError
...
The api gives additional detail that we want to know when debugging.
5 years ago
Mike Lang
73d5941e05
downloader: Track timestamp of latest segment
...
This gives us a "stream delay" metric.
Prom doesn't have any native way to check the current value of a metric,
in order to take max(). It only offers increment and set.
We reach into some internals to do this in a hacky way,
but the cleaner way would be to track the value ourselves and have a prom callback
that gets the value.
Sigh, I hate this prom library. I might write my own that's less dumb.
5 years ago
Mike Lang
e4d3e418c8
transcode checker: longer retry while waiting for videos to finish
...
but still check db often.
This prevents us from using too much api quota on these checks,
while still letting us spot new videos quickly.
5 years ago
Mike Lang
1f15900b6f
cutter: At least for now, don't auto-retry errors
...
This leads to rapidly exhausting our upload limit since even a fast failed request
costs the same amount of usage quota as a 1-hour long video.
5 years ago
Mike Lang
fbef4725d7
cutter: Handle case where we are told to stop while looking for candidates
...
Previously, it would return None and things would break. Now the None is handled
correctly, and is documented.
5 years ago
Mike Lang
5cec6ec96e
cutter: Reconnect after any error that might be a database error
...
After certain kinds of DB error (eg. lost conn), we need to make a new conn
to have things work again. To be safe, we just do it after every error where it might
be a problem.
5 years ago
Mike Lang
fea9ff6c1d
cutter: Fix dockerfile, which was missing ffmpeg dependency
5 years ago
Mike Lang
f50276bd01
backfiller: Expose recent_cutoff as CLI arg and increase it to 120s default
...
In testing, GDQ's stream delay went up over 1min, which caused backfillers to backfill
segments at the same time they were downloaded. We increase the window for now,
and also make it configurable.
5 years ago
MasterGunner
6fa9d9d388
Merge pull request #64 from ekimekim/gunner/additional-thrimbletrimmer-integration
...
Gunner/additional thrimbletrimmer integration
5 years ago
MasterGunner
2e953eddde
Cleanup from Ekim's comments, removed auth placeholder until I know what I'm doing.
5 years ago
Mike Lang
ca925ae2e6
dashboard: Add some extra detail sections for backfiller and downloader
5 years ago
MasterGunner
6a171130e8
Updated Get All Rows route.
5 years ago
Mike Lang
39e7a5c2e6
Add overview dashboard
5 years ago
MasterGunner
736f0e0fe4
Adding get_all_row and auth function stubs
5 years ago
Mike Lang
41fffc2809
Merge pull request #62 from ekimekim/mike/monitoring
...
Scripts for running prometheus/grafana for monitoring
5 years ago
MasterGunner
4423ddee3c
Update SecurityModel.md
...
Simplified the document based on our discussions.
5 years ago
Mike Lang
612e34b88d
Merge pull request #61 from ekimekim/mike/backfiller/concurrent
...
backfiller: Allow multiple concurrent segment downloads
5 years ago
Mike Lang
29040a166c
backfiller: Allow multiple concurrent segment downloads
...
This will signifigantly increase throughput when downloading
large ranges of segments.
The max concurrency is exposed as a cli arg.
We also slightly modify the logged info, so it reports segments downloaded,
not just number of missing segments (which we might skip downloading for various reasons).
5 years ago