henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,459 Commits
34 Branches
0 Tags
19 MiB
Commit Graph

1459 Commits (5bd8fbd65f00b9ca4ac56dde06f8eed5ff82632f)
 

Author SHA1 Message Date
Mike Lang 1915d98143 thrimbletrimmer: Split error path for submission into a function 
and use it correctly when erroring in transition code
Mike Lang ca60852b3d thrimbletrimmer: Add transitions to download URL
Mike Lang 8c6141b5ca thrimbletrimmer: Correct video time conversions due to overlap from transitions
Mike Lang c96c7cedd6 thrimbletrimmer: add transitions when submitting
Mike Lang 54695bcad4 thrimbletrimmer: Set existing transition info when loading a video
Mike Lang 72110821d5 thrimbletrimmer: Show/hide transition duration if "cut" is selected
Mike Lang 5d09115a8a thrimbletrimmer: Load transition types from thrimshim and populate dropdown
Mike Lang 87d4520e61 thrimbletrimmer: Add new DOM elements for specifying transitions 
Expects a "transitions" list on the video info that currently isn't there.
Not actually used anywhere yet.
Mike Lang 23e64116bd thrimbletrimmer: Add some helpers for creating elements 
No functional changes intended.
Mike Lang 3fb6676fbe pubbot: Add more detail to logging 
Wrap raw pubnub messages in a metadata object containing:
- time
- our hostname
- our pid
- any inferred information

Plus make a message-less log write on startup so we know where there might be gaps.

Also do more error handling so that we don't completely lose messages on error.
Mike Lang 9b19c12a53 Add pubnub-watching zulip bot 
Which also records all pubnub messages for archival to a file SEGMENTS/pubnub-log.json.
We ran this for most of last year, but not as part of wubloader.
ZeldaZach c378a1e4ab Add Audit Logging for several endpoints 
- Use transactions for DB commits to avoid audit-less logs

Endpoints Supported:
- Manual Link
- Reset Row
- Update Row
Mike Lang fd78ff288e fix bugs in postgres schema from earlier changes
Mike Lang 7483794a23 docker-compose: Set up sheetsync for downloading media links 
This involves giving it access to the SEGMENTS/media directory.
Mike Lang 3e7cb38cf0 sheetsync: Optionally download media linked in image links column 
To enable this, you need to:
- set --media-dir globally for sheetsync
- enable download_media=true for the events sync config
To disable for individual rows (eg. because of known issues), put "[nodownload]" in the notes column.
Mike Lang 7b590cf574 chat-archiver: Some cleanups to the URL matching regex 
With thanks to Me-Me for review
Mike Lang 15f86551d4 docker-compose: Backfill of media, with or without chat archiver
Mike Lang 9dfb00f4ab chat_archiver: Logic for checking and downloading media links
Mike Lang 2855ec759d download_media: Add pdf to default allowed content types 
We want to capture linked PDFs in addition to videos and images
Mike Lang b46c577014 download_media: Add function for checking if a URL has been downloaded before
Mike Lang 352c9e9081 download_media: Get data from potentially malicious URLs and store in the filesystem 
This is suitable for taking arbitary URLs from chat, etc and trying to fetch them.
It downloads them to a filepath that contains a hash of the URL and content.
Mike Lang 07055e3605 chat-archiver: extract the ensure_emotes greenlet management to a class
Mike Lang 3f6263a037 playlist-manager: More fixes
Mike Lang d8953568b2 playlist_manager: Add --once option
Mike Lang 9d12d37053 playlist_manager: Fixes
Mike Lang 23ad78d592 Record in database when end time is "--" 
We need this so that reverse sync reproduces these values correctly.

To handle this in the database, we have a composite type (dashed: boolean, value: timestamp).
Value is always valid and is equivalent to the old timestamp column,
but must be equal to start_time if dashed is true.

The only place we directly reference this column outside sheetsync is thrimshim, where we
always consider the value only.
Mike Lang e50adbf2da Fix a bug where transitions past the first are not timed correctly 
The video offset is timed relative to the full video up until that point, not the previous range.
Mike Lang 69bfa79e24 Make video transition durations floats, not intervals 
Almost all code was already expecting this.
Mike Lang 649a0fb2dd cutter: Improve error message for unhandled cutting exceptions 
In particular, use exception chaining instead of printing the old error.
Mike Lang 55f48e1881 cutter: Add unlisted-only safety flag to youtube upload backend
Mike Lang 12150a4005 thrimshim: Return list of available transitions 
For use in thrimbletrimmer for a drop-down.
Mike Lang d4de1f94be Add descriptions to xfade transitions
Mike Lang 1dec53924f fix typo
Mike Lang 91511295c0 Fix poster moment tag
Mike Lang 25e5e933b5 Fix typo
Mike Lang 7d89569ead Code more defensively around out-of-order or missing parent ids
Mike Lang 760dbd1e07 sheetsync: Convey row parent info 
by prefixing with a number of ^ characters and noting the parent id in the notes column.
Mike Lang 23960d947b streamlog: implicit tags must be first
Mike Lang 664f98150f sheets implicit tags fix
Mike Lang 62491d119f debugging
Mike Lang 06931db26b more sheetsync fixes
Mike Lang 1a5687204d debugging
Mike Lang 820f7322f9 more streamlog fixes
Matija Rezar fbb6c4dca0 current stat dump
Mike Lang b68abb92e2 more streamlog column names
Mike Lang 484e768303 correct streamlog column name
Mike Lang 20ee382b7f sheetsync: Fix missing arg 
Just hard-code that the tags sheet is always called "Tags" for now, we can fix later
if it ever needs to change.
Mike Lang c6058cbdd0 sheetsync: Update streamlog playlist middleware with API changes 
To allow setting playlist name and show_in_description
Mike Lang 367e6a7a7a sheetsync fixes 
typos and omissions
Mike Lang d5f73c226c GoogleAPIClient: Improve error reporting