Commit Graph

124 Commits (master)

Author SHA1 Message Date
Mike Lang 3e7cb38cf0 sheetsync: Optionally download media linked in image links column
To enable this, you need to:
- set --media-dir globally for sheetsync
- enable download_media=true for the events sync config
To disable for individual rows (eg. because of known issues), put "[nodownload]" in the notes column.
3 days ago
Mike Lang 23ad78d592 Record in database when end time is "--"
We need this so that reverse sync reproduces these values correctly.

To handle this in the database, we have a composite type (dashed: boolean, value: timestamp).
Value is always valid and is equivalent to the old timestamp column,
but must be equal to start_time if dashed is true.

The only place we directly reference this column outside sheetsync is thrimshim, where we
always consider the value only.
4 weeks ago
Mike Lang 1dec53924f fix typo 1 month ago
Mike Lang 91511295c0 Fix poster moment tag 1 month ago
Mike Lang 25e5e933b5 Fix typo 1 month ago
Mike Lang 7d89569ead Code more defensively around out-of-order or missing parent ids 1 month ago
Mike Lang 760dbd1e07 sheetsync: Convey row parent info
by prefixing with a number of ^ characters and noting the parent id in the notes column.
1 month ago
Mike Lang 23960d947b streamlog: implicit tags must be first 1 month ago
Mike Lang 664f98150f sheets implicit tags fix 1 month ago
Mike Lang 62491d119f debugging 1 month ago
Mike Lang 06931db26b more sheetsync fixes 1 month ago
Mike Lang 1a5687204d debugging 1 month ago
Mike Lang 820f7322f9 more streamlog fixes 1 month ago
Mike Lang b68abb92e2 more streamlog column names 1 month ago
Mike Lang 484e768303 correct streamlog column name 1 month ago
Mike Lang 20ee382b7f sheetsync: Fix missing arg
Just hard-code that the tags sheet is always called "Tags" for now, we can fix later
if it ever needs to change.
1 month ago
Mike Lang c6058cbdd0 sheetsync: Update streamlog playlist middleware with API changes
To allow setting playlist name and show_in_description
1 month ago
Mike Lang 367e6a7a7a sheetsync fixes
typos and omissions
1 month ago
Mike Lang 44f4433cad more typos in streamlog archive middleware 1 month ago
Mike Lang eeffeeed10 sheetsync: Deal with reverse syncing properly when not all events are in the list of worksheets
This is important because archive events should not be reversed.
We only want to create new rows when the row's intended worksheet is in our list of worksheets we sync.
1 month ago
Mike Lang 96181fd875 Support archive sync in sheetsync again 1 month ago
Mike Lang a3aaa37bb0 Restructure playlists table
to have all tags, not just playlists.
Notably, this means playlist ids may be null.
1 month ago
Mike Lang 145ddfc7ff Fix some typos in StreamLogPlaylistsMiddleware 1 month ago
Mike Lang 0f06f96a7c Fix: Move events-specific observe_rows() out of SheetSync 1 month ago
Mike Lang 73eaac16f3 Update sheetsync CLI interface to allow any combination of types and backends 1 month ago
Mike Lang bb4cc8c668 sheetsync: Replace old special-case PlaylistSync with SheetSync subclass 1 month ago
Mike Lang ef11b69f4d Fix sheetsync to use Sheets client from common 1 month ago
Mike Lang 98b1749528 implement SheetsPlaylistMiddleware and fix lots of SheetsMiddleware issues 1 month ago
Mike Lang c84b2254ee sheetsync: Streamlog middleware for playlists 1 month ago
Mike Lang e9b6e27598 sheetsync: Make the middleware interface explicit with a base class
The main purpose here is just documentation, so it's clear what the semantics of a middleware is.
1 month ago
Mike Lang 3ffbefea4f sheetsync: Split SheetsMiddleware into a common base class and events specific 1 month ago
Mike Lang bebce3df4c sheetsync: Only consider a row missing if we did a full sync
This prevents things being spuriously missing because they were in a worksheet we didn't sync.
1 month ago
Mike Lang 3cdd8f22ad Split SheetSync object to be fully generic and have an EventsSync object for events 1 month ago
Mike Lang 6a9884ee5b sheetsync: Move extra metrics columns into a variable 1 month ago
Mike Lang 37dd7c3e4f sheetsync: Move various config fields to be defined on the class object
Also make the columns lists into sets
1 month ago
Mike Lang 5559dffec6 sheetsync: pull metric gathering into a seperate function 1 month ago
Mike Lang 37f42853ec sheetsync: Extract db table name into a variable
So we can swap it out for playlists later
1 month ago
Mike Lang 00d0ecc3bb sheetsync: Handle sheet_name as an input column to avoid special casing
sheet_name must always be present on a sheet row, but is only present on a db row
if the db actually stores that data.

As a side benefit, the db will now update if sheet_name changes.
1 month ago
Mike Lang 2db20d202b sheetsync: Update streamlog middleware for section -> tab rename 1 month ago
Mike Lang 9fb356bf45 sheetsync: Better error handling for running out of space when creating rows 1 month ago
Mike Lang 29ff11457e sheetsync: Namespace all logs and metrics behind a sheetsync "name"
This helps differentiate the multiple syncs we now have and will have:
- syncing events from streamlog
- reverse syncing events to sheets
- syncing playlists
1 month ago
Mike Lang 87b504a00a sheetsync: Rename "row" and "event" to "sheet_row" and "db_row"
First step to combining event and playlist sync into one codepath.

No actual behaviour changes intended.
1 month ago
Mike Lang 20ee79cdb1 Get sheets working again 1 month ago
Mike Lang f89ab6fa43 Don't make sheet name an input column, go back to special casing it on row create
This means it won't update if put in the wrong place,
but avoids issues with reverse sync trying to write it out when it's not an actual column
1 month ago
Mike Lang 430938dc49 error is always a string, it just might be empty 1 month ago
Mike Lang f8d3eb7f00 wip: 1 month ago
Mike Lang ee4a68af50 clear up confusion with empty string vs None 1 month ago
Mike Lang 3e873ca5f6 wip: fixes 1 month ago
Mike Lang eebfa5885b sheetsync: pass in event id instead of event name 1 month ago
Mike Lang cf41f572f5 Fix streamlog formatting 1 month ago