1213 Commits (3007bf91a43a72c44ca5db0575d6dc3cd6ea5d16)

 

All Branches   

Search

Author	SHA1	Message	Date
Mike Lang	3007bf91a4	Update sheetsync CLI interface to allow any combination of types and backends	12 months ago
Mike Lang	65cb58db4f	sheetsync: Replace old special-case PlaylistSync with SheetSync subclass	12 months ago
Mike Lang	e26949c078	Fix sheetsync to use Sheets client from common	12 months ago
Mike Lang	5b4769cf90	implement SheetsPlaylistMiddleware and fix lots of SheetsMiddleware issues	12 months ago
Mike Lang	909e198eff	sheetsync: Streamlog middleware for playlists	12 months ago
Mike Lang	176e37cccb	sheetsync: Make the middleware interface explicit with a base class ... The main purpose here is just documentation, so it's clear what the semantics of a middleware is.	12 months ago
Mike Lang	83ff3c69c9	sheetsync: Split SheetsMiddleware into a common base class and events specific	12 months ago
Mike Lang	b3713362d0	sheetsync: Only consider a row missing if we did a full sync ... This prevents things being spuriously missing because they were in a worksheet we didn't sync.	12 months ago
Mike Lang	95ca38fe1c	Split SheetSync object to be fully generic and have an EventsSync object for events	12 months ago
Mike Lang	da33235f20	sheetsync: Move extra metrics columns into a variable	12 months ago
Mike Lang	7b4ca69acf	sheetsync: Move various config fields to be defined on the class object ... Also make the columns lists into sets	12 months ago
Mike Lang	86dc30b52c	sheetsync: pull metric gathering into a seperate function	12 months ago
Mike Lang	e0c48fb989	sheetsync: Extract db table name into a variable ... So we can swap it out for playlists later	12 months ago
Mike Lang	33986cb7d9	sheetsync: Handle sheet_name as an input column to avoid special casing ... sheet_name must always be present on a sheet row, but is only present on a db row if the db actually stores that data. As a side benefit, the db will now update if sheet_name changes.	12 months ago
Mike Lang	3b78322092	sheetsync: Update streamlog middleware for section -> tab rename	12 months ago
Mike Lang	8a7a489c97	sheetsync: Better error handling for running out of space when creating rows	12 months ago
Mike Lang	e1020bd90b	sheetsync: Namespace all logs and metrics behind a sheetsync "name" ... This helps differentiate the multiple syncs we now have and will have: - syncing events from streamlog - reverse syncing events to sheets - syncing playlists	12 months ago
Mike Lang	35f5d3d3d9	sheetsync: Rename "row" and "event" to "sheet_row" and "db_row" ... First step to combining event and playlist sync into one codepath. No actual behaviour changes intended.	12 months ago
Mike Lang	c0420f960f	Get sheets working again	12 months ago
Mike Lang	f4672aad8c	Don't make sheet name an input column, go back to special casing it on row create ... This means it won't update if put in the wrong place, but avoids issues with reverse sync trying to write it out when it's not an actual column	12 months ago
Mike Lang	1c935765cc	error is always a string, it just might be empty	12 months ago
Mike Lang	54e0ce59a5	wip:	12 months ago
Mike Lang	2854728e75	clear up confusion with empty string vs None	12 months ago
Mike Lang	9038c3a5a7	wip: fixes	12 months ago
Mike Lang	eaff77f6b0	sheetsync: pass in event id instead of event name	12 months ago
Mike Lang	cc010b210a	Fix streamlog formatting	12 months ago
Mike Lang	bbb05d9f34	docker-compose: Allow setting up sheetsync for sheets or streamlog	12 months ago
Mike Lang	a5745f7bbd	sheetsync: Change how options are specified to allow multiple backends / syncs	12 months ago
Mike Lang	045c5057b4	Implement reverse sync mode ... This is a mode where all data flows one-way from the database to the sheet. It is intended to be used to populate an empty sheet from database events, possibly sourced from somewhere else. To make this work, a few changes were required: * Track which ids we've seen so we know what events were not matched with a row * Allow `row` to be None in sync_rows * When it is, call the middleware to create a new row with a new id * In sheets, this is implemented by tracking the last empty rows we saw, and claiming them as needed.	12 months ago
Mike Lang	9409943f67	sheetsync: Remove pick_worksheets() from middleware api ... Instead, get_rows() makes that decision internally if needed.	12 months ago
Mike Lang	61c5cba5d2	sheetsync: Remove worksheet from middleware apis ... since it's now baked into the row dict	12 months ago
Mike Lang	2d7dcbc64e	sheetsync: Always have sheet name as part of row dict	12 months ago
Mike Lang	4ed6b5fc0f	bugfix	12 months ago
Mike Lang	3e9f6ec4d1	sheetsync: Use streamlog section instead of deriving day from start time	12 months ago
Mike Lang	931f193fb5	sheetsync: Implement Streamlog middleware	12 months ago
Mike Lang	cdf185cafb	sheetsync: Move edit url management into Sheets middleware ... As streamlog doesn't require it.	12 months ago
Mike Lang	190ddd66c8	Change database primary key from UUID to TEXT ... We still store uuids, but in text form. This allows us to store non-UUID ids for systems that have other ids.	12 months ago
Mike Lang	5c6cb2900c	Sheetsync: Split into the main loop logic + sheets-specific middleware ... NOTE ON CONFLICTS In master, we moved sheets.py to common as it only contained a generic client. Now sheets.py also contains specific sheetsync stuff. Our resolution: - Keep the generic version in common - Keep the old version verbatim (including the now-redundant generic client) in sheetsync We will move the sheetsync implementation to the generic client after the rebase is complete.	12 months ago
Mike Lang	13a76c335b	sheetsync: Split playlist runloop from normal sheets	12 months ago
Mike Lang	9122ba0ae0	sheetsync: Move id allocation out of sync_row()	12 months ago
Mike Lang	e505a09081	sheetsync: Move row_index variable into row dict	12 months ago
Mike Lang	f6658721b3	Basic streamlog api	12 months ago
Mike Lang	6d3c901433	Revert "sheetsync: Support archive sheet" ... This reverts commit b93597c274.	12 months ago
Mike Lang	d1439e7376	Revert "sheetsync: unmapped columns aren't a problem." ... This reverts commit 5256577d00.	12 months ago
Mike Lang	9dec3cb729	Allow the youtube-manual upload backend to modify videos ... So we can modify youtube videos we manually uploaded. If the video isn't actually ours, it will error when the modify is attempted which is ok.	12 months ago
Mike Lang	49dd25c0fa	playlist_manager: Improved handling of playlist outdated errors ... By restarting the update immediately instead of waiting for the next one. We only try this up to 3 times to prevent excessive quota usage if it keeps happening.	12 months ago
Mike Lang	90a6b5ee17	playlist_manager: Check for re-orderings without doing them initially ... Previously, we would only do reorderings if we were refreshing the playlist for some other reason (eg. a video was inserted). We want to refresh the playlist before attempting reorderings, so we split the routine into two parts: - A part that finds out-of-order videos and returns a list of moves to make. - A part that executes those moves. We do the former before AND after refreshing, and the latter only with the result from after.	12 months ago
Mike Lang	0a55a74efd	playlist_manager: Reorder first/last videos when needed	12 months ago
Mike Lang	ce8e7bb3b4	playlist_manager: Make a specialized error for api requests ... So we can easily check the status code.	12 months ago
Mike Lang	a802cbb1d2	playlist_manager: Sort first/last videos correctly ... In addition to sorting the videos themselves into the correct spots, we need to special case them when scanning for the correct place to insert other videos. Note this only places them in the correct place on insert, which requires they be set in the playlist config BEFORE being inserted. A follow-up commit will handle the case of needing to re-order them post insert.	12 months ago