Add playlist manager to all the various other places

INSTALL.md

@@ -47,7 +47,7 @@ To generate the `docker-compose.yml` file used by `docker-compose`, run `generat
   
 After making any changes to `docker-compose.jsonnet`, you will need to rerun `generate-docker-compose`.
 
-By default the `downloader`, `restreamer`, `backfiller`, `cutter`, `thrimshim`, `segment_coverage` and `nginx` services of the wubloader will be run. To change which services are run edit the `enabled` object in `docker-compose.jsonnet`. A complete wubloader set up also requires one and only one `database` service (though having a backup database is a good idea) and one and only one `sheetsync` service.
+By default the `downloader`, `restreamer`, `backfiller`, `cutter`, `thrimshim`, `segment_coverage` and `nginx` services of the wubloader will be run. To change which services are run edit the `enabled` object in `docker-compose.jsonnet`. A complete wubloader set up also requires one and only one `database` service (though having a backup database is a good idea), one and only one `sheetsync` service and one and only one `playlist_manager` service.
 
 If you are running a `cutter` you will have to place the appropriate Google credentials in a JSON file given by the `cutter_creds_file`. Likewise, if you are running the `sheetsync` service, you will have to place the appropriate credentials in the JSON file pointed to by `sheetsync_creds_file` as well as set the appropriate `sheet_id` and `worksheets` for the Google sheet to sync with. You will also need to set the appropriate `edit_url` to access `thrimbletrimmer`.  
 

README.md

@@ -16,6 +16,7 @@ but a brief overview of the components:
 * `thrimshim` acts as an interface between the `thrimbletrimmer` editor and the database.
 * `thrimbletrimmer` is a browser based video editor.
 * `segment_coverage` regularly checks whether there is complete segment coverage for each hour. 
+* `playlist_manager` adds videos to youtube playlists depending on tags.
 * `database` hosts a Postgres database to store events to be edited.
 * `nginx` provides a webserver through which the other components are exposed to the outside world.
 * `common` provides code shared between the other components.

build

@@ -11,7 +11,7 @@ cd "$(dirname "$(realpath "$0")")"
 # Pass PUSH=true to also push the resulting images, or PUSH=latest to push them as :latest tag
 
 # The different images we can build
-COMPONENTS=(downloader restreamer backfiller thrimshim cutter sheetsync nginx postgres segment_coverage)
+COMPONENTS=(downloader restreamer backfiller thrimshim cutter sheetsync nginx postgres segment_coverage playlist_manager)
 
 # Define push if not already defined
 PUSH=${PUSH:-}

docker-compose.jsonnet

@@ -23,6 +23,7 @@
     sheetsync: false,
     thrimshim: true,
     segment_coverage: true,
+    playlist_manager: false,
     nginx: true,
     postgres: false,
   },
@@ -56,6 +57,7 @@
     thrimshim: 8004,
     sheetsync: 8005,
     segment_coverage: 8006,
+    playlist_manager: 8007,
     nginx: 80,
     nginx_ssl: 443,
     postgres: 5432,
@@ -135,6 +137,15 @@
   sheet_id:: "your_id_here",
   worksheets:: ["Tech Test & Preshow"] + ["Day %d" % n for n in std.range(1, 7)],
 
+  // A map from youtube playlist IDs to a list of tags.
+  // Playlist manager will populate each playlist with all videos which have all those tags.
+  // For example, tags ["Day 1", "Technical"] will populate the playlist with all Technical
+  // youtube videos from Day 1.
+  // Note that you can make an "all videos" playlist by specifying no tags (ie. []).
+  playlists:: {
+    "YOUR-PLAYLIST-ID": ["some tag"],
+  },
+
   // Extra options to pass via environment variables,
   // eg. log level, disabling stack sampling.
   env:: {
@@ -155,6 +166,12 @@
   // Cleaned up version of $.channels without importance markers
   clean_channels:: [std.split(c, '!')[0] for c in $.channels],
 
+  // Which upload locations have type youtube, needed for playlist manager
+  youtube_upload_locations:: [
+    location for location in $.cutter_config
+    if $.cutter_config[location].type == "youtube"
+  ],
+
   // docker-compose version
   version: "3",
 
@@ -311,6 +328,30 @@
       environment: $.env,
     },
 
+    [if $.enabled.playlist_manager then "playlist_manager"]: {
+      image: "quay.io/ekimekim/wubloader-playlist_manager:%s" % $.image_tag,
+      // Args for the playlist_manager
+      command: [
+        "--backdoor-port", std.toString($.backdoor_port),
+        "--upload-location-allowlist", std.join(",", $.youtube_upload_locations),
+        $.db_connect,
+        "/etc/wubloader-creds.json",
+      ] + [
+        "%s=%s" % [playlist, ",".join($.playlists[playlist])]
+        for playlist in std.objectFields($.playlists)
+      ],
+      volumes: [
+        // Mount the creds file into /etc
+        "%s:/etc/wubloader-creds.json" % $.cutter_creds_file,
+      ],
+      // If the application crashes, restart it.
+      restart: "on-failure",
+      // Expose on the configured host port by mapping that port to the default
+      // port for playlist_manager, which is 8007.
+      [if "playlist_manager" in $.ports then "ports"]: ["%s:8007" % $.ports.playlist_manager],
+      environment: $.env,
+    },
+
     [if $.enabled.nginx then "nginx"]: {
       # mapping of services to internal ports for nginx to forward
       local forward_ports = {