henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,455 Commits
34 Branches
0 Tags
19 MiB
master
mike/faster-chat-merge
HubbeKing-multiplatform-builds
thrimbletrimmer-solid
bus-progress-ui
bus-post-processing
chrusher/bus_synthesizer
mike/chat-no-end-time
chrusher-playlist-thumbnails
chrusher-thumbnail-frontend
mike/prizebot
condor-scripts
mike/chat-subs
mike/bad-segment-hack
mike/backfill-extras
mike/cropping
mike/shorts
mike/download-frame-2
mike/debug-playlists
chrusher/backfiller_error_handler
sheet-updates
mike/generate-videos
suspect-segments
mike/testing
gunner/thrimbletrimmer-improvements
mike/render-dashboard
mike/database/initial
downloader/follow-game
mike/downloader/conn-pooling
mike/common/split-up
mike/backdoor
mike/downloader/handle-404
mike/non-restreamer-stats
archive/original-v2.1
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'mike/faster-chat-merge'
${ noResults }
Commit Graph

1455 Commits (mike/faster-chat-merge)
 

Author SHA1 Message Date
Mike Lang 50c233b58c fix copy paste error in v3/v4 and benchmark duplicate merges 
Merging 1000 duplicates of the same batch:
v1: 98ms
v2: 101ms
v3: 116ms
v4: 116ms

So as expected v3 and v4 have a small cost with duplicates for optimizing for non-overlapping,
but even with ridiculous numbers duplicates are fast to process anyway.
 1 month ago
Mike Lang c87a25e875 merge_v4 is more specialized to merge large numbers of non-overlapping batches 
It can do 24h of batches in 180ms, ~180x faster than v1.

I'm a little concerned it's overly specialized for that role and will be slow to merge duplicate batches,
but those operations involve tiny amounts of data by comparison so it doesn't really matter.
 1 month ago
Mike Lang eeeb48c0f8 add third implementation 
which is really just the second implementation but generalized to N-way merge instead of 2-way.

Basic benchmark results:
                         v1   v2   v3
    1 hour (60 batches): 19ms 31ms 23ms
24 hours (1440 batches): 34s  130s 17s

So it's about as fast as v1 when it doesn't matter, and twice as fast when it does.
 1 month ago
Mike Lang 91e3e54a14 add ability to dump output to debug 1 month ago
Mike Lang b090f99f9a REPORT: merge v2 is half the speed of v1 but is producing different results 
I was hoping for better than 2x, but this may improve on larger batches.
Different results is probably a trivial-ish bug
 5 months ago
Mike Lang 3ad24f5aec get working 5 months ago
Mike Lang 335e29e1bc wip: 6 months ago
Mike Lang a56f7e7ecb blogbot: can't hash strings 6 months ago
Mike Lang 2a12687473 blogbot: need to save the string version of the html 6 months ago
Mike Lang a648dee902 Fix typo in blogbot 6 months ago
Mike Lang fbbe37ca9e blogbot: Save blog contents by hash when seen 
For every unique hash of the blog html, it will save a copy.
Blogs are saved as JSON files in {segments_dir}/blogs
 6 months ago
Dan Collins 3b737f8155 Add 'batman logo wipe' transition 6 months ago
Mike Lang 18905a3c17 buscribe: Tell nginx to actually serve those pages 6 months ago
Mike Lang e91d6b65ac blogbot: add to docker-compose 6 months ago
Mike Lang ce9a8309a2 blogbot: Changes to integrate into wubloader repo 6 months ago
Mike Lang 2618cc076d blogbot changes to work with 2024 website 6 months ago
Mike Lang de52f7dbff Import blogbot from previous years 6 months ago
Mike Lang a0ca96aff6 download_media: Get imgur links working and add test script 6 months ago
Mike Lang c2ff2dfbb1 download_imgur_url: Clarify not handling i.imgur.com 6 months ago
Mike Lang 83f18eda84 download_media: Special casing for imgur 6 months ago
Mike Lang 4dbe1dce5e Use cache in readonly mode for PRs in forked repos 6 months ago
Mike Lang bff3fa7800 Refactor build script to actually have a working cache using buildah 
The old "docker build" no longer does caching the way it used to, and our cache logic doesn't work.

The new cache logic uses buildah, which is an alternate image build tool.
Buildah comes pre-installed on GHA.

When building, it pushes each layer as it goes to the cache repo.
It queries the repo for layers that are already built, so we don't need to explicitly pull
any specific tags and cache from them.

If caching is not enabled we still use docker as normal, so local development is not affected.
Local automatic caching will still apply.
 6 months ago
Hubbe 4238ab5d9d Use "latest" in workflow manifest 6 months ago
Hubbe f122ef0d95 Special-case "latest" in CACHE_FROM in build script 6 months ago
Christopher Usher b1f6d8d0de Fixed bug in authenticate_artist 6 months ago
Christopher Usher 1ba8597957 Made including a new image optional when updating a template 6 months ago
Hubbe 3ba01dda11 Don't check for active giveaways 
The old API is no more, so we can't check for active giveaway. Simply disable it for now.
 6 months ago
Mike Lang f2a6420634 pubbot: only listen on this year's channel 
the other one was last year's
 6 months ago
Mike Lang 177fad17e2 pubbot: listen on both "total" channels we've seen 
The difference? No idea.
 6 months ago
Thomas Chick f15dcaf545 fix message parsing bug with donation total 6 months ago
Mike Lang 211036a4d4 pubbot: try to get correct new url for db total 6 months ago
Dan Collins 3c746e569e Match aspect ratio from frame to template or vice versa, or lock 
template to frame
 6 months ago
Christopher Usher 2c5e1058e8 Added missing authentication decorators on calls that modify the 
templates database table
 6 months ago
Dan Collins 33f3b0e0c8 Attach validation to the first chapter marker description field 6 months ago
Christopher Usher 7f46b56ed4 Changed from editors to roles in the setup script 6 months ago
Christopher Usher 853a8f0ff4 Fixed SQL bug in the roles table 6 months ago
Mike Lang 713c0a6363 playlist-audit: Script to help with playlist auditing 
As per Sokar's request.
It's not entirely ideal that we're copying the "should be in playlists" criteria here
and in the playlist manager, but it's simple enough (is DONE, is public, matching upload location, tags are a superset of playlist tags) that it should be fine.
 6 months ago
Dan Collins 94aefc46fc Scale thumbnail pixel coordinates for a 1920x1080 video frame in a 1280x720 template 6 months ago
Mike Lang 39c7215910 No longer resize frames to template size before cropping 
This was originally done to ensure the crop settings worked no matter what the source resolution was,
but in practice the source resolution is stable (1080p) and the double-resize loses a lot of quality
if you actually want to scale *up* the cropped image.
 6 months ago
Dan Collins 952d760f8d Remove the :focus CSS rule, it obstructed the image making precise positioning difficult. 6 months ago
Dan Collins c3cb931e3e Download jcrop, fix newlines, use let instead of var 6 months ago
Dan Collins 1c5d37cc04 Thrimbletrimmer: Add JS cropping tool to advanced template options 6 months ago
Mike Lang b866b5d21c playlist manager: run more often 6 months ago
Mike Lang 9ad0f5f512 Pass db connect to restreamer 6 months ago
Christopher Usher a33b27dea3 Fixed error in thumbnail generation logging 6 months ago
Christopher Usher 8020df8e60 Logging for thumbnail template management 6 months ago
Mike Lang 2391a73ced thrimletrimmer: Support one-off overlays for thumbnails 
By picking the "one-off overlay" option for a thumbnail, you swap specifying a template name
for being able to upload a one-off template that is then combined with the requested frame.

The rendering is done by restreamer, and we do it explicitly whenever
a) Generate Thumbnail Preview is pressed
b) The video is submitted

The rendered thumbnail is then included in the submission as a "custom" thumbnail.

The default thumbnail template params (crop and location) do not change
when this mode is selected, so they'll effectively be the default params of the previously-selected
template. In most cases this will be what you want since almost all our templates share the same
params, and custom one-offs will too.
 6 months ago
Mike Lang 867ec8411b restreamer: Make get_thumbnail_uploaded_template() work 6 months ago
Mike Lang 751271d3c1 thrimbletrimmer: Replace ad-hoc query string building with URLSearchParams 
This handles edge cases like special characters properly, and is easier/more readable.
 6 months ago
Mike Lang f46481af0c thrimbletrimmer: Add support for advanced crop options 
- On load + change of template, set from template defaults
- On load, set from video if not null
- Show only in template mode
- Use when previewing image
- Send when submitting
 6 months ago