henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
726 Commits
34 Branches
0 Tags
19 MiB
Commit Graph

30 Commits (d5b5b012bff03b0e3374219fdc3ec8c1a02e3ac6)

Author SHA1 Message Date
Mike Lang 3a19ba744d segment_coverage: Raise default check interval to 5min
Mike Lang 5235c3281a segment_coverage: Allow setting of check interval via cli flag
Mike Lang 15c357509f segment_coverage: Fix a problem where metrics would fail 
Because the checking process is entirely CPU-bound, it does not give any other
greenlets a chance to run while it is processing. This prevents us from responding
to metrics queries, and prometheus then times out.

By stopping to handle all other traffic in between each hour processed, we ensure metrics
remain responsive while processing.
HubbeKing 86f7823348 Replace calls to gevent.signal() with gevent.signal_handler() 
gevent.signal() was removed in gevent 1.5a4, see http://www.gevent.org/api/gevent.signal.html
Removed on Feb 5th, see https://github.com/gevent/gevent/pull/1530
Mike Lang a53786dc2d Add file and make as build dependencies 
gevent now requires these to build. I'm not sure when this changed.
Christopher Usher d56801014b added support for suspect segments to segment_coverage
Christopher Usher 9c77dd1f40 added the ability to generate a webpage with all coverage maps
Mike Lang 967ac7b856 segment_coverage: Reduce "no hours" warning to info 
This is too noisy at warning level, and comes up for non-main channels.
Mike Lang 6724027e5a segment_coverage: Reduce missing channel/quality to warning
Mike Lang 731ef9e2d0 Refactor dockerfiles for more shared layers 
By carefully ensuring most of our dockerfiles are identical in their first few layers,
we only need to build those layers once instead of every time.

In particular, we move installing gevent to before installing common,
so that even when common changes gevent doesn't need to be reinstalled.

This is important because gevent takes ages to install.

Also fixes segment_coverage, which wasn't being installed.
Christopher Usher 28ef77b5a7 error handling changes as suggested by ekim
Christopher Usher 34e8d0a64b Handle the case where an hour directory disappears between listing the 
hours and trying to list the segments in that hour. This could happen if the
backfiller is deleting old hours.
Christopher Usher 3207dd6878 Fixed temporary segments crashing segment coverage
Christopher Usher 49ccb6df86 warns when a directory does not exist or when there are no hours to make 
a map from
Christopher Usher 3130e770c8 made while loop more pythonic
Christopher Usher a1880b2414 fixes based on ekim's suggestions
Christopher Usher bea876e0cc removed obsolete code
Christopher Usher 43e19c3c56 removed unneeded package
Christopher Usher 44390173ed comments, code style and better handling of empty hours
Christopher Usher 003261eae4 Promethous gauges and new style coverage plots
Christopher Usher 46b7c7a3b6 new plotting
Christopher Usher 9711dbab0e changing what I mean by overlap
Christopher Usher 8e79ac772a started on the guages
Christopher Usher 92a4cf0d7b bit of a clean up
Christopher Usher 20a8a214d6 working!
Christopher Usher ac72f775c9 functional
Christopher Usher 722cbd20fa first pass at checking for holes and repeats
Christopher Usher 66f5a06a5c basic segment counting working
Christopher Usher 3618510f35 basic functionality
Christopher Usher 929308f3e7 started on the segment_coverage service