Commit Graph

30 Commits (condor-scripts)

Author SHA1 Message Date
Mike Lang 3a19ba744d segment_coverage: Raise default check interval to 5min 4 years ago
Mike Lang 5235c3281a segment_coverage: Allow setting of check interval via cli flag 4 years ago
Mike Lang 15c357509f segment_coverage: Fix a problem where metrics would fail
Because the checking process is entirely CPU-bound, it does not give any other
greenlets a chance to run while it is processing. This prevents us from responding
to metrics queries, and prometheus then times out.

By stopping to handle all other traffic in between each hour processed, we ensure metrics
remain responsive while processing.
4 years ago
HubbeKing 86f7823348 Replace calls to gevent.signal() with gevent.signal_handler()
gevent.signal() was removed in gevent 1.5a4, see http://www.gevent.org/api/gevent.signal.html
Removed on Feb 5th, see https://github.com/gevent/gevent/pull/1530
4 years ago
Mike Lang a53786dc2d Add file and make as build dependencies
gevent now requires these to build. I'm not sure when this changed.
4 years ago
Christopher Usher d56801014b added support for suspect segments to segment_coverage 5 years ago
Christopher Usher 9c77dd1f40 added the ability to generate a webpage with all coverage maps 5 years ago
Mike Lang 967ac7b856 segment_coverage: Reduce "no hours" warning to info
This is too noisy at warning level, and comes up for non-main channels.
5 years ago
Mike Lang 6724027e5a segment_coverage: Reduce missing channel/quality to warning 5 years ago
Mike Lang 731ef9e2d0 Refactor dockerfiles for more shared layers
By carefully ensuring most of our dockerfiles are identical in their first few layers,
we only need to build those layers once instead of every time.

In particular, we move installing gevent to before installing common,
so that even when common changes gevent doesn't need to be reinstalled.

This is important because gevent takes ages to install.

Also fixes segment_coverage, which wasn't being installed.
5 years ago
Christopher Usher 28ef77b5a7 error handling changes as suggested by ekim 5 years ago
Christopher Usher 34e8d0a64b Handle the case where an hour directory disappears between listing the
hours and trying to list the segments in that hour. This could happen if the
backfiller is deleting old hours.
5 years ago
Christopher Usher 3207dd6878 Fixed temporary segments crashing segment coverage 5 years ago
Christopher Usher 49ccb6df86 warns when a directory does not exist or when there are no hours to make
a map from
5 years ago
Christopher Usher 3130e770c8 made while loop more pythonic 5 years ago
Christopher Usher a1880b2414 fixes based on ekim's suggestions 5 years ago
Christopher Usher bea876e0cc removed obsolete code 5 years ago
Christopher Usher 43e19c3c56 removed unneeded package 5 years ago
Christopher Usher 44390173ed comments, code style and better handling of empty hours 5 years ago
Christopher Usher 003261eae4 Promethous gauges and new style coverage plots 5 years ago
Christopher Usher 46b7c7a3b6 new plotting 5 years ago
Christopher Usher 9711dbab0e changing what I mean by overlap 5 years ago
Christopher Usher 8e79ac772a started on the guages 5 years ago
Christopher Usher 92a4cf0d7b bit of a clean up 5 years ago
Christopher Usher 20a8a214d6 working! 5 years ago
Christopher Usher ac72f775c9 functional 5 years ago
Christopher Usher 722cbd20fa first pass at checking for holes and repeats 5 years ago
Christopher Usher 66f5a06a5c basic segment counting working 5 years ago
Christopher Usher 3618510f35 basic functionality 5 years ago
Christopher Usher 929308f3e7 started on the segment_coverage service 5 years ago