henine
/
wubloader
mirror of https://github.com/ekimekim/wubloader
1
0
Fork 0
Code Issues Releases Wiki Activity
1,133 Commits
34 Branches
0 Tags
19 MiB
Commit Graph

12 Commits (be7a9cd1fae20895a2b1a8d831fbde8a0b1bd21c)

Author SHA1 Message Date
Mike Lang 62bd6539ea Unpin gevent as that was a workaround for a py2 issue
Mike Lang f0546e2ee3 Pin gevent to 1.5a2 to avoid https://github.com/gevent/gevent/issues/1711
Christopher Usher 76bc629720 moved flask monitoring to its own module
Christopher Usher 6c633df3ee move restreamer.stats to common.stats
Mike Lang bf08aa29b8 parse_segment_path: Use datetime.strptime instead of dateutil.parser 
strptime is much faster but can't handle as varied formats.
But in this case we fully control the format, so there's no reason not to use it.

Profiling suggests we spend about 80% of our time in get_best_segments just parsing dates,
so this is a signifigant performance gain.
Mike Lang 10cca18922 Fix a deadlock due to signal interactions with prometheus client 
The prometheus client uses a threading.Lock() to prevent shared access to
certain metric state. This lock is taken as part of doing collection, as well
as during metric.labels().

We hit a deadlock where our stack sampler signal arrived during a collection,
when the lock was held. This meant that flamegraph.labels() blocked forever,
and the lock was never released, hanging all metrics collection.

Our solution is a hack, which is to reach into the internals of our metric object
and replace its lock with a dummy one. This is reasonably safe, but only as long as
the prometheus_client internal structure doesn't change signfigiantly.
Mike Lang 8fe2fec958 common: convert from module to package
Mike Lang d90f01b8ce common: Create general function for timing things, and use it to time get_best_segments 
The function is quite customizable and therefore quite complex, but it allows us to
easily annotate a function to be timed with labels based on input and output,
as well as normalize results based on amount of work done to get a better
picture of the actual amount of time taken per unit of work.
This will help us monitor for performance issues.
Mike Lang b0ded641c3 Add a logging handler which counts logs for prometheus stats 
This isn't as good as having a full centralised logging system, but should
suffice to know if anything funny is happening.
Mike Lang d7641aecf5 common: Fix bugs and issues with bustime utils
Mike Lang 048277b003 common: Basic config and bustime code
Mike Lang a361ab7a63 Add a common package for common bits in multiple components