Alert on sustained CPU usage

[FiveElementNinja]

Apologies if this is already possible, but I was wondering if we could send alerts on sustained CPU usage. Some of my containers spike during normal operation and I get an alert every time, but I really only want to know if they stay above a certain percentage for a period of time.

If that's not possible, no worries. Just thought I'd ask.

[amir20]

Not possible. But if I wanted to implement it, how would you propose? Maybe a moving average or something? Docker reports per second so what would it mean period of time?

[Jurrer]

yes, a configurable threshold duration is very nice to have and will reduce alert spam.

I would like to suggest another option - percentage of duration, meaning if the threshold is breached for X percent of the time window, alert will trigger.
Coralogix has such option - for_over_pct

[amir20]

@Jurrer do you think the average user would understand that? I think I want it to be simple but yet powerful. What would the UI say? Most people don't even know I hold 5 minutes of data. Perhaps default at leas 30 seconds of the last 5 minutes and make that the default. Then introduce a parameter as needed.

[Jurrer]

okay, if the goal is simplicity it could be just optional parameter like "sensitive threshold" that would be triggering instantly and a default threshold like you suggested

[amir20]

I ended up doing something similar

#4498

Try it out. amir20/dozzle:pr-4498.

The idea is to chose the time window. Eg 30s. I check for at least 80% match. That's hard coded for now to reduce the numbers of parameters. Defaults to 15 seconds.

[amir20]

OK merging. Haven't heard back 🤷