#metabrainz

zas: Hello! Hope you are not too busy battling AI vermin, if you have a moment I have a question regarding a Grafana alert.... (full message at <https://matrix.chatbrainz.org/_matrix/media/v3/...>)

Why doesn't it provide data sometimes? I mean, if the reason is the service doesn't run we expect an alert, right?

Transient issues that resolve themselves

We have other alerts for non-responding website and API, so I'm not expecting these stats alerts to trigger when connection is temporarily lost.

After a delay of 10 minutes, then yes it would make sense to trigger.

There's already a 5m pending period for this alert, we can increase this

Ah, yes please!

Is that a 5m delay on triggering the alert in any case, or just for loss of data (just out of curiosity)

10m or more?

Any contributors who are cooking a GSoC proposal towards ListenBrainz Android project should take a look at ideas page again. Eased some things and added more context.

Any alert for this alert rule

We can't dissociate them

That's what I figured. Thanks for confirming

To me this reeads as a 1m delay, not 5. Am I looking at the wrong thing?

That's the time the rule is evaluated, but the pending period is the time before it actually alerts (if the state is the same)

So basically it evaluates the state every minute, but wait for 5 minutes to see if it was transient or not

It limits false alerts

We can evaluate the state less often too

but then the pending period has to be longer

I was lookign at the pending period, it seemed to me to be set to 1m when I opened the alert edit page

If you look carefully Grafana shows alerts as activated as "Pending" when it happens, notifications are sent at the end of the pending period

s/as//

Anyway, thanks for the assist. Let's see how it goes with 5m delay.

Wait, which alert did you copied from? because I have 5m for stats

<zas[m]> "There's already a 5m pending..." <- This is the bit I was confused about. I can only see 1m pending period, nothing that says it was set to 5m

I was on https://stats.metabrainz.org/alerting/debdj8a19...

I see, those are thesitewide

You can change pending period to 5m for it then

OK, and it wasn't configured with the same delay. Mystery solved :)

I'll change that for the sitewide stats alerts

Then I think it will work fine, the other (non-sitewide) alerts have been behaving better

The pending period limits the number of notifications if states change too quickly$

s/$/./

Yep, that's what I was looking for.

Thank you!

np :)