Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up default format of alerts (if possible ;) #46

Open
mrjones-plip opened this issue May 12, 2023 · 2 comments
Open

Clean up default format of alerts (if possible ;) #46

mrjones-plip opened this issue May 12, 2023 · 2 comments
Assignees

Comments

@mrjones-plip
Copy link
Contributor

mrjones-plip commented May 12, 2023

Now that we've enabled alerts (#14), it turns out they're very verbose.

This is what is seen right now:

[FIRING:28] DB Fragmentation CHT (cht)
FiringValue: B=27.055714740528494, C=1
Labels:
- alertname = DB Fragmentation
- db = _users
- grafana_folder = CHT
- instance = INSTANCE-HERE.medicmobile.org
- job = cht
Annotations:
- description = The [_users] database for the CHT Server [INSTANCE-HERE.medicmobile.org] is highly fragmented.
Source: https://allies-monitoring-alerting.dev.medicmobile.org/alerting/grafana/ot6lYCYVz/view?orgId=1
Silence: https://allies-monitoring-alerting.dev.medicmobile.org/alerting/silence/new?alertmanager=grafana&matcher=alertname%3DDB+Fragmentation&matcher=db%3D_users&matcher=grafana_folder%3DCHT&matcher=instance%3DINSTANCE-HERE.medicmobile.org&matcher=job%3Dcht
Dashboard: https://allies-monitoring-alerting.dev.medicmobile.org/d/oa2OfL-Vk?orgId=1
Panel: https://allies-monitoring-alerting.dev.medicmobile.org/d/oa2OfL-Vk?orgId=1&viewPanel=13

And it'd be great if we greatly condense it and remove unused info:

  • make all Long URLs markdown/HTML links for Source, Silence, Dashboard and Panel
  • remove db - _users
  • remove grafana_folder = CHT
  • remove instance = INSTANCE-HERE.medicmobile.org
  • remove empty Annotations
  • remove title for description =

Like so:

[FIRING:28] DB Fragmentation: [_users] database on INSTANCE-HERE.dev.medicmobile.org.

Value: 27.055714740528494, Alert threshold: 1

Links: Source - Silence - Dashboard - Panel

@mrjones-plip
Copy link
Contributor Author

This post might help.

@mrjones-plip mrjones-plip changed the title Clean up default alerts if possible Clean up default format of alerts (if possible ;) May 12, 2023
@eljhkrr eljhkrr self-assigned this May 6, 2024
@eljhkrr
Copy link
Member

eljhkrr commented May 10, 2024

Notifications reduced to single-line messages:
image

TODO: Default message formatting needs to be upstreamed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants