Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

grafana dashboard broken for SLOs with dots in the name #527

Open
problame opened this issue Oct 31, 2023 · 0 comments
Open

grafana dashboard broken for SLOs with dots in the name #527

problame opened this issue Oct 31, 2023 · 0 comments

Comments

@problame
Copy link

problame commented Oct 31, 2023

Problem

One of our SLOs has the name xyz-latency-10ms-99.99%.

The Grafana Dashboard linked from the README.md is broken for such SLOs.

Specifically, it sends queries like this

sloth_slo_info{sloth_service="pageserver", sloth_slo="xxx-latency-10ms-99\\.99%"}

because the underlying query (quoting from the dashboard JSON, hence double-escaping)

          "expr": "sloth_slo_info{sloth_service=\"${service}\", sloth_slo=\"${slo}\"}",

Root-Cause

Before queries are sent to your data source the query is interpolated, meaning the variable is replaced with its current value. During interpolation, the variable value might be escaped in order to conform to the syntax of the query language and where it is used. For example, a variable used in a regex expression in an InfluxDB or Prometheus query will be regex escaped. Read the data source specific documentation topic for details on value escaping during interpolation.

https://grafana.com/docs/grafana/latest/dashboards/variables/variable-syntax/

Solution

Use the regex label filter =~ instead of a plain =.

          "expr": "sloth_slo_info{sloth_service=~\"${service}\", sloth_slo=~\"${slo}\"}",

I have done the work already to fix the dashboard JSON, we're using it internally.

I don't know what the process is to make a PR for the dashboard.

IMO it would be good if the JSON is versioned in this repository, instead of just being on Grafana.

@problame problame changed the title grafana dashboard doesn't use correct interpolation grafana dashboard broken for SLOs with dots in the name Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant