-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add metric for command processor latency #1501
Comments
This metric may cause a high cardinality problem, so I got two suggestions from Frank Rosner:
After some investigations for these two suggestions, I found:
|
In PR #1029 we removed the histograms from the
command_processor_process
metrics because of the extremely high cardinality that was causing problems for Grafana dashboards and metrics forwarding.The high cardinality was caused by the multiplicity of tags we supported including
command
,tenant
,sort type
,error
,error class
,error_code
, andvector_enabled
. When adding histogram buckets to this, it potentially results in a huge number of series.However it would still be useful to be able to track latency by command for debugging purposes. We should add a new metric called
command_processor_latency
that is a histogram metric tagged bycommand
andtenant
only.The text was updated successfully, but these errors were encountered: