Is it possible to add capability to collect metrics from selected indices

[jacmath]

Hi,
Currently there is only an option to disable the metric collection for indices as a whole using the setting prometheus.indices: false. Is it possible to add support to collect metrics only for a limited set of indices?

Once the number of indices grow above 50 or so, we are seeing metrics collection is taking long resulting in gaps in the metrics scraped by Prometheus. In such cases, it would be nice to be able to monitor metrics for a few critical indices instead of disabling it completely.

Thanks
Jacob Mathews

[lukas-vlcek]

Hello Jacob,

this is actually very good point. Right now it is only possible to enable or disable indices metrics. But as you point out this is far from ideal. Actually, prometheus can have problems (label explosion) when number of indices grow beyond certain level as well.

How would you like to specify which indices the metrics should be collected from?

[jacmath]

Hi Lukas,
Thanks for looking into this request.
Is it possible to supply the list of indices for which metrics need to be collected as a string with comma-separated list of index names (possibly with wildcards).
For example,
"logstash*,events*,.kibana_1"

Regards
Jacob Mathews

[lukas-vlcek]

Hi Jacob,
this means we will need to introduce a new configuration field into elasticsearch.yml that will hold this string (or an array that will be turned into this string). I can imagine this could work. I can even imagine we can make this field dynamically updatable so that users will have the option to change the list of indices on the fly.
WDYT?
Lukáš

[jacmath]

Yes, this looks like a very good proposal.

[jacmath]

Hi Lukas,
Any chance of getting this capability added to the exporter?
Thanks
Jacob Mathews