docs(spec): Add specification for output rate limiting

[srebhan]

Summary

Propose a new rate-limiting feature for output plugins allowing to limit the metric-rate sent to services.

Checklist

• No AI generated code was used in this PR

Related issues

related to #14802

[powersj]

My only concern is that in many cases a cloud or database or service will put limits in terms of "MB per second". Here is some prior art from InfluxDB: influxdata/influxdb#19660

The logical next question from a user is "how do I know how many metrics translate to N MB per second".

[srebhan]

I think this must be done in the output plugin as the message size depends on the serialization of the metrics and this is NOT known to the framework. The proposed solution is to limit the peak sends of metrics, not about server-side rate limiting.

[powersj]

I think this must be done in the output plugin as the message size depends on the serialization of the metrics and this is NOT known to the framework.

I have no disagreement with this and certainly wasn't implying moving this out of the output plugin.

The proposed solution is to limit the peak sends of metrics, not about server-side rate limiting.

Correct, but one of the reasons people want this option is the service they are sending to will have limits as to how much it can accept. For example, using the influxdb output, we know some free plans are limited to 17 kb/s. How can a user take that limit, and apply it their telegraf config?

I don't see customers or users asking for "limit to N metrics per second". Rather it is a data rate per second.

[powersj]

Maybe I am getting this convolluted by trying to solve two things at once:

1. the scenario where telegraf reconnects to an output, and then sends a DOS-like set of metric batches to "catch up"
2. the scenario where telegraf sends too many metrics over time to an output

Your proposal is clearly tackling the first, but I see users grabbing this option to also tackle 2 as well.

Can we consider 2 to avoid yet another option.