[DOC] Update overrides content in Helm chart to clarify configuration for new, legacy

[knylander-grafana]

Update the overrides configuration documentation to address changes to the Overrides module. The configuration examples need to be checked to make sure they are for the updated overrdies and which ones are not.

Update these pages:

• https://grafana.com/docs/tempo/latest/release-notes/v2-3/#changes-to-the-overrides-module-configuration

The configuratio file in this doc works, but it's a YAML file:
"A snippet of a config.yaml file showing how the overrides section is here.

Reference: #3795, grafana/helm-charts#2802 (comment)

• You need per-tenant overrides. Same format in both overrides configs
• How does Tempo tell which configuration is new and which one is legacy?

Notes

Same issue happens in the tempo-distributed Helm chart:

global_overrides:
  defaults:
    metrics_generator:
      processors:
        - span-metrics
      processor:
        span_metrics:
          histogram_buckets: [null]
          dimensions:
            - kubernetes_cluster
            - kubernetes.cluster
            - http.status_code

As a rule of thumb: long, unindented params are legacy (eg. metrics_generator_forwarder_queue_size), indented params are new (eg. metrics_generator.forwarder.queue_size).

With the new config, a new key defaults was also added after the overrides key, to be more explicit of the contents of the config block.

Finally, there is a command in the tempo-cli that allows you to convert legacy config to new overrides config.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days.
The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity.
Please apply keepalive label to exempt this Issue.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity in the past 60 days.
The next time this stale check runs, the stale label will be removed if there is new activity. The issue will be closed after 15 days if there is no new activity.
Please apply keepalive label to exempt this Issue.

[knylander-grafana]

Mario found an issue in the tempo-distributed Helm chart that prevented the overrides examples from working. The issue was corrected in this PR: grafana/helm-charts#3236

[knylander-grafana]

There is an issue in the Helm chart that needs to be fixed. Outlined here: grafana/helm-charts#2802 (comment)

The tempo-distributed Helm chart doesn't have the same implementation of the overrides as the standard Tempo configuration. The per tenant overrides is being implemented with the global overrides. The docs are correct, but how the overrides are implemented in the Helm chart is slightly different. We should update the tempo-distributed Helm chart documentation with a work around until this

[knylander-grafana]

The README needs to have these updates:

• Explain what the overrides values are in the table
• Update the S3 example with the correct overrides

[knylander-grafana]

Related Helm chart PRs that we might want to check:

• [tempo-distributed] Added LegacyOverrides for processors helm-charts#3463

[knylander-grafana]

• [tempo-distributed] Clarifies use of global_overrides and overrides objects. helm-charts#3468
• [DOC] Add overrides info to tempo-distributed doc #4415