Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify usage of otelcol.processor.discovery #5378

Merged
merged 3 commits into from
Oct 11, 2023
Merged

Clarify usage of otelcol.processor.discovery #5378

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1c305df
Clarify usage of otelcol.processor.discovery
ptodev Oct 5, 2023
271c4d9
Merge branch 'main' into 5306-otel-processor-discovery-component-clar…
clayton-cornell Oct 5, 2023
126031b
Update docs/sources/flow/reference/components/otelcol.processor.disco…
ptodev Oct 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 24 additions & 2 deletions component/otelcol/processor/discovery/discovery_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,8 @@ func Test_Insert(t *testing.T) {
targets = [{
"__address__" = "1.2.2.2",
"__internal_label__" = "test_val",
"test_label" = "test_val2"}]
"test_label" = "test_val2",
"test.label.with.dots" = "test.val2.with.dots"}]

operation_type = "insert"

Expand All @@ -169,6 +170,10 @@ func Test_Insert(t *testing.T) {
{
"key": "test_label",
"value": { "stringValue": "old_val" }
},
{
"key": "test.label.with.dots",
"value": { "stringValue": "old_val" }
}]
},
"scopeSpans": [{
Expand Down Expand Up @@ -210,6 +215,10 @@ func Test_Insert(t *testing.T) {
{
"key": "test_label",
"value": { "stringValue": "old_val" }
},
{
"key": "test.label.with.dots",
"value": { "stringValue": "old_val" }
}]
},
"scopeSpans": [{
Expand All @@ -231,6 +240,10 @@ func Test_Insert(t *testing.T) {
{
"key": "test_label",
"value": { "stringValue": "test_val2" }
},
{
"key": "test.label.with.dots",
"value": { "stringValue": "test.val2.with.dots" }
}]
},
"scopeSpans": [{
Expand All @@ -253,7 +266,8 @@ func Test_Update(t *testing.T) {
targets = [{
"__address__" = "1.2.2.2",
"__internal_label__" = "test_val",
"test_label" = "test_val2"}]
"test_label" = "test_val2",
"test.label.with.dots" = "test.val2.with.dots"}]

operation_type = "update"

Expand All @@ -277,6 +291,10 @@ func Test_Update(t *testing.T) {
{
"key": "test_label",
"value": { "stringValue": "old_val" }
},
{
"key": "test.label.with.dots",
"value": { "stringValue": "old_val" }
}]
},
"scopeSpans": [{
Expand Down Expand Up @@ -318,6 +336,10 @@ func Test_Update(t *testing.T) {
{
"key": "test_label",
"value": { "stringValue": "test_val2" }
},
{
"key": "test.label.with.dots",
"value": { "stringValue": "test.val2.with.dots" }
}]
},
"scopeSpans": [{
Expand Down
44 changes: 36 additions & 8 deletions docs/sources/flow/reference/components/otelcol.processor.discovery.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,38 @@ of labels for each discovered target.
`otelcol.processor.discovery` adds resource attributes to spans which have a hostname
matching the one in the `__address__` label provided by the `discovery.*` component.

> **NOTE**: `otelcol.processor.discovery` is a custom component unrelated to any
> processors from the OpenTelemetry Collector.
{{% admonition type="note" %}}
`otelcol.processor.discovery` is a custom component unrelated to any
processors from the OpenTelemetry Collector.
{{% /admonition %}}

Multiple `otelcol.processor.discovery` components can be specified by giving them
different labels.

{{% admonition type="note" %}}
It can be difficult to follow [OpenTelemetry semantic conventions][OTEL sem conv] when
adding resource attributes via `otelcol.processor.discovery`:
* `discovery.relabel` and most `discovery.*` processes such as `discovery.kubernetes`
can only emit [Prometheus-compatible labels][Prometheus data model].
* Prometheus labels use underscores (`_`) in labels names, whereas
[OpenTelemetry semantic conventions][OTEL sem conv] use dots (`.`).
* Although `otelcol.processor.discovery` is able to work with non-Prometheus labels
such as ones containing dots, the fact that `discovery.*` components are generally
only compatible with Prometheus naming conventions makes it hard to follow OpenTelemetry
semantic conventions in `otelcol.processor.discovery`.

If your use case is to add resource attributes which contain Kubernetes metadata,
consider using `otelcol.processor.k8sattributes` instead.

------
The main use case for `otelcol.processor.discovery` is for users who migrate to Grafana Agent Flow mode
from Static mode's `prom_sd_operation_type`/`prom_sd_pod_associations` [configuration options][Traces].

[Prometheus data model]: https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels
[OTEL sem conv]: https://github.com/open-telemetry/semantic-conventions/blob/main/docs/README.md
[Traces]: {{< relref "../../../static/configuration/traces-config.md" >}}
{{% /admonition %}}

## Usage

```river
Expand Down Expand Up @@ -146,16 +172,18 @@ otelcol.processor.discovery "default" {

### Using a preconfigured list of attributes

It is not necessary to use a discovery component. In the example below, a `test_label` resource
attribute will be added to a span if its IP address is "1.2.2.2". The `__internal_label__` will
be not be added to the span, because it begins with a double underscore (`__`).
It is not necessary to use a discovery component. In the example below, both a `test_label` and
a `test.label.with.dots` resource attributes will be added to a span if its IP address is
"1.2.2.2". The `__internal_label__` will be not be added to the span, because it begins with
a double underscore (`__`).

```river
otelcol.processor.discovery "default" {
targets = [{
"__address__" = "1.2.2.2",
"__internal_label__" = "test_val",
"test_label" = "test_val2"}]
"__address__" = "1.2.2.2",
"__internal_label__" = "test_val",
"test_label" = "test_val2",
"test.label.with.dots" = "test.val2.with.dots"}]

output {
traces = [otelcol.exporter.otlp.default.input]
Expand Down