Skip to content

Commit

Permalink
[DOCS-9246] Add group by explanation to generate metrics processor (#…
Browse files Browse the repository at this point in the history
…25923)

* add group by explanation

* Update layouts/shortcodes/observability_pipelines/processors/generate_metrics.en.md

* add examples and remove duplicate info

* remove examples from metrics types shortcode

* add anchor links
  • Loading branch information
maycmlee authored Oct 31, 2024
1 parent a73cb94 commit 544fdf6
Show file tree
Hide file tree
Showing 2 changed files with 44 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,4 @@ disable_toc: false

{{% observability_pipelines/processors/generate_metrics %}}

##### Metrics Types

{{% observability_pipelines/metrics_types %}}

{{% observability_pipelines/processors/filter_syntax %}}
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,10 @@ Click **Manage Metrics** to create new metrics or edit existing metrics. This op

1. Enter a [filter query](#filter-query-syntax). Only logs that match the specified filter query are processed. All logs, regardless of whether they match the filter query, are sent to the next step in the pipeline. **Note**: Since a single processor can generate multiple metrics, you can define a different filter query for each metric.
1. Enter a name for the metric.
1. In the **Define parameters** section, select the metric type (count, gauge, or distribution). See [Metrics Types](#metrics-types) for more information.
1. In the **Define parameters** section, select the metric type (count, gauge, or distribution). See the [Count metric example](#count-metric-example) and [Distribution metric example](#distribution-metric-example). Also see [Metrics Types](#metrics-types) for more information.
- For gauge and distribution metric types, select a log field which has a numeric (or parseable numeric string) value that is used for the value of the generated metric.
- For the distribution metric type, the log field's value can be an array of (parseable) numerics, which is used for the generated metric's sample set.
- The **Group by** field determines how the metric values are grouped together. For example, if you have hundreds of hosts spread across four regions, grouping by region allows you to graph one line for every region. The fields listed in the **Group by** setting are set as tags on the configured metric.
1. Click **Add Metric**.

##### Metrics Types
Expand All @@ -28,7 +29,48 @@ You can generate these types of metrics for your logs. See the [Metrics Types][1
| GAUGE | Represents a snapshot of events in one time interval. | You want to measure the latest CPU utilization per host for all logs in the production environment. |
| DISTRIBUTION | Represent the global statistical distribution of a set of values calculated across your entire distributed infrastructure in one time interval. | You want to measure the average time it takes for an API call to be made. |

##### Count metric example

For this `status:error` log example:

```
{"status": "error", "env": "prod", "host": "ip-172-25-222-111.ec2.internal"}
```

To create a count metric that counts the number of logs that contain `"status":"error"` and groups them by `env` and `host`, enter the following information:

| Input parameters | Value |
|------------------|---------------------|
| Filter query | `@status:error` |
| Metric name | `status_error_total`|
| Metric type | Count |
| Group by | `env`, `prod` |

##### Distribution metric example

For this example of an API response log:

```
{
"timestamp": "2018-10-15T17:01:33Z",
"method": "GET",
"status": 200,
"request_body": "{"information"}",
"response_time_seconds: 10
}
```

To create a distribution metric that measures the average time it takes for an API call to be made, enter the following information:

| Input parameters | Value |
|------------------------|-------------------------|
| Filter query | `@method` |
| Metric name | `status_200_response` |
| Metric type | Distribution |
| Select a log attribute | `response_time_seconds` |
| Group by | `method` |

[10031]: /metrics/custom_metrics/
[10032]: /account_management/billing/custom_metrics/
[10033]: /metrics/types/
[10034]: /metrics/distributions/
[10034]: /metrics/distributions/

0 comments on commit 544fdf6

Please sign in to comment.