Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Surface change alert guide to monitor types docs #23096

Merged
merged 4 commits into from
May 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 22 additions & 17 deletions config/_default/menus/main.en.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1354,66 +1354,71 @@ menu:
parent: monitor_types
identifier: monitor_types_auditlogs
weight: 205
- name: Change
url: monitors/types/change-alert/
parent: monitor_types
identifier: monitor_types_change
weight: 206
- name: CI
url: monitors/types/ci/
parent: monitor_types
identifier: monitor_types_ci
weight: 206
weight: 207
- name: Cloud Cost
url: monitors/types/cloud_cost/
parent: monitor_types
identifier: monitor_types_cloud_cost
weight: 207
weight: 208
- name: Composite
url: monitors/types/composite/
parent: monitor_types
identifier: monitor_types_composite
weight: 208
weight: 209
- name: Database Monitoring
url: monitors/types/database_monitoring/
parent: monitor_types
identifier: monitor_types_database_monitoring
weight: 209
weight: 210
- name: Error Tracking
url: monitors/types/error_tracking/
parent: monitor_types
identifier: monitor_types_error_tracking
weight: 210
weight: 211
- name: Event
url: monitors/types/event/
parent: monitor_types
identifier: monitor_types_event
weight: 211
weight: 212
- name: Forecast
url: monitors/types/forecasts/
parent: monitor_types
identifier: monitor_types_forecasts
weight: 212
weight: 213
- name: Integration
url: monitors/types/integration/
parent: monitor_types
identifier: monitor_types_integration
weight: 213
weight: 214
- name: Live Process
url: monitors/types/process/
parent: monitor_types
identifier: monitor_types_process
weight: 214
weight: 215
- name: Logs
url: monitors/types/log/
parent: monitor_types
identifier: monitor_types_log
weight: 215
weight: 216
- name: Network
url: monitors/types/network/
parent: monitor_types
identifier: monitor_types_network
weight: 216
weight: 217
- name: Network Performance
url: monitors/types/network_performance/
parent: monitor_types
identifier: monitor_types_network_performance
weight: 217
weight: 218
- name: Outlier
url: monitors/types/outlier/
parent: monitor_types
Expand All @@ -1423,27 +1428,27 @@ menu:
url: monitors/types/process_check/
parent: monitor_types
identifier: monitor_types_process_check
weight: 219
weight: 220
- name: Real User Monitoring
url: monitors/types/real_user_monitoring/
parent: monitor_types
identifier: monitor_types_rum
weight: 220
weight: 221
- name: Service Check
url: monitors/types/service_check/
parent: monitor_types
identifier: monitor_types_service_check
weight: 221
weight: 222
- name: SLO Alerts
url: monitors/types/slo/
parent: monitor_types
identifier: monitor_types_slo
weight: 222
weight: 223
- name: Watchdog
url: monitors/types/watchdog/
parent: monitor_types
identifier: monitor_types_watchdog
weight: 223
weight: 224
- name: Notifications
url: monitors/notify/
parent: alerting
Expand Down
2 changes: 1 addition & 1 deletion content/en/dashboards/widgets/change.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,4 +66,4 @@ This widget can be used with the **[Dashboards API][2]**. See the following tabl
[2]: /api/latest/dashboards/
[3]: /dashboards/graphing_json/widget_json/
[6]: /monitors/types/metric/?tab=change
[7]: /monitors/guide/change-alert/
[7]: /monitors/types/change-alert/
1 change: 0 additions & 1 deletion content/en/monitors/guide/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ cascade:
{{< nextlink href="synthetics/guide/synthetic-test-monitors" >}}How to create monitors in synthetic tests{{< /nextlink >}}
{{< nextlink href="monitors/guide/non_static_thresholds" >}}How to monitor non-static thresholds{{< /nextlink >}}
{{< nextlink href="monitors/guide/anomaly-monitor" >}}Anomaly monitors{{< /nextlink >}}
{{< nextlink href="monitors/guide/change-alert" >}}Change Alert monitors{{< /nextlink >}}
{{< nextlink href="monitors/guide/monitor-ephemeral-servers-for-reboots" >}}Monitor ephemeral servers for reboots{{< /nextlink >}}
{{< nextlink href="monitors/guide/how-to-update-anomaly-monitor-timezone" >}}How to update an anomaly detection monitor to account for local timezone{{< /nextlink >}}
{{< nextlink href="monitors/guide/history_and_evaluation_graphs" >}}Monitor History and Evaluation Graph{{< /nextlink >}}
Expand Down
1 change: 1 addition & 0 deletions content/en/monitors/types/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ further_reading:
{{< nextlink href="/monitors/types/anomaly" >}}<strong>Anomaly</strong>: Detect anomalous behavior for a metric based on historical data.{{< /nextlink >}}
{{< nextlink href="/monitors/types/apm" >}}<strong>APM</strong>: Monitor APM metrics or trace queries.{{< /nextlink >}}
{{< nextlink href="/monitors/types/audit_trail" >}}<strong>Audit Trail</strong>: Alert when a specified type of audit log exceeds a user-defined threshold over a given period of time.{{< /nextlink >}}
{{< nextlink href="/monitors/types/change-alert" >}}<strong>Change Alert</strong>: Alert when the absolute or relative value changes against a user-defined threshold over a given period of time.{{< /nextlink >}}
{{< nextlink href="/monitors/types/ci" >}}<strong>CI</strong>: Monitor CI pipelines and tests data gathered by Datadog.{{< /nextlink >}}
{{< nextlink href="/monitors/types/cloud_cost" >}}<strong>Cloud Cost</strong>: Monitor cost changes associated with cloud platforms.{{< /nextlink >}}
{{< nextlink href="/monitors/types/composite" >}}<strong>Composite</strong>: Alert on an expression combining multiple monitors.{{< /nextlink >}}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
---
title: Change alert monitors
kind: Guide
title: Change Alert Monitor
kind: Documentation
disable_toc: false
aliases:
- monitors/guide/change-alert
further_reading:
- link: "/monitors/types/metric/?tab=change#choose-the-detection-method"
tag: "Documentation"
Expand All @@ -23,11 +25,15 @@ Here is a breakdown of how monitors with the change detection method work:
1. Aggregation is applied over the query in (3) which returns a single value.
1. The threshold defined in **Set alert conditions** is compared to the single value returned in (4).

## Monitor creation

To create a [Change Alert monitor][9] in Datadog, use the main navigation: *Monitors --> New Monitor --> Change*.

## Evaluation conditions

Here are the different options that you need to configure in a change alert monitor.

{{< img src="/monitors/guide/change-alert/configure_define_the_metrics.png" alt="Configuration options for change alert detection method" style="width:100%;" >}}
{{< img src="/monitors/monitor_types/change-alert/configure_define_the_metrics.png" alt="Configuration options for change alert detection method" style="width:100%;" >}}

The example shows the following alert condition:
The **average** of the **change** over **1 hour** compared to **5 minutes**
Expand All @@ -51,12 +57,16 @@ This determines the way the monitor evaluates as expressed in the formula sectio

In both cases, `Change`, and `% Change` can be either positive or negative.

## Notifications

For instructions on the **Notify your team** section, see the [Notifications][7] and [Monitor configuration][8] pages.

## Troubleshooting a change alert evaluation

To verify the results of your change alert evaluation, reconstruct the metric queries with a Notebook.
Take this change alert monitor with the following settings.

{{< img src="monitors/guide/change-alert/example_monitor_config.png" alt="The create monitor page with a change alert selected, evaluating the percent change of the average of the metric system.load.1 over the last 5 minutes compared to the last 30 minutes" style="width:100%;" >}}
{{< img src="monitors/monitor_types/change-alert/example_monitor_config.png" alt="The create monitor page with a change alert selected, evaluating the percent change of the average of the metric system.load.1 over the last 5 minutes compared to the last 30 minutes" style="width:100%;" >}}

Monitor Query:
```pct_change(avg(last_5m),last_30m):<METRIC> > -50```
Expand All @@ -75,7 +85,7 @@ This is a break down of the query with the following conditions:
- Query of data points N minutes ago (this is the normal query + timeshift(-1800)).
- The timeshift function uses a **negative** duration because you're shifting the data back. Combine these queries along with the % change formula from the table.
- **Note**: Since this example only has one metric, it's also possible to use a single query (a) and add the formula `((a - timeshift(a, -1800)) / timeshift(a, -1800)) * 100`
{{< img src="monitors/guide/change-alert/notebook_query_reconstruct_timeshift.png" alt="The edit screen of a cell in a notebook, titled Reconstruct Change Alert query, configured as a timeseries using the average of the metric system.load.1, from everywhere, with the formula ((a - timeshift(a, -1800)) / timeshift(a, -1800)) * 100 being applied" style="width:100%;" >}}
{{< img src="monitors/monitor_types/change-alert/notebook_query_reconstruct_timeshift.png" alt="The edit screen of a cell in a notebook, titled Reconstruct Change Alert query, configured as a timeseries using the average of the metric system.load.1, from everywhere, with the formula ((a - timeshift(a, -1800)) / timeshift(a, -1800)) * 100 being applied" style="width:100%;" >}}
2. Compare the monitor's history graph with the notebook graph. Are the values comparable?
3. Apply the aggregation.
- To compare your notebook graph to the change alert monitor evaluation, scope your timeframe to match the change alert.
Expand All @@ -88,3 +98,6 @@ This is a break down of the query with the following conditions:
[1]: /monitors/configuration/#evaluation-window
[2]: /monitors/manage/status/#investigate-a-monitor-in-a-notebook
[3]: /dashboards/functions/timeshift/
[7]: /monitors/notify/
[8]: /monitors/configuration/?tab=thresholdalert#notify-your-team
[9]: https://app.datadoghq.com/monitors/create/metric/change
4 changes: 2 additions & 2 deletions content/en/monitors/types/metric.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ further_reading:
- link: "/monitors/manage/status/"
tag: "Documentation"
text: "Consult your monitor status"
- link: "/monitors/guide/change-alert"
- link: "/monitors/types/change-alert"
tag: "Documentation"
text: "Troubleshoot change alert monitors"
---
Expand Down Expand Up @@ -50,7 +50,7 @@ This type of alert is useful to track spikes, drops, or slow changes in a metric

For more information, see the [Change alert monitors][1] guide.

[1]: /monitors/guide/change-alert/
[1]: /monitors/types/change-alert/
{{% /tab %}}
{{% tab "Anomaly" %}}

Expand Down
Loading