Skip to content

Commit

Permalink
add gcs rehydration instructions and remove beta flag for rehydration (
Browse files Browse the repository at this point in the history
…#6063)

* add gcs rehydration instructions and remove beta flag for rehydration

* remove unused link - accept comments - remove reference to service argument requirement
  • Loading branch information
estib authored and ruthnaebeck committed Nov 27, 2019
1 parent 1935630 commit bf35c15
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 15 deletions.
36 changes: 21 additions & 15 deletions content/en/logs/archives/rehydrating.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,6 @@ aliases:
- /logs/historical-views
---

<div class="alert alert-warning">
Log Archive Rehydrating is in public beta. Request early access by contacting <a href="/help">Datadog Support</a>. Rehydrating is currently only supported for users of the Datadog US site. <a href="https://forms.gle/X4jhi13Rd2pFSuSHA">Send feedback for this feature</a>.
</div>

## Overview

Log Rehydration* enables you to capture log events from customer-owned storage-optimized archives back into Datadog's search-optimized [Log Explorer][1], so that you can use Datadog to analyze or investigate log events that are either old or were excluded from indexing.
Expand All @@ -26,7 +22,7 @@ With historical views, teams rehydrate archived log events precisely by timefram

2. **Choose the time period** for which you wish to rehydrate log events. The time period must be older than 24 hours.

3. **Input the query**. The query syntax is the same as that of the [log explorer search][5], but is limited to log attributes, [reserved attributes][6], and free text search on the message. You must include a service argument.
3. **Input the query**. The query syntax is the same as that of the [log explorer search][5], but is limited to log attributes, [reserved attributes][6], and free text search on the message.

4. **Name your historical view**. Names must begin with a lowercase letter and can only contain lowercase letters, numbers, and the `-` character.

Expand Down Expand Up @@ -68,9 +64,10 @@ An external archive must be configured in order to rehydrate data from it. [Foll

Datadog requires the permission to read from your archives in order to rehydrate content from them. This permission can be changed at any time.

#### AWS S3 Bucket
{{< tabs >}}
{{% tab "AWS S3" %}}

In order to rehydrate log events from your archives, Datadog uses the IAM Role in your AWS account that you configured for [your AWS integration][8]. If you have not yet created that Role, [follow these steps to do so][9]. To allow that Role to rehydrate log events from your archives, add the following permission statement to its IAM policies. Be sure to edit the bucket names and, if desired, specify the paths that contain your log archives.
In order to rehydrate log events from your archives, Datadog uses the IAM Role in your AWS account that you configured for [your AWS integration][1]. If you have not yet created that Role, [follow these steps to do so][2]. To allow that Role to rehydrate log events from your archives, add the following permission statement to its IAM policies. Be sure to edit the bucket names and, if desired, specify the paths that contain your log archives.

```
{
Expand Down Expand Up @@ -103,13 +100,26 @@ In order to rehydrate log events from your archives, Datadog uses the IAM Role i

#### Adding role delegation to S3 archives

Datadog only supports rehydrating from archives that have been configured to use role delegation to grant access. Once you have modified your Datadog IAM role to include the IAM policy above, ensure that each archive in your [archive configuration page][10] has the correct AWS Account + Role combination.
Datadog only supports rehydrating from archives that have been configured to use role delegation to grant access. Once you have modified your Datadog IAM role to include the IAM policy above, ensure that each archive in your [archive configuration page][3] has the correct AWS Account + Role combination.

{{< img src="logs/archives/log_archives_rehydrate_configure_s3.png" alt="Adding role delegation to S3 archives" responsive="true" style="width:75%;">}}

### Feedback
[1]: https://app.datadoghq.com/account/settings#integrations/amazon-web-services
[2]: /integrations/amazon_web_services/?tab=allpermissions#installation
[3]: https://app.datadoghq.com/logs/pipelines/archives

{{% /tab %}}

{{% tab "Google Cloud Storage" %}}

In order to rehydrate log events from your archives, Datadog uses a service account with the Storage Object Viewer role. You can grant this role to your Datadog service account from the [GCP IAM Admin page][1] by editing the service account's permissions, adding another role, and then selecting Storage > Storage Object Viewer.

{{< img src="logs/archives/log_archives_gcs_role.png" alt="Rehydration from GCS requires the Storage Object Viewer role" responsive="true" style="width:75%;">}}

[1]: https://console.cloud.google.com/iam-admin/iam

Datadog would love to receive your feedback on this feature, especially as it is in early beta. You can submit your feedback directly [from this form][11].
{{% /tab %}}
{{< /tabs >}}

*Log Rehydration is a trademark of Datadog, Inc.

Expand All @@ -119,8 +129,4 @@ Datadog would love to receive your feedback on this feature, especially as it is
[4]: #permissions
[5]: /logs/explorer/search
[6]: /logs/?tab=ussite#reserved-attributes
[7]: /logs/archives/s3/?tab=roledelegation#create-and-configure-an-s3-bucket
[8]: https://app.datadoghq.com/account/settings#integrations/amazon-web-services
[9]: /integrations/amazon_web_services/?tab=allpermissions#installation
[10]: https://app.datadoghq.com/logs/pipelines/archives
[11]: https://forms.gle/X4jhi13Rd2pFSuSHA
[7]: /logs/archives/
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit bf35c15

Please sign in to comment.