Skip to content

Commit

Permalink
docs: AWS Cloud Guide Update (grafana#14687)
Browse files Browse the repository at this point in the history
  • Loading branch information
Jayclifford345 authored Oct 31, 2024
1 parent d1668f6 commit 061bdfe
Showing 1 changed file with 8 additions and 84 deletions.
92 changes: 8 additions & 84 deletions docs/sources/setup/install/helm/deployment-guides/aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@ There are two methods for authenticating and connecting Loki to AWS S3. We will
## Considerations

{{< admonition type="caution" >}}
This guide was accurate at the time it was last updated on **21st October, 2024**. As cloud providers frequently update their services and offerings, as a best practice, you should refer to the [AWS S3 documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) before creating your buckets and assigning roles.
This guide was accurate at the time it was last updated on **31st October, 2024**. As cloud providers frequently update their services and offerings, as a best practice, you should refer to the [AWS S3 documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) before creating your buckets and assigning roles.
{{< /admonition >}}

- **IAM Role:** The IAM role created in this guide is a basic role that allows Loki to read and write to the S3 bucket. You may wish to add more granular permissions based on your requirements.

- **Authentication:** Grafana Loki comes with a basic authentication layer. The Loki gateway (NGINX) is exposed to the internet using basic authentication in this example. NGINX can also be replaced with other open-source reverse proxies. Refer to [Authentication](https://grafana.com/docs/loki/<LOKI_VERSION/operations/authentication/) for more information.
- **Authentication:** Grafana Loki comes with a basic authentication layer. The Loki gateway (NGINX) is exposed to the internet using basic authentication in this example. NGINX can also be replaced with other open-source reverse proxies. Refer to [Authentication](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/authentication/) for more information.

- **Retention:** The retention period is set to 28 days in the `values.yaml` file. You may wish to adjust this based on your requirements.

Expand All @@ -48,7 +48,7 @@ The minimum requirements for deploying Loki on EKS are:

- Kubernetes version `1.30` or above.
- `3` nodes for the EKS cluster.
- Instance type depends on your workload. A good starting point is `m5.xlarge`.
- Instance type depends on your workload. A good starting point for a production cluster is `m7i.2xlarge`.

Here is the EKSctl cluster configuration file used in this guide:

Expand All @@ -59,37 +59,34 @@ apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
name: <INSERT-NAME>
region: <INSERT-REGION>
name: <INSERT-CLUSTER-NAME>
region: <INSERT-REGION-FOR-CLUSTER>
version: "1.31"

iam:
withOIDC: true

addons:
- name: aws-ebs-csi-driver
- name: eks-pod-identity-agent

managedNodeGroups:
- name: loki-workers
instanceType: m5.xlarge
instanceType: m7i.2xlarge
desiredCapacity: 3
minSize: 2
maxSize: 3
amiFamily: AmazonLinux2
amiFamily: AmazonLinux2023
iam:
withAddonPolicies:
ebs: true
volumeSize: 80
volumeType: gp2
volumeType: gp3
ebsOptimized: true

```
The following plugins must also be installed within the EKS cluster:
- **Amazon EBS CSI Driver**: Enables Kubernetes to dynamically provision and manage EBS volumes as persistent storage for applications. We use this to provision the node volumes for Loki.
- **Amazon EKS Pod Identity Agent**: Manages AWS IAM roles for pods, allowing fine-grained access control to AWS resources without needing to store credentials in containers. This is how Loki will access the S3 bucket.
- **CoreDNS**: Provides internal DNS service for Kubernetes clusters, ensuring that services and pods can communicate with each other using DNS names.
- **kube-proxy**: Maintains network rules on nodes, enabling communication between pods and services within the cluster.
Expand Down Expand Up @@ -198,77 +195,6 @@ The recommended method for connecting Loki to AWS S3 is to use an IAM role. This
```
**Make sure to replace the placeholder with your AWS account ID.**

### Adding the policy to the S3 buckets

To allow the IAM role to access the S3 buckets, you need to add the policy to the bucket. You can do this using the AWS Management Console or the AWS CLI. The below steps show how to add the policy using the AWS CLI.

1. Create a bucket policy file named `bucket-policy-chunk.json` with the following content:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<ACCOUNT ID>:role/LokiServiceAccountRole"
},
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::< CHUNK BUCKET NAME >",
"arn:aws:s3:::< CHUNK BUCKET NAME >/*"
]
}
]
}
```
**Make sure to replace the placeholders with your AWS account ID and the bucket names.**

1. Add the policy to the bucket:

```bash
aws s3api put-bucket-policy --bucket <CHUNK BUCKET NAME eg. `loki-aws-dev-chunks`> --policy file://bucket-policy-chunk.json
```
1. Create a bucket policy file named `bucket-policy-ruler.json` with the following content:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<ACCOUNT ID>:role/LokiServiceAccountRole"
},
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::< RULER BUCKET NAME >",
"arn:aws:s3:::< RULER BUCKET NAME >/*"
]
}
]
}
```
**Make sure to replace the placeholders with your AWS account ID and the bucket names.**

1. Add the policy to the bucket:

```bash
aws s3api put-bucket-policy --bucket <RULER BUCKET NAME eg. `loki-aws-dev-ruler`> --policy file://bucket-policy-ruler.json
```

## Deploying the Helm chart

Before we can deploy the Loki Helm chart, we need to add the Grafana chart repository to Helm. This repository contains the Loki Helm chart.
Expand Down Expand Up @@ -322,8 +248,6 @@ Loki by default does not come with any authentication. Since we will be deployin
We create a literal secret with the username and password for Loki canary to authenticate with the Loki gateway.
**Make sure to replace the placeholders with your desired username and password.**



### Loki Helm chart configuration

Create a `values.yaml` file choosing the configuration options that best suit your requirements. Below there is an example of `values.yaml` files for the Loki Helm chart in [microservices](https://grafana.com/docs/loki/<LOKI_VERSION>/get-started/deployment-modes/#microservices-mode) mode.
Expand Down

0 comments on commit 061bdfe

Please sign in to comment.