From 8ed29556e199ccf00fd42ac871cc94e6b12c0568 Mon Sep 17 00:00:00 2001 From: J Stickler Date: Wed, 30 Oct 2024 14:16:05 -0400 Subject: [PATCH] docs: Deploy Loki Helm on AWS guide (#14517) (#14662) Co-authored-by: Jay Clifford <45856600+Jayclifford345@users.noreply.github.com> --- docs/sources/setup/install/helm/_index.md | 9 +- .../install/helm/deployment-guides/_index.md | 13 + .../install/helm/deployment-guides/aws.md | 676 ++++++++++++++++++ .../helm/install-microservices/_index.md | 216 +++--- .../install/helm/install-monolithic/_index.md | 470 +++++++++--- .../install/helm/install-scalable/_index.md | 215 +++--- 6 files changed, 1293 insertions(+), 306 deletions(-) create mode 100644 docs/sources/setup/install/helm/deployment-guides/_index.md create mode 100644 docs/sources/setup/install/helm/deployment-guides/aws.md diff --git a/docs/sources/setup/install/helm/_index.md b/docs/sources/setup/install/helm/_index.md index ab06ae644a801..5392838e2ca74 100644 --- a/docs/sources/setup/install/helm/_index.md +++ b/docs/sources/setup/install/helm/_index.md @@ -22,6 +22,13 @@ This guide references the Loki Helm chart version 3.0 or greater and contains th If you are installing Grafana Enterprise Logs, follow the [GEL Helm installation](https://grafana.com/docs/enterprise-logs//setup/helm/). + +## Cloud Deployment Guides + +The following guides provide step-by-step instructions for deploying Loki on cloud providers: + +- [Amazon EKS](https://grafana.com/docs/loki//setup/install/helm/deployment-guides/aws/) + ## Reference -[Values reference]({{< relref "./reference" >}}) +[Values reference](https://grafana.com/docs/loki//setup/install/helm/reference/) diff --git a/docs/sources/setup/install/helm/deployment-guides/_index.md b/docs/sources/setup/install/helm/deployment-guides/_index.md new file mode 100644 index 0000000000000..4ff7d5dcaa983 --- /dev/null +++ b/docs/sources/setup/install/helm/deployment-guides/_index.md @@ -0,0 +1,13 @@ +--- +title: Cloud Deployment Guides +menuTitle: Cloud Deployment Guides +description: Step-by-step instructions for deploying Loki on cloud providers. +weight: 500 +keywords: +--- + +# Cloud Deployment Guides + +The following guides provide step-by-step instructions for deploying Loki on cloud providers: + +- [Deploy Loki on AWS](https://grafana.com/docs/loki//setup/install/helm/deployment-guides/aws/) \ No newline at end of file diff --git a/docs/sources/setup/install/helm/deployment-guides/aws.md b/docs/sources/setup/install/helm/deployment-guides/aws.md new file mode 100644 index 0000000000000..bbe80da5e7761 --- /dev/null +++ b/docs/sources/setup/install/helm/deployment-guides/aws.md @@ -0,0 +1,676 @@ +--- +title: Deploy the Loki Helm chart on AWS +menuTitle: Deploy on AWS +description: Installing the Loki Helm chart on AWS. +keywords: +--- + +# Deploy the Loki Helm chart on AWS + +This guide shows how to deploy a minimally viable Loki in **microservice** mode on AWS using the Helm chart. To run through this guide, we expect you to have the necessary tools and permissions to deploy resources on AWS, such as: + +- Full access to EKS (Amazon Elastic Kubernetes Service) +- Full access to S3 (Amazon Simple Storage Service) +- Sufficient permissions to create IAM (Identity Access Management) roles and policies + +There are two methods for authenticating and connecting Loki to AWS S3. We will guide you through the recommended method of granting access via an IAM role. + +## Considerations + +{{< admonition type="caution" >}} +This guide was accurate at the time it was last updated on **21st October, 2024**. As cloud providers frequently update their services and offerings, as a best practice, you should refer to the [AWS S3 documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) before creating your buckets and assigning roles. +{{< /admonition >}} + +- **IAM Role:** The IAM role created in this guide is a basic role that allows Loki to read and write to the S3 bucket. You may wish to add more granular permissions based on your requirements. + +- **Authentication:** Grafana Loki comes with a basic authentication layer. The Loki gateway (NGINX) is exposed to the internet using basic authentication in this example. NGINX can also be replaced with other open-source reverse proxies. Refer to [Authentication](https://grafana.com/docs/loki/}} +These EKS requirements are the minimum specification needed to deploy Loki using this guide. You may wish to adjust plugins and instance types based on your AWS environment and workload. **If you choose to do so, we cannot guarantee that this sample configuration will still meet your needs.** + +In this guide, we deploy Loki using `m5.xlarge` instances. This is a instance type that should work for most scenarios. However, you can modify the instance types and count based on your specific needs. +{{< /admonition >}} + +The minimum requirements for deploying Loki on EKS are: + +- Kubernetes version `1.30` or above. +- `3` nodes for the EKS cluster. +- Instance type depends on your workload. A good starting point is `m5.xlarge`. + +Here is the EKSctl cluster configuration file used in this guide: + +```yaml +# A simple example of ClusterConfig object: +--- +apiVersion: eksctl.io/v1alpha5 +kind: ClusterConfig + +metadata: + name: + region: + version: "1.31" + +iam: + withOIDC: true + +addons: + - name: aws-ebs-csi-driver + - name: eks-pod-identity-agent + +managedNodeGroups: + - name: loki-workers + instanceType: m5.xlarge + desiredCapacity: 3 + minSize: 2 + maxSize: 3 + amiFamily: AmazonLinux2 + iam: + withAddonPolicies: + ebs: true + volumeSize: 80 + volumeType: gp2 + ebsOptimized: true + +``` + + +The following plugins must also be installed within the EKS cluster: +- **Amazon EBS CSI Driver**: Enables Kubernetes to dynamically provision and manage EBS volumes as persistent storage for applications. We use this to provision the node volumes for Loki. +- **Amazon EKS Pod Identity Agent**: Manages AWS IAM roles for pods, allowing fine-grained access control to AWS resources without needing to store credentials in containers. This is how Loki will access the S3 bucket. +- **CoreDNS**: Provides internal DNS service for Kubernetes clusters, ensuring that services and pods can communicate with each other using DNS names. +- **kube-proxy**: Maintains network rules on nodes, enabling communication between pods and services within the cluster. + +You must also install an **OIDC (OpenID Connect) provider** on the EKS cluster. This is required for the IAM roles and policies to work correctly. If you are using EKSctl, you can install the OIDC provider using the following command: + +{{< admonition type="tip" >}} +If you used the above EKSctl configuration file to create the cluster, you do not need to run this command. The OIDC provider is automatically installed. +{{< /admonition >}} + +```bash +eksctl utils associate-iam-oidc-provider --cluster loki --approve +``` + +## Create S3 buckets + +{{< admonition type="warning" >}} + **DO NOT** use the default bucket names; `chunk`, `ruler` and `admin`. Choose a **unique** name for each bucket. For more information see the following [security update](https://grafana.com/blog/2024/06/27/grafana-security-update-grafana-loki-and-unintended-data-write-attempts-to-amazon-s3-buckets/). +{{< /admonition >}} + +Before deploying Loki, you need to create two S3 buckets; one to store logs (chunks), the second to store alert rules. You can create the bucket using the AWS Management Console or the AWS CLI. The bucket name must be globally unique. + +{{< admonition type="note" >}} +GEL customers will require a third bucket to store the admin data. This bucket is not required for OSS users. +{{< /admonition >}} + +```bash +aws s3api create-bucket --bucket --region --create-bucket-configuration LocationConstraint= \ +aws s3api create-bucket --bucket --region --create-bucket-configuration LocationConstraint= +``` +Make sure to replace the `region` and `bucket` name with your desired values. We will revisit the bucket policy later in this guide. + +## Defining IAM roles and policies + +The recommended method for connecting Loki to AWS S3 is to use an IAM role. This method is more secure than using access keys and secret keys which are directly stored in the Loki configuration. The role and policy can be created using the AWS CLI or the AWS Management Console. The below steps show how to create the role and policy using the AWS CLI. + +1. Create a new directory and navigate to it. Make sure to create the files in this directory. All commands in this guide assume you are in this directory. + +1. Create a `loki-s3-policy.json` file with the following content: + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "LokiStorage", + "Effect": "Allow", + "Action": [ + "s3:ListBucket", + "s3:PutObject", + "s3:GetObject", + "s3:DeleteObject" + ], + "Resource": [ + "arn:aws:s3:::< CHUNK BUCKET NAME >", + "arn:aws:s3:::< CHUNK BUCKET NAME >/*", + "arn:aws:s3:::< RULER BUCKET NAME >", + "arn:aws:s3:::< RULER BUCKET NAME >/*" + ] + } + ] + } + ``` + + **Make sure to replace the placeholders with the names of the buckets you created earlier.** + +1. Create the IAM policy using the AWS CLI: + + ```bash + aws iam create-policy --policy-name LokiS3AccessPolicy --policy-document file://loki-s3-policy.json + ``` + +2. Create a trust policy document named `trust-policy.json` with the following content: + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Federated": "arn:aws:iam::< ACCOUNT ID >:oidc-provider/oidc.eks..amazonaws.com/id/< OIDC ID >" + }, + "Action": "sts:AssumeRoleWithWebIdentity", + "Condition": { + "StringEquals": { + "oidc.eks..amazonaws.com/id/< OIDC ID >:sub": "system:serviceaccount:loki:loki", + "oidc.eks..amazonaws.com/id/< OIDC ID >:aud": "sts.amazonaws.com" + } + } + } + ] + } + ``` + **Make sure to replace the placeholders with your AWS account ID, region, and the OIDC ID (you can find this in the EKS cluster configuration).** + +3. Create the IAM role using the AWS CLI: + + ```bash + aws iam create-role --role-name LokiServiceAccountRole --assume-role-policy-document file://trust-policy.json + ``` + +4. Attach the policy to the role: + + ```bash + aws iam attach-role-policy --role-name LokiServiceAccountRole --policy-arn arn:aws:iam:::policy/LokiS3AccessPolicy + ``` + **Make sure to replace the placeholder with your AWS account ID.** + +### Adding the policy to the S3 buckets + +To allow the IAM role to access the S3 buckets, you need to add the policy to the bucket. You can do this using the AWS Management Console or the AWS CLI. The below steps show how to add the policy using the AWS CLI. + +1. Create a bucket policy file named `bucket-policy-chunk.json` with the following content: + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "Statement1", + "Effect": "Allow", + "Principal": { + "AWS": "arn:aws:iam:::role/LokiServiceAccountRole" + }, + "Action": [ + "s3:PutObject", + "s3:GetObject", + "s3:DeleteObject", + "s3:ListBucket" + ], + "Resource": [ + "arn:aws:s3:::< CHUNK BUCKET NAME >", + "arn:aws:s3:::< CHUNK BUCKET NAME >/*" + ] + } + ] + } + ``` + **Make sure to replace the placeholders with your AWS account ID and the bucket names.** + +1. Add the policy to the bucket: + + ```bash + aws s3api put-bucket-policy --bucket --policy file://bucket-policy-chunk.json + ``` +1. Create a bucket policy file named `bucket-policy-ruler.json` with the following content: + + ```json + { + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "Statement1", + "Effect": "Allow", + "Principal": { + "AWS": "arn:aws:iam:::role/LokiServiceAccountRole" + }, + "Action": [ + "s3:PutObject", + "s3:GetObject", + "s3:DeleteObject", + "s3:ListBucket" + ], + "Resource": [ + "arn:aws:s3:::< RULER BUCKET NAME >", + "arn:aws:s3:::< RULER BUCKET NAME >/*" + ] + } + ] + } + ``` + **Make sure to replace the placeholders with your AWS account ID and the bucket names.** + +1. Add the policy to the bucket: + + ```bash + aws s3api put-bucket-policy --bucket --policy file://bucket-policy-ruler.json + ``` + +## Deploying the Helm chart + +Before we can deploy the Loki Helm chart, we need to add the Grafana chart repository to Helm. This repository contains the Loki Helm chart. + +1. Add the Grafana chart repository to Helm: + + ```bash + helm repo add grafana https://grafana.github.io/helm-charts + ``` +1. Update the chart repository: + + ```bash + helm repo update + ``` +1. Create a new namespace for Loki: + + ```bash + kubectl create namespace loki + ``` +### Loki Basic Authentication + +Loki by default does not come with any authentication. Since we will be deploying Loki to AWS and exposing the gateway to the internet, we recommend adding at least basic authentication. In this guide we will give Loki a username and password: + +1. To start we will need create a `.htpasswd` file with the username and password. You can use the `htpasswd` command to create the file: + + {{< admonition type="tip" >}} + If you don't have the `htpasswd` command installed, you can install it using `brew` or `apt-get` or `yum` depending on your OS. + {{< /admonition >}} + + ```bash + htpasswd -c .htpasswd + ``` + This will create a file called `auth` with the username `loki`. You will be prompted to enter a password. + + 1. Create a Kubernetes secret with the `.htpasswd` file: + + ```bash + kubectl create secret generic loki-basic-auth --from-file=.htpasswd -n loki + ``` + + This will create a secret called `loki-basic-auth` in the `loki` namespace. We will reference this secret in the Loki Helm chart configuration. + +1. Create a `canary-basic-auth` secret for the canary: + + ```bash + kubectl create secret generic canary-basic-auth \ + --from-literal=username= \ + --from-literal=password= \ + -n loki + ``` + We create a literal secret with the username and password for Loki canary to authenticate with the Loki gateway. + **Make sure to replace the placeholders with your desired username and password.** + + + +### Loki Helm chart configuration + +Create a `values.yaml` file choosing the configuration options that best suit your requirements. Below there is an example of `values.yaml` files for the Loki Helm chart in [microservices](https://grafana.com/docs/loki//get-started/deployment-modes/#microservices-mode) mode. + +```yaml +loki: + schemaConfig: + configs: + - from: "2024-04-01" + store: tsdb + object_store: s3 + schema: v13 + index: + prefix: loki_index_ + period: 24h + storage_config: + aws: + region: # for example, eu-west-2 + bucketnames: # Your actual S3 bucket name, for example, loki-aws-dev-chunks + s3forcepathstyle: false + ingester: + chunk_encoding: snappy + pattern_ingester: + enabled: true + limits_config: + allow_structured_metadata: true + volume_enabled: true + retention_period: 672h # 28 days retention + compactor: + retention_enabled: true + delete_request_store: s3 + ruler: + enable_api: true + storage: + type: s3 + s3: + region: # for example, eu-west-2 + bucketnames: # Your actual S3 bucket name, for example, loki-aws-dev-ruler + s3forcepathstyle: false + alertmanager_url: http://prom:9093 # The URL of the Alertmanager to send alerts (Prometheus, Mimir, etc.) + + querier: + max_concurrent: 4 + + storage: + type: s3 + bucketNames: + chunks: "" # Your actual S3 bucket name (loki-aws-dev-chunks) + ruler: "" # Your actual S3 bucket name (loki-aws-dev-ruler) + # admin: "" # Your actual S3 bucket name (loki-aws-dev-admin) - GEL customers only + s3: + region: # eu-west-2 + #insecure: false + # s3forcepathstyle: false + +serviceAccount: + create: true + annotations: + "eks.amazonaws.com/role-arn": "arn:aws:iam:::role/LokiServiceAccountRole" # The service role you created + +deploymentMode: Distributed + +ingester: + replicas: 3 + persistence: + storageClass: gp2 + accessModes: + - ReadWriteOnce + size: 10Gi + +querier: + replicas: 3 + maxUnavailable: 2 + persistence: + storageClass: gp2 + accessModes: + - ReadWriteOnce + size: 10Gi +queryFrontend: + replicas: 2 + maxUnavailable: 1 +queryScheduler: + replicas: 2 +distributor: + replicas: 3 + maxUnavailable: 2 +compactor: + replicas: 1 + persistence: + storageClass: gp2 + accessModes: + - ReadWriteOnce + size: 10Gi +indexGateway: + replicas: 2 + maxUnavailable: 1 + persistence: + storageClass: gp2 + accessModes: + - ReadWriteOnce + size: 10Gi +ruler: + replicas: 1 + maxUnavailable: 1 + persistence: + storageClass: gp2 + accessModes: + - ReadWriteOnce + size: 10Gi + + +# This exposes the Loki gateway so it can be written to and queried externaly +gateway: + service: + type: LoadBalancer + basicAuth: + enabled: true + existingSecret: loki-basic-auth + +# Since we are using basic auth, we need to pass the username and password to the canary +lokiCanary: + extraArgs: + - -pass=$(LOKI_PASS) + - -user=$(LOKI_USER) + extraEnv: + - name: LOKI_PASS + valueFrom: + secretKeyRef: + name: canary-basic-auth + key: password + - name: LOKI_USER + valueFrom: + secretKeyRef: + name: canary-basic-auth + key: username + +# Enable minio for storage +minio: + enabled: false + +backend: + replicas: 0 +read: + replicas: 0 +write: + replicas: 0 + +singleBinary: + replicas: 0 +``` + +{{< admonition type="caution" >}} +Make sure to replace the placeholders with your actual values. +{{< /admonition >}} + +It is critical to define a valid `values.yaml` file for the Loki deployment. To remove the risk of misconfiguration, let's break down the configuration options to keep in mind when deploying to AWS: + +- **Loki Config vs. Values Config:** + - The `values.yaml` file contains a section called `loki`, which contains a direct representation of the Loki configuration file. + - This section defines the Loki configuration, including the schema, storage, and querier configuration. + - The key configuration to focus on for chunks is the `storage_config` section, where you define the S3 bucket region and name. This tells Loki where to store the chunks. + - The `ruler` section defines the configuration for the ruler, including the S3 bucket region and name. This tells Loki where to store the alert and recording rules. + - For the full Loki configuration, refer to the [Loki Configuration](https://grafana.com/docs/loki//configure/) documentation. + +- **Storage:** + - Defines where the Helm chart stores data. + - Set the type to `s3` since we are using Amazon S3. + - Configure the bucket names for the chunks and ruler to match the buckets created earlier. + - The `s3` section specifies the region of the bucket. + +- **Service Account:** + - The `serviceAccount` section is used to define the IAM role for the Loki service account. + - This is where the IAM role created earlier is linked. + +- **Gateway:** + - Defines how the Loki gateway will be exposed. + - We are using a `LoadBalancer` service type in this configuration. + + +### Deploy Loki + +Now that you have created the `values.yaml` file, you can deploy Loki using the Helm chart. + +1. Deploy using the newly created `values.yaml` file: + + ```bash + helm install --values values.yaml loki grafana/loki -n loki --create-namespace + ``` + **It is important to create a namespace called `loki` as our trust policy is set to allow the IAM role to be used by the `loki` service account in the `loki` namespace. This is configurable but make sure to update your service account.** + +1. Verify the deployment: + + ```bash + kubectl get pods -n loki + ``` + You should see the Loki pods running. + + ```console + NAME READY STATUS RESTARTS AGE + loki-canary-crqpg 1/1 Running 0 10m + loki-canary-hm26p 1/1 Running 0 10m + loki-canary-v9wv9 1/1 Running 0 10m + loki-chunks-cache-0 2/2 Running 0 10m + loki-compactor-0 1/1 Running 0 10m + loki-distributor-78ccdcc9b4-9wlhl 1/1 Running 0 10m + loki-distributor-78ccdcc9b4-km6j2 1/1 Running 0 10m + loki-distributor-78ccdcc9b4-ptwrb 1/1 Running 0 10m + loki-gateway-5f97f78755-hm6mx 1/1 Running 0 10m + loki-index-gateway-0 1/1 Running 0 10m + loki-index-gateway-1 1/1 Running 0 10m + loki-ingester-zone-a-0 1/1 Running 0 10m + loki-ingester-zone-b-0 1/1 Running 0 10m + loki-ingester-zone-c-0 1/1 Running 0 10m + loki-querier-89d4ff448-4vr9b 1/1 Running 0 10m + loki-querier-89d4ff448-7nvrf 1/1 Running 0 10m + loki-querier-89d4ff448-q89kh 1/1 Running 0 10m + loki-query-frontend-678899db5-n5wc4 1/1 Running 0 10m + loki-query-frontend-678899db5-tf69b 1/1 Running 0 10m + loki-query-scheduler-7d666bf759-9xqb5 1/1 Running 0 10m + loki-query-scheduler-7d666bf759-kpb5q 1/1 Running 0 10m + loki-results-cache-0 2/2 Running 0 10m + loki-ruler-0 1/1 Running 0 10m + ``` + +### Find the Loki Gateway Service + +The Loki Gateway service is a LoadBalancer service that exposes the Loki gateway to the internet. This is where you will write logs to and query logs from. By default NGINX is used as the gateway. + +{{< admonition type="caution" >}} +The Loki Gateway service is exposed to the internet. We provide basic authentication using a username and password in this tutorial. Refer to the [Authentication](https://grafana.com/docs/loki//operations/authentication/) documentation for more information. +{{< /admonition >}} + +To find the Loki Gateway service, run the following command: + +```bash +kubectl get svc -n loki +``` +You should see the Loki Gateway service with an external IP address. This is the address you will use to write to and query Loki. + +```console + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE +loki-gateway LoadBalancer 10.100.201.74 12345678975675456-1433434453245433545656563.eu-west-2.elb.amazonaws.com 80:30707/TCP 46m +``` + +Congratulations! You have successfully deployed Loki on AWS using the Helm chart. Before we finish, let's test the deployment. + +## Testing Your Loki Deployment + +k6 is one of the fastest ways to test your Loki deployment. This will allow you to both write and query logs to Loki. To get started with k6, follow the steps below: + +1. Install k6 with the Loki extension on your local machine. Refer to [Installing k6 and the xk6-loki extension](https://grafana.com/docs/loki//send-data/k6/). + +2. Create a `aws-test.js` file with the following content: + + ```javascript + import {sleep, check} from 'k6'; + import loki from 'k6/x/loki'; + + /** + * URL used for push and query requests + * Path is automatically appended by the client + * @constant {string} + */ + + const username = ''; + const password = ''; + const external_ip = ''; + + const credentials = `${username}:${password}`; + + const BASE_URL = `http://${credentials}@${external_ip}`; + + /** + * Helper constant for byte values + * @constant {number} + */ + const KB = 1024; + + /** + * Helper constant for byte values + * @constant {number} + */ + const MB = KB * KB; + + /** + * Instantiate config and Loki client + */ + + const conf = new loki.Config(BASE_URL); + const client = new loki.Client(conf); + + /** + * Define test scenario + */ + export const options = { + vus: 10, + iterations: 10, + }; + + **It is important to create a namespace called `loki` as our trust policy is set to allow the IAM role to be used by the `loki` service account in the `loki` namespace. This is configurable but make sure to update your service account.** + * "main" function for each VU iteration + */ + export default () => { + // Push request with 10 streams and uncompressed logs between 800KB and 2MB + var res = client.pushParameterized(10, 800 * KB, 2 * MB); + // Check for successful write + check(res, { 'successful write': (res) => res.status == 204 }); + + // Pick a random log format from label pool + let format = randomChoice(conf.labels["format"]); + + // Execute instant query with limit 1 + res = client.instantQuery(`count_over_time({format="${format}"}[1m])`, 1) + // Check for successful read + check(res, { 'successful instant query': (res) => res.status == 200 }); + + // Execute range query over last 5m and limit 1000 + res = client.rangeQuery(`{format="${format}"}`, "5m", 1000) + // Check for successful read + check(res, { 'successful range query': (res) => res.status == 200 }); + + // Wait before next iteration + sleep(1); + } + + /** + * Helper function to get random item from array + */ + function randomChoice(items) { + return items[Math.floor(Math.random() * items.length)]; + } + ``` + + **Replace `` with the external IP address of the Loki Gateway service.** + + This script will write logs to Loki and query logs from Loki. It will write logs in a random format between 800KB and 2MB and query logs in a random format over the last 5 minutes. + +3. Run the test: + + ```bash + ./k6 run aws-test.js + ``` + + This will run the test and output the results. You should see the test writing logs to Loki and querying logs from Loki. + +Now that you have successfully deployed Loki in microservices mode on AWS, you may wish to explore the following: + +- [Sending data to Loki](https://grafana.com/docs/loki//query/) +- [Manage](https://grafana.com/docs/loki/}}). The microservices deployment mode runs components of Loki as distinct processes. +This Helm chart deploys Loki to run Loki in [microservice mode](https://grafana.com/docs/loki//get-started/deployment-modes/#microservices-mode) within a Kubernetes cluster. The microservices deployment mode runs components of Loki as distinct processes. The default Helm chart deploys the following components: - **Compactor component** (1 replica): Compacts and processes stored data. @@ -21,16 +21,19 @@ The default Helm chart deploys the following components: - **QueryFrontend component** (2 replicas, maxUnavailable: 1): Manages frontend queries. Up to 1 replica can be unavailable during updates. - **QueryScheduler component** (2 replicas): Schedules queries. -It is not recommended to run scalable mode with `filesystem` storage. For the purpose of this guide, we will use MinIO as the object storage to provide a complete example. +{{< admonition type="note" >}} +We do not recommend running in Microservice mode with `filesystem` storage. For the purpose of this guide, we will use MinIO as the object storage to provide a complete example. +{{< /admonition >}} -**Prerequisites** + +## Prerequisites - Helm 3 or above. See [Installing Helm](https://helm.sh/docs/intro/install/). - A running Kubernetes cluster (must have at least 3 nodes). -- (Optional) A Memcached deployment for better query performance. For information on configuring Memcached, refer to the [caching section](https://grafana.com/docs/loki//operations/caching/). -**To deploy Loki in microservice mode (with MinIO):** + +## Deploying the Helm chart for development and testing 1. Add [Grafana's chart repository](https://github.com/grafana/helm-charts) to Helm: @@ -39,48 +42,46 @@ It is not recommended to run scalable mode with `filesystem` storage. For the pu helm repo add grafana https://grafana.github.io/helm-charts ``` -2. Update the chart repository: +1. Update the chart repository: ```bash helm repo update ``` -3. Create the configuration file `values.yaml`. The example below illustrates how to deploy Loki in test mode using MinIO as storage: +1. Create the configuration file `values.yaml`. The example below illustrates how to deploy Loki in test mode using MinIO as storage: ```yaml loki: - schemaConfig: - configs: - - from: "2024-04-01" - store: tsdb - object_store: s3 - schema: v13 - index: - prefix: loki_index_ - period: 24h - ingester: - chunk_encoding: snappy - tracing: - enabled: true - querier: - # Default is 4, if you have enough memory and CPU you can increase, reduce if OOMing - max_concurrent: 4 - - #gateway: - # ingress: - # enabled: true - # hosts: - # - host: FIXME - # paths: - # - path: / - # pathType: Prefix + schemaConfig: + configs: + - from: "2024-04-01" + store: tsdb + object_store: s3 + schema: v13 + index: + prefix: loki_index_ + period: 24h + ingester: + chunk_encoding: snappy + querier: + # Default is 4, if you have enough memory and CPU you can increase, reduce if OOMing + max_concurrent: 4 + pattern_ingester: + enabled: true + limits_config: + allow_structured_metadata: true + volume_enabled: true + retention_period: 672h + compactor: + retention_enabled: true + delete_request_store: s3 deploymentMode: Distributed ingester: - replicas: 3 + replicas: 3 # To ensure data durability with replication querier: - replicas: 3 + replicas: 3 # Improve query performance via parallelism maxUnavailable: 2 queryFrontend: replicas: 2 @@ -88,7 +89,7 @@ It is not recommended to run scalable mode with `filesystem` storage. For the pu queryScheduler: replicas: 2 distributor: - replicas: 3 + replicas: 3 maxUnavailable: 2 compactor: replicas: 1 @@ -100,24 +101,29 @@ It is not recommended to run scalable mode with `filesystem` storage. For the pu replicas: 0 bloomGateway: replicas: 0 - - # Enable minio for storage - minio: - enabled: true - - # Zero out replica counts of other deployment modes + backend: - replicas: 0 + replicas: 0 read: - replicas: 0 + replicas: 0 write: - replicas: 0 + replicas: 0 singleBinary: - replicas: 0 + replicas: 0 + + # This exposes the Loki gateway so it can be written to and queried externaly + gateway: + service: + type: LoadBalancer + + + # Enable minio for storage + minio: + enabled: true ``` -4. Install or upgrade the Loki deployment. +1. Install or upgrade the Loki deployment. - To install: ```bash helm install --values values.yaml loki grafana/loki @@ -138,7 +144,7 @@ It is not recommended to run scalable mode with `filesystem` storage. For the pu loki-canary-8thrx 1/1 Running 0 167m loki-canary-h965l 1/1 Running 0 167m loki-canary-th8kb 1/1 Running 0 167m - loki-chunks-cache-0 0/2 Pending 0 167m + loki-chunks-cache-0 2/2 Running 0 167m loki-compactor-0 1/1 Running 0 167m loki-compactor-1 1/1 Running 0 167m loki-distributor-7c9bb8f4dd-bcwc5 1/1 Running 0 167m @@ -165,55 +171,69 @@ It is not recommended to run scalable mode with `filesystem` storage. For the pu ## Object Storage Configuration -After testing Loki with MinIO, it is recommended to configure Loki with an object storage provider. The following examples shows how to configure Loki with different object storage providers: +After testing Loki with [MinIO](https://min.io/docs/minio/kubernetes/upstream/index.html), we recommend configuring Loki with an object storage provider. The following examples shows how to configure Loki with different object storage providers: + +{{< admonition type="caution" >}} +When deploying Loki using S3 Storage **DO NOT** use the default bucket names; `chunk`, `ruler` and `admin`. Choose a unique name for each bucket. For more information see the following [security update](https://grafana.com/blog/2024/06/27/grafana-security-update-grafana-loki-and-unintended-data-write-attempts-to-amazon-s3-buckets/). This caution does not apply when you are using MinIO. When using MinIO we recommend using the default bucket names. +{{< /admonition >}} -{{< code >}} -```s3 +{{< collapse title="S3" >}} + +```yaml # Example configuration for Loki with S3 storage - loki: - schemaConfig: - configs: - - from: "2024-04-01" - store: tsdb - object_store: s3 - schema: v13 - index: - prefix: loki_index_ - period: 24h - ingester: +loki: + schemaConfig: + configs: + - from: 2024-04-01 + store: tsdb + object_store: s3 + schema: v13 + index: + prefix: loki_index_ + period: 24h + storage_config: + aws: + region: + bucketnames: + s3forcepathstyle: false + ingester: chunk_encoding: snappy - tracing: + pattern_ingester: enabled: true - querier: + limits_config: + allow_structured_metadata: true + volume_enabled: true + retention_period: 672h # 28 days retention + querier: max_concurrent: 4 - storage: - type: s3 - bucketNames: - chunks: "chunks" - ruler: "ruler" - admin: "admin" - s3: - # s3 URL can be used to specify the endpoint, access key, secret key, and bucket name - s3: s3://access_key:secret_access_key@custom_endpoint/bucket_name - # AWS endpoint URL - endpoint: - # AWS region where the S3 bucket is located - region: - # AWS secret access key - secretAccessKey: - # AWS access key ID - accessKeyId: - # AWS signature version (e.g., v2 or v4) - signatureVersion: - # Forces the path style for S3 (true/false) - s3ForcePathStyle: false - # Allows insecure (HTTP) connections (true/false) - insecure: false - # HTTP configuration settings - http_config: {} + storage: + type: s3 + bucketNames: + chunks: + ruler: + admin: + s3: + # s3 URL can be used to specify the endpoint, access key, secret key, and bucket name this works well for S3 compatible storage or if you are hosting Loki on-premises and want to use S3 as the storage backend. Either use the s3 URL or the individual fields below (AWS endpoint, region, secret). + s3: s3://access_key:secret_access_key@custom_endpoint/bucket_name + # AWS endpoint URL + endpoint: + # AWS region where the S3 bucket is located + region: + # AWS secret access key + secretAccessKey: + # AWS access key ID + accessKeyId: + # AWS signature version (e.g., v2 or v4) + signatureVersion: + # Forces the path style for S3 (true/false) + s3ForcePathStyle: false + # Allows insecure (HTTP) connections (true/false) + insecure: false + # HTTP configuration settings + http_config: {} deploymentMode: Distributed @@ -256,8 +276,11 @@ After testing Loki with MinIO, it is recommended to configure Loki with an objec replicas: 0 ``` +{{< /collapse >}} -```azure +{{< collapse title="Azure" >}} + +```yaml # Example configuration for Loki with Azure Blob Storage loki: @@ -337,10 +360,19 @@ singleBinary: replicas: 0 ``` -{{< /code >}} +{{< /collapse >}} To configure other storage providers, refer to the [Helm Chart Reference]({{< relref "../reference" >}}). +## Deploying the Loki Helm chart to a Production Environment + +{{< admonition type="note" >}} +We are actively working on providing more guides for deploying Loki in production. +{{< /admonition >}} + +We recommend running Loki at scale within a cloud environment like AWS, Azure, or GCP. The below guides will show you how to deploy a minimally viable production environment. +- [Deploy Loki on AWS]({{< relref "../deployment-guides/aws" >}}) + ## Next Steps * Configure an agent to [send log data to Loki](/docs/loki//send-data/). * Monitor the Loki deployment using the [Meta Monitoring Healm chart](/docs/loki//setup/install/helm/monitor-and-alert/) diff --git a/docs/sources/setup/install/helm/install-monolithic/_index.md b/docs/sources/setup/install/helm/install-monolithic/_index.md index 4373907dcfbb1..fe790b0b6f890 100644 --- a/docs/sources/setup/install/helm/install-monolithic/_index.md +++ b/docs/sources/setup/install/helm/install-monolithic/_index.md @@ -10,107 +10,411 @@ weight: 100 # Install the monolithic Helm chart -This Helm Chart installation runs the Grafana Loki *single binary* within a Kubernetes cluster. +This Helm Chart installation deploys Grafana Loki in [monolithic mode](https://grafana.com/docs/loki//get-started/deployment-modes/#monolithic-mode) within a Kubernetes cluster. -If you set the `singleBinary.replicas` value to 1 and set the deployment mode to `SingleBinary`, this chart configures Loki to run the `all` target in a [monolithic mode](https://grafana.com/docs/loki//get-started/deployment-modes/#monolithic-mode), designed to work with the filesystem storage configuration. It will also configure meta-monitoring of metrics and logs. +## Prerequisites + +- Helm 3 or above. See [Installing Helm](https://helm.sh/docs/intro/install/). +- A running Kubernetes cluster. + +## Single Replica or Multiple Replicas + +There are two ways to deploy Loki in monolithic mode: +1. **Single Replica**: Run Loki with a single replica. This mode is useful for testing and development or if you are planning to run Loki as a meta-monitoring system. +2. **Multiple Replicas**: Run Loki with multiple replicas. This mode is useful for high availability. This mode is less economical than microservice mode, but it is simpler to operate. We recommend running at least three replicas for high availability. + +Once you have selected how many replicas you would like to deploy, choose the appropriate `values.yaml` configuration file below and then continue with the deployment steps. + +### Single Replica + +Deploying the Helm chart with a single replica deploys the following components: +- Loki (1 replica) +- Loki Canary (1 DaemonSet) +- Loki Gateway (1 NGINX replica) +- Loki Chunk and Result Cache (1 DaemonSet) +- Minio (optional, if `minio.enabled=true`) + +Create the configuration file `values.yaml`: {{< admonition type="note" >}} You must specify `commonConfig.replication_factor: 1` if you are only using 1 replica, otherwise requests will fail. {{< /admonition >}} -If you set the `singleBinary.replicas` value to 2 or more, this chart configures Loki to run a *single binary* in a replicated, highly available mode. When running replicas of a single binary, you must configure object storage. +```yaml +loki: + commonConfig: + replication_factor: 1 + schemaConfig: + configs: + - from: "2024-04-01" + store: tsdb + object_store: s3 + schema: v13 + index: + prefix: loki_index_ + period: 24h + pattern_ingester: + enabled: true + limits_config: + allow_structured_metadata: true + volume_enabled: true + retention_period: 672h # 28 days retention + compactor: + retention_enabled: true + delete_request_store: s3 + ruler: + enable_api: true -**Before you begin: Software Requirements** +minio: + enabled: true + +deploymentMode: SingleBinary -- Helm 3 or above. See [Installing Helm](https://helm.sh/docs/intro/install/). -- A running Kubernetes cluster +singleBinary: + replicas: 1 + +# Zero out replica counts of other deployment modes +backend: + replicas: 0 +read: + replicas: 0 +write: + replicas: 0 + +ingester: + replicas: 0 +querier: + replicas: 0 +queryFrontend: + replicas: 0 +queryScheduler: + replicas: 0 +distributor: + replicas: 0 +compactor: + replicas: 0 +indexGateway: + replicas: 0 +bloomCompactor: + replicas: 0 +bloomGateway: + replicas: 0 +``` + +In this configuration, we are deploying Loki with MinIO as the object storage. We recommend configuring object storage via cloud provider or pointing Loki at a MinIO cluster for production deployments. + +### Multiple Replicas + +Deploying the Helm chart with multiple replicas deploys the following components: +- Loki (3 replicas) +- Loki Canary (1 DaemonSet) +- Loki Gateway (1 NGINX replica) +- Loki Chunk and Result Cache (1 DaemonSet) +- Minio (optional, if `minio.enabled=true`) + +Create the configuration file `values.yaml`: -**To deploy Loki in monolithic mode:** +{{< admonition type="note" >}} +If you set the `singleBinary.replicas` value to 2 or more, this chart configures Loki to run a *single binary* in a replicated, highly available mode. When running replicas of a single binary, you must configure object storage. +{{< /admonition >}} + +```yaml +loki: + commonConfig: + replication_factor: 3 + schemaConfig: + configs: + - from: "2024-04-01" + store: tsdb + object_store: s3 + schema: v13 + index: + prefix: loki_index_ + period: 24h + pattern_ingester: + enabled: true + limits_config: + allow_structured_metadata: true + volume_enabled: true + retention_period: 672h # 28 days retention + compactor: + retention_enabled: true + delete_request_store: s3 + ruler: + enable_api: true + +minio: + enabled: true + +deploymentMode: SingleBinary + +singleBinary: + replicas: 3 + +# Zero out replica counts of other deployment modes +backend: + replicas: 0 +read: + replicas: 0 +write: + replicas: 0 + +ingester: + replicas: 0 +querier: + replicas: 0 +queryFrontend: + replicas: 0 +queryScheduler: + replicas: 0 +distributor: + replicas: 0 +compactor: + replicas: 0 +indexGateway: + replicas: 0 +bloomCompactor: + replicas: 0 +bloomGateway: + replicas: 0 +``` +In this configuration, we need to make sure to update the `commonConfig.replication_factor` and `singleBinary.replicas` to the desired number of replicas. We are deploying Loki with MinIO as the object storage. We recommend configuring object storage via cloud provider or pointing Loki at a MinIO cluster for production deployments. + +## Deploying the Helm chart for development and testing 1. Add [Grafana's chart repository](https://github.com/grafana/helm-charts) to Helm: - ```bash - helm repo add grafana https://grafana.github.io/helm-charts - ``` + ```bash + helm repo add grafana https://grafana.github.io/helm-charts + ``` 1. Update the chart repository: + ```bash + helm repo update + ``` + +1. Deploy Loki using the configuration file `values.yaml`: + + ```bash + helm install loki grafana/loki-stack -f values.yaml + ``` +1. Install or upgrade the Loki deployment. + - To install: + ```bash + helm install --values values.yaml loki grafana/loki + ``` + - To upgrade: + ```bash + helm upgrade --values values.yaml loki grafana/loki + ``` + +1. Verify that Loki is running: ```bash - helm repo update + kubectl get pods -n loki ``` -1. Create the configuration file `values.yaml`: - - - If running a single replica of Loki, configure the `filesystem` storage: - - ```yaml - deploymentMode: SingleBinary - loki: - commonConfig: - replication_factor: 1 - storage: - type: 'filesystem' - schemaConfig: - configs: - - from: "2024-01-01" - store: tsdb - index: - prefix: loki_index_ - period: 24h - object_store: filesystem # we're storing on filesystem so there's no real persistence here. - schema: v13 - singleBinary: - replicas: 1 - read: - replicas: 0 - backend: - replicas: 0 - write: - replicas: 0 - ``` - - - If running Loki with a replication factor greater than 1, set the desired number replicas and provide object storage credentials: - - ```yaml - loki: - commonConfig: - replication_factor: 3 - schemaConfig: - configs: - - from: "2024-01-01" - store: tsdb - index: - prefix: loki_index_ - period: 24h - object_store: s3 - schema: v13 - storage: - type: 's3' - bucketNames: - chunks: loki-chunks - ruler: loki-ruler - admin: loki-admin - s3: - endpoint: foo.aws.com - region: - secretAccessKey: supersecret - accessKeyId: secret - s3ForcePathStyle: false - insecure: false - singleBinary: - replicas: 3 - ``` - -1. Deploy the Loki cluster using one of these commands. - - - Deploy with the defined configuration: +## Object Storage Configuration - ```bash - helm install --values values.yaml loki grafana/loki - ``` +After testing Loki with MinIO, we recommend configuring Loki with an object storage provider. The following examples shows how to configure Loki with different object storage providers: + +{{< admonition type="caution" >}} +When deploying Loki using S3 Storage **DO NOT** use the default bucket names; `chunk`, `ruler` and `admin`. Choose a unique name for each bucket. For more information see the following [security update](https://grafana.com/blog/2024/06/27/grafana-security-update-grafana-loki-and-unintended-data-write-attempts-to-amazon-s3-buckets/). This caution does not apply when you are using MinIO. When using MinIO we recommend using the default bucket names. +{{< /admonition >}} - - Deploy with the defined configuration in a custom Kubernetes cluster namespace: +{{< collapse title="S3" >}} + +```yaml +loki: + commonConfig: + replication_factor: 3 + schemaConfig: + configs: + - from: "2024-04-01" + store: tsdb + object_store: s3 + schema: v13 + index: + prefix: loki_index_ + period: 24h + storage_config: + aws: + region: + bucketnames: + s3forcepathstyle: false + pattern_ingester: + enabled: true + limits_config: + allow_structured_metadata: true + volume_enabled: true + retention_period: 672h # 28 days retention + + storage: + type: s3 + bucketNames: + chunks: + ruler: + admin: + s3: + # s3 URL can be used to specify the endpoint, access key, secret key, and bucket name this works well for S3 compatible storages or are hosting Loki on-premises and want to use S3 as the storage backend. Either use the s3 URL or the individual fields below (AWS endpoint, region, secret). + s3: s3://access_key:secret_access_key@custom_endpoint/bucket_name + # AWS endpoint URL + endpoint: + # AWS region where the S3 bucket is located + region: + # AWS secret access key + secretAccessKey: + # AWS access key ID + accessKeyId: + # AWS signature version (e.g., v2 or v4) + signatureVersion: + # Forces the path style for S3 (true/false) + s3ForcePathStyle: false + # Allows insecure (HTTP) connections (true/false) + insecure: false + # HTTP configuration settings + http_config: {} + +# Disable minio storage +minio: + enabled: false + +singleBinary: + replicas: 3 + persistence: + storageClass: gp2 + accessModes: + - ReadWriteOnce + size: 30Gi + +# Zero out replica counts of other deployment modes +backend: + replicas: 0 +read: + replicas: 0 +write: + replicas: 0 + +ingester: + replicas: 0 +querier: + replicas: 0 +queryFrontend: + replicas: 0 +queryScheduler: + replicas: 0 +distributor: + replicas: 0 +compactor: + replicas: 0 +indexGateway: + replicas: 0 +bloomCompactor: + replicas: 0 +bloomGateway: + replicas: 0 +``` + +{{< /collapse >}} + +{{< collapse title="Azure" >}} + +```yaml +loki: + schemaConfig: + configs: + - from: "2024-04-01" + store: tsdb + object_store: azure + schema: v13 + index: + prefix: loki_index_ + period: 24h + ingester: + chunk_encoding: snappy + + storage: + type: azure + azure: + # Name of the Azure Blob Storage account + accountName: + # Key associated with the Azure Blob Storage account + accountKey: + # Comprehensive connection string for Azure Blob Storage account (Can be used to replace endpoint, accountName, and accountKey) + connectionString: + # Flag indicating whether to use Azure Managed Identity for authentication + useManagedIdentity: false + # Flag indicating whether to use a federated token for authentication + useFederatedToken: false + # Client ID of the user-assigned managed identity (if applicable) + userAssignedId: + # Timeout duration for requests made to the Azure Blob Storage account (in seconds) + requestTimeout: + # Domain suffix of the Azure Blob Storage service endpoint (e.g., core.windows.net) + endpointSuffix: + bucketNames: + chunks: "chunks" + ruler: "ruler" + admin: "admin" + +# Disable minio storage +minio: + enabled: false + +singleBinary: + replicas: 3 + persistence: + storageClass: gp2 + accessModes: + - ReadWriteOnce + size: 30Gi + +# Zero out replica counts of other deployment modes +backend: + replicas: 0 +read: + replicas: 0 +write: + replicas: 0 + +ingester: + replicas: 0 +querier: + replicas: 0 +queryFrontend: + replicas: 0 +queryScheduler: + replicas: 0 +distributor: + replicas: 0 +compactor: + replicas: 0 +indexGateway: + replicas: 0 +bloomCompactor: + replicas: 0 +bloomGateway: + replicas: 0 + +``` + +{{< /collapse >}} + + + +To configure other storage providers, refer to the [Helm Chart Reference](https://grafana.com/docs/loki//setup/install/helm/reference/). + +## Deploying the Loki Helm chart to a Production Environment + +{{< admonition type="note" >}} +We are actively working on providing more guides for deploying Loki in production. +{{< /admonition >}} + +We recommend running Loki at scale within a cloud environment like AWS, Azure, or GCP. The below guides will show you how to deploy a minimally viable production environment. +- [Deploy Loki on AWS](https://grafana.com/docs/loki//setup/install/helm/deployment-guides/aws) + + +## Next Steps +* Configure an agent to [send log data to Loki](/docs/loki//send-data/). +* Monitor the Loki deployment using the [Meta Monitoring Helm chart](/docs/loki//setup/install/helm/monitor-and-alert/) - ```bash - helm install --values values.yaml loki --namespace=loki grafana/loki - ``` diff --git a/docs/sources/setup/install/helm/install-scalable/_index.md b/docs/sources/setup/install/helm/install-scalable/_index.md index c11f37c2d58f0..3b5a13c572fc5 100644 --- a/docs/sources/setup/install/helm/install-scalable/_index.md +++ b/docs/sources/setup/install/helm/install-scalable/_index.md @@ -11,7 +11,7 @@ keywords: # Install the simple scalable Helm chart -This Helm Chart deploys Grafana Loki on Kubernetes. +This Helm Chart deploys Grafana Loki in [simple scalable mode](https://grafana.com/docs/loki//get-started/deployment-modes/#simple-scalable) within a Kubernetes cluster. This chart configures Loki to run `read`, `write`, and `backend` targets in a [scalable mode]({{< relref "../../../../get-started/deployment-modes#simple-scalable" >}}). Loki’s simple scalable deployment mode separates execution paths into read, write, and backend targets. @@ -22,17 +22,20 @@ The default Helm chart deploys the following components: - Loki Canary (1 DaemonSet) - Gateway (1 NGINX replica) - Minio (optional, if `minio.enabled=true`) +- Index and Chunk cache (1 replica) -It is not recommended to run scalable mode with `filesystem` storage. For the purpose of this guide, we will use MinIO as the object storage to provide a complete example. +{{< admonition type="note" >}} +We do not recommended running scalable mode with `filesystem` storage. For the purpose of this guide, we will use MinIO as the object storage to provide a complete example. +{{< /admonition >}} -**Prerequisites** +## Prerequisites - Helm 3 or above. See [Installing Helm](https://helm.sh/docs/intro/install/). - A running Kubernetes cluster (must have at least 3 nodes). -- (Optional) A Memcached deployment for better query performance. For information on configuring Memcached, refer to [caching section]({{< relref "../../../../operations/caching" >}}). +## Deploying the Helm chart for development and testing -**To deploy Loki in simple scalable mode:** +The following steps show how to deploy the Loki Helm chart in simple scalable mode using the included MinIO as the storage backend. Our recommendation is to start here for development and testing purposes. Then configure Loki with an object storage provider when moving to production. 1. Add [Grafana's chart repository](https://github.com/grafana/helm-charts) to Helm: @@ -49,72 +52,52 @@ It is not recommended to run scalable mode with `filesystem` storage. For the pu 3. Create the configuration file `values.yaml`. The example below illustrates how to deploy Loki in test mode using MinIO as storage: - ```yaml - loki: - schemaConfig: - configs: - - from: "2024-04-01" - store: tsdb - object_store: s3 - schema: v13 - index: - prefix: loki_index_ - period: 24h - ingester: - chunk_encoding: snappy - tracing: - enabled: true - querier: - # Default is 4, if you have enough memory and CPU you can increase, reduce if OOMing - max_concurrent: 4 - - #gateway: - # ingress: - # enabled: true - # hosts: - # - host: FIXME - # paths: - # - path: / - # pathType: Prefix - - deploymentMode: SimpleScalable - - backend: - replicas: 3 - read: - replicas: 3 - write: - replicas: 3 - - # Enable minio for storage - minio: - enabled: true - - # Zero out replica counts of other deployment modes - singleBinary: - replicas: 0 - - ingester: - replicas: 0 - querier: - replicas: 0 - queryFrontend: - replicas: 0 - queryScheduler: - replicas: 0 - distributor: - replicas: 0 - compactor: - replicas: 0 - indexGateway: - replicas: 0 - bloomCompactor: - replicas: 0 - bloomGateway: - replicas: 0 - ``` - -4. Install or upgrade the Loki deployment. + ```yaml + loki: + schemaConfig: + configs: + - from: "2024-04-01" + store: tsdb + object_store: s3 + schema: v13 + index: + prefix: loki_index_ + period: 24h + ingester: + chunk_encoding: snappy + querier: + # Default is 4, if you have enough memory and CPU you can increase, reduce if OOMing + max_concurrent: 4 + pattern_ingester: + enabled: true + limits_config: + allow_structured_metadata: true + volume_enabled: true + retention_period: 672h + compactor: + retention_enabled: true + delete_request_store: s3 + + deploymentMode: SimpleScalable + + backend: + replicas: 2 + read: + replicas: 2 + write: + replicas: 3 # To ensure data durability with replication + + # Enable minio for storage + minio: + enabled: true + + gateway: + service: + type: LoadBalancer + ``` + +1. Install or upgrade the Loki deployment. + - To install: ```bash helm install --values values.yaml loki grafana/loki @@ -126,11 +109,15 @@ It is not recommended to run scalable mode with `filesystem` storage. For the pu ## Object Storage Configuration -After testing Loki with MinIO, it is recommended to configure Loki with an object storage provider. The following examples shows how to configure Loki with different object storage providers: +After testing Loki with MinIO, we recommend configuring Loki with an object storage provider. The following examples shows how to configure Loki with different object storage providers: + +{{< admonition type="caution" >}} +When deploying Loki using S3 Storage **DO NOT** use the default bucket names; `chunk`, `ruler` and `admin`. Choose a unique name for each bucket. For more information see the following [security update](https://grafana.com/blog/2024/06/27/grafana-security-update-grafana-loki-and-unintended-data-write-attempts-to-amazon-s3-buckets/). This caution does not apply when you are using MinIO. When using MinIO we recommend using the default bucket names. +{{< /admonition >}} -{{< code >}} +{{< collapse title="S3" >}} -```s3 +```yaml loki: schemaConfig: configs: @@ -141,21 +128,28 @@ loki: index: prefix: loki_index_ period: 24h - ingester: - chunk_encoding: snappy - tracing: - enabled: true + storage_config: + aws: + region: + bucketnames: + s3forcepathstyle: false + pattern_ingester: + enabled: true + limits_config: + allow_structured_metadata: true + volume_enabled: true + retention_period: 672h # 28 days retention querier: max_concurrent: 4 storage: type: s3 bucketNames: - chunks: "chunks" - ruler: "ruler" - admin: "admin" + chunks: + ruler: + admin: s3: - # s3 URL can be used to specify the endpoint, access key, secret key, and bucket name + # s3 URL can be used to specify the endpoint, access key, secret key, and bucket name this works well for S3 compatible storages or if you are hosting Loki on-premises and want to use S3 as the storage backend. Either use the s3 URL or the individual fields below (AWS endpoint, region, secret). s3: s3://access_key:secret_access_key@custom_endpoint/bucket_name # AWS endpoint URL endpoint: @@ -186,31 +180,14 @@ write: # Disable minio storage minio: enabled: false - -# Zero out replica counts of other deployment modes -singleBinary: - replicas: 0 - -ingester: - replicas: 0 -querier: - replicas: 0 -queryFrontend: - replicas: 0 -queryScheduler: - replicas: 0 -distributor: - replicas: 0 -compactor: - replicas: 0 -indexGateway: - replicas: 0 -bloomCompactor: - replicas: 0 -bloomGateway: - replicas: 0 ``` -```azure + +{{< /collapse >}} + +{{< collapse title="Azure" >}} + +```yaml + loki: schemaConfig: configs: @@ -265,31 +242,9 @@ write: minio: enabled: false -# Zero out replica counts of other deployment modes -singleBinary: - replicas: 0 - -ingester: - replicas: 0 -querier: - replicas: 0 -queryFrontend: - replicas: 0 -queryScheduler: - replicas: 0 -distributor: - replicas: 0 -compactor: - replicas: 0 -indexGateway: - replicas: 0 -bloomCompactor: - replicas: 0 -bloomGateway: - replicas: 0 -``` -{{< /code >}} +``` +{{< /collapse >}} To configure other storage providers, refer to the [Helm Chart Reference]({{< relref "../reference" >}}).