Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: [release-3.0.x] docs: Update storage topics (v3.0) #12781

Merged
merged 1 commit into from
Apr 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions docs/sources/configure/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@ Single Store refers to using object storage as the storage medium for both Loki'

### TSDB (recommended)

Starting in Loki 2.8, the [TSDB index store](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/storage/tsdb/) improves query performance, reduces TCO and has the same feature parity as "boltdb-shipper".
Starting in Loki 2.8, the [TSDB index store](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/storage/tsdb/) improves query performance, reduces TCO and has the same feature parity as "boltdb-shipper". TSDB is the recommended index store for Loki 2.8 and newer.

### BoltDB (deprecated)

Also known as "boltdb-shipper" during development (and is still the schema `store` name). The single store configurations for Loki utilize the chunk store for both chunks and the index, requiring just one store to run Loki.
Also known as "boltdb-shipper" during development (and is still the schema `store` name). The single store configurations for Loki utilize the chunk store for both chunks and the index, requiring just one store to run Loki. BoldDB is the recommended index store for Loki v2.0.0 through v2.7x.

Performance is comparable to a dedicated index type while providing a much less expensive and less complicated deployment.
When using Single Store, no extra [Chunk storage](#chunk-storage) and [Index storage](#index-storage) are necessary.
Expand Down Expand Up @@ -172,7 +172,7 @@ For more information, see the [table manager](https://grafana.com/docs/loki/<LOK

In the case of AWS DynamoDB, you'll likely want to tune the provisioned throughput for your tables as well. This is to prevent your tables being rate limited on one hand and assuming unnecessary cost on the other. By default Loki uses a [provisioned capacity](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadWriteCapacityMode.html) strategy for DynamoDB tables like so:

```
```yaml
table_manager:
index_tables_provisioning:
# Read/write throughput requirements for the current table
Expand All @@ -194,6 +194,7 @@ When a new schema is released and you want to gain the advantages it provides, y
First, you'll want to create a new [period_config](https://grafana.com/docs/loki/<LOKI_VERSION>/configure/#period_config) entry in your [schema_config](https://grafana.com/docs/loki/<LOKI_VERSION>/configure/#schema_config). The important thing to remember here is to set this at some point in the _future_ and then roll out the config file changes to Loki. This allows the table manager to create the required table in advance of writes and ensures that existing data isn't queried as if it adheres to the new schema.

As an example, let's say it's 2023-07-14 and we want to start using the `v13` schema on the 20th:

```yaml
schema_config:
configs:
Expand Down Expand Up @@ -223,7 +224,6 @@ We're interested in adding targeted deletion in future Loki releases (think tena

For more information, see the [retention configuration](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/storage/retention/) documentation.


## Examples

### Single machine/local development (boltdb+filesystem)
Expand Down Expand Up @@ -314,7 +314,7 @@ The role should have a policy with the following permissions attached.
}
```

**To setup an S3 bucket and an IAM role and policy:**
**To setup an S3 bucket and an IAM role and policy:**

This guide assumes a provisioned EKS cluster.

Expand All @@ -324,14 +324,14 @@ This guide assumes a provisioned EKS cluster.

3. Export the AWS profile and region if not done so:

```
```bash
export AWS_PROFILE=<profile in ~/.aws/config>
export AWS_REGION=<region of EKS cluster>
```

4. Save the OIDC provider in an environment variable:

```
```bash
oidc_provider=$(aws eks describe-cluster --name <EKS cluster> --query "cluster.identity.oidc.issuer" --output text | sed -e "s/^https:\/\///")
```

Expand All @@ -342,7 +342,6 @@ This guide assumes a provisioned EKS cluster.
Note, the bucket name defaults to `loki-data` but can be changed via the
`bucket_name` variable.


### Azure deployment (Azure Blob Storage Single Store)

#### Using account name and key
Expand Down
8 changes: 2 additions & 6 deletions docs/sources/operations/storage/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,8 @@ For more information:

### ✅ Supported index stores

- [Single Store TSDB](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/storage/tsdb/) index store which stores TSDB index files in the object store.
**This is the recommended index store for Loki 2.8 and newer.**
- [Single Store BoltDB (boltdb-shipper)](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/storage/boltdb-shipper/) index store which stores boltdb index files in the object store.
- [Single Store TSDB](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/storage/tsdb/) index store which stores TSDB index files in the object store. This is the recommended index store for Loki 2.8 and newer.
- [Single Store BoltDB (boltdb-shipper)](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/storage/boltdb-shipper/) index store which stores boltdb index files in the object store. Recommended store for Loki 2.0 through 2.7.x.

### ❌ Deprecated index stores

Expand Down Expand Up @@ -61,7 +60,6 @@ For more information:
- [Google Bigtable](https://cloud.google.com/bigtable). Support for this is deprecated and will be removed in a future release.
- [Apache Cassandra](https://cassandra.apache.org). Support for this is deprecated and will be removed in a future release.


## Cloud Storage Permissions

### S3
Expand Down Expand Up @@ -128,7 +126,6 @@ Resources: `*`

Resources: `arn:aws:iam::<aws_account_id>:role/<role_name>`


### IBM Cloud Object Storage

When using IBM Cloud Object Storage (COS) as object storage, IAM `Writer` role is needed.
Expand Down Expand Up @@ -164,4 +161,3 @@ See the [IBM Cloud Object Storage section](https://grafana.com/docs/loki/<LOKI_V
| metasOffset - offset to the point with #blocks |
-------------------------------------------------------------------
```

4 changes: 1 addition & 3 deletions docs/sources/operations/storage/boltdb-shipper.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ weight: 200
# Single Store BoltDB (boltdb-shipper)

{{% admonition type="note" %}}
Single store BoltDB Shipper is a legacy storage option and is not recommended for new deployments. The [TSDB]({{< relref "./tsdb" >}}) index is the recommended index.
Single store BoltDB Shipper is a legacy storage option recommended for Loki 2.0 through 2.7.x and is not recommended for new deployments. The [TSDB](https://grafana.com/docs/loki/<LOKI_VERSION>/operations/storage/tsdb/) is the recommended index for Loki 2.8 and newer.
{{% /admonition %}}

BoltDB Shipper lets you run Grafana Loki without any dependency on NoSQL stores for storing index.
Expand Down Expand Up @@ -155,5 +155,3 @@ storage_config:
gcs:
bucket_name: GCS_BUCKET_NAME
```


Loading