forked from grafana/loki
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
docs: add migration guide for tsdb (grafana#10978)
**What this PR does / why we need it**: Adds a migration guide for moving from any of the older indexes to TSDB **Special notes for your reviewer**: **Checklist** - [x] Reviewed the [`CONTRIBUTING.md`](https://github.com/grafana/loki/blob/main/CONTRIBUTING.md) guide (**required**) - [x] Documentation added - [ ] Tests updated - [ ] `CHANGELOG.md` updated - [ ] If the change is worth mentioning in the release notes, add `add-to-release-notes` label - [ ] Changes that require user attention or interaction to upgrade are documented in `docs/sources/setup/upgrade/_index.md` - [ ] For Helm chart changes bump the Helm chart version in `production/helm/loki/Chart.yaml` and update `production/helm/loki/CHANGELOG.md` and `production/helm/loki/README.md`. [Example PR](grafana@d10549e)
- Loading branch information
1 parent
1c56aa9
commit a596161
Showing
2 changed files
with
75 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,74 @@ | ||
--- | ||
title: Migrate to TSDB | ||
menuTitle: Migrate to TSDB | ||
description: Migration guide for moving from any of the older indexes to TSDB | ||
weight: 100 | ||
keywords: | ||
- migrate | ||
- tsdb | ||
--- | ||
|
||
# Migrate to TSDB | ||
|
||
[TSDB]({{< relref "../../../operations/storage/tsdb" >}}) is the recommeneded index type for Loki and is where the current development lies. | ||
If you are running Loki with [boltb-shipper]({{< relref "../../../operations/storage/boltdb-shipper" >}}) or any of the [legacy index types]({{< relref "../../../storage#index-storage" >}}) that have been deprecated, | ||
we strongly recommend migrating to TSDB. | ||
|
||
|
||
### Configure TSDB index for an upcoming period | ||
|
||
To begin the migration, add a new [period_config]({{< relref "../../../configure#period_config" >}}) entry in your [schema_config]({{< relref "../../../configure#schema_config" >}}). | ||
You can read more about schema config [here]({{< relref "../../../storage#schema-config" >}}). | ||
|
||
{{% admonition type="note" %}} | ||
You must roll out the new `period_config` change to all Loki components in order for it to take effect. | ||
{{% /admonition %}} | ||
|
||
This example adds a new `period_config` which configures Loki to start using the TSDB index for the data ingested starting from `2023-10-20`. | ||
|
||
``` | ||
schema_config: | ||
configs: | ||
- from: 2023-01-01 | ||
store: boltdb-shipper | ||
object_store: filesystem | ||
schema: v11 | ||
index: | ||
prefix: index_ | ||
period: 24h | ||
- from: 2023-10-20 ① | ||
store: tsdb ② | ||
object_store: filesystem ③ | ||
schema: v12 ④ | ||
index: | ||
prefix: index_ | ||
period: 24h | ||
``` | ||
|
||
① You must set the new period `from` to a date in the future. | ||
|
||
② Update the new period to use TSDB as the index type by setting `store: tsdb`. | ||
|
||
③ This sample configuration uses filesystem as the storage in both the periods. If you want to use a different storage for the TSDB index and chunks, you can specify a different `object_store` in the new period. | ||
|
||
④ Update the schema to v12 which is the recommended version at the time of writing. Please refer to the [configure page]({{< relref "../../../configure#period_config" >}}) for the current recommend version. | ||
|
||
### Configure TSDB shipper | ||
|
||
It's also important that you configure the `tsdb_shipper` block in [storage_config]({{< relref "../../../configure#storage_config" >}}). Specifically the following options: | ||
- `active_index_directory`: directory where ingesters would write index files which will then be uploaded by shipper to configured storage. | ||
- `cache_location`: cache location for downloading index files from the storage for use in query path. | ||
|
||
``` | ||
storage_config: | ||
tsdb_shipper: | ||
active_index_directory: /data/tsdb-index | ||
cache_location: /data/tsdb-cache | ||
``` | ||
|
||
### Run compactor | ||
|
||
We strongly recommended running the [compactor]({{< relref "../../../operations/storage/retention#compactor" >}}) when using TSDB index. It is responsible for running compaction and retention on TSDB index. | ||
Not running index compaction will result in sub-optimal query performance. | ||
|
||
Please refer to the [compactor section]({{< relref "../../../operations/storage/retention#compactor" >}}) for more information and configuration examples. |