Skip to content

Commit

Permalink
fix: udpated note content and location
Browse files Browse the repository at this point in the history
  • Loading branch information
harshini-rangaswamy committed Sep 11, 2024
1 parent 11f158d commit 636a71e
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 45 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -52,11 +52,6 @@ Before starting the migration, ensure the following:

Migrate your security configuration from Opendistro to Aiven for OpenSearch.

:::note
Exclude the `.opendistro_security` index pattern from your snapshot restore before
running the migration script.
:::

### Access the migration script

The migration script is available in the [Aiven examples GitHub repository](https://github.com/aiven/aiven-examples/blob/main/solutions/migrate-opendistro-security-to-aiven-for-opensearch/avn-migrate-os-security-config.py).
Expand Down
83 changes: 43 additions & 40 deletions docs/products/opensearch/howto/migrate-snapshot-data-opensearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,65 +31,64 @@ Before you begin, ensure that:
- Your OpenSearch or Elasticsearch snapshot is from version 7.10.2 or earlier and is
compatible with your target service.


### Gather required parameters

Gather these details before registering the snapshot repository:

- `API_TOKEN`: Your [Aiven token](/docs/platform/concepts/authentication-tokens).
- `PROJECT_NAME`: Name of your Aiven project.
- `SERVICE_NAME`: Name of your Aiven for OpenSearch service.
- `API_TOKEN`: Your [Aiven token](/docs/platform/concepts/authentication-tokens)
- `PROJECT_NAME`: Name of your Aiven project
- `SERVICE_NAME`: Name of your Aiven for OpenSearch service

Information specific to cloud providers:

- **Amazon S3**

- `snapshot_name`: The name of the snapshot to restore.
- `base_path`: The path within the S3 bucket where the snapshot data is stored.
- `bucket`: The S3 bucket name.
- `region`: The AWS region of the S3 bucket.
- `access_key`: AWS access key for accessing the S3 bucket.
- `secret_key`: AWS secret key associated with the access key.
- `server_side_encryption`: (Optional) Enable server-side encryption for files in the
S3 bucket.
- `endpoint`: (Optional) The endpoint for S3-compatible services if not using AWS S3
directly.
- `indices`: (Optional) A comma-separated list of index patterns to restore specific
indices. If no patterns are provided, all indices are restored by default.
- `snapshot_name`: The name of the snapshot to restore
- `base_path`: The path within the S3 bucket where the snapshot data is stored
- `bucket`: The S3 bucket name
- `region`: The AWS region of the S3 bucket
- `access_key`: AWS access key for accessing the S3 bucket
- `secret_key`: AWS secret key associated with the access key
- `server_side_encryption`: Optional. Enable server-side encryption for files in the
S3 bucket
- `endpoint`: Optional. The endpoint for S3-compatible services if not using AWS S3
directly
- `indices`: Optional. Comma-separated list of index patterns to restore specific
indices. If no patterns are provided, all indices are restored by default

- **Google Cloud Storage (GCS)**

- `credentials`: GCS credentials file content.
- `bucket`: Name of the GCS bucket that contains the snapshot.
- `base_path`: Path to repository data within the bucket.
- `credentials`: GCS credentials file content
- `bucket`: Name of the GCS bucket that contains the snapshot
- `base_path`: Path to repository data within the bucket
- `snapshot_name`: Name of the snapshot to restore.
- `compress` and `chunk_size`: (Optional) Settings for metadata compression
and file chunking.
- `indices`: (Optional) A comma-separated list of index patterns to restore specific
indices. If no patterns are provided, all indices are restored by default.
- `compress` and `chunk_size`: Optional. Settings for metadata compression
and file chunking
- `indices`: Optional. A comma-separated list of index patterns to restore specific
indices. If no patterns are provided, all indices are restored by default

- **Microsoft Azure**

- `account`: Azure account name.
- `key` or `sas_token`: Azure secret key or shared access signature token.
- `container`: Name of the Azure container that contains the snapshot.
- `base_path`: Path to repository data within the container.
- `snapshot_name`: Name of the snapshot to restore.
- `compress`, `chunk_size`, `endpoint_suffix`: (Optional) Additional configuration
settings.
- `indices`: (Optional) A comma-separated list of index patterns to restore specific
indices. If no patterns are provided, all indices are restored by default.
- `account`: Azure account name
- `key` or `sas_token`: Azure secret key or shared access signature token
- `container`: Name of the Azure container that contains the snapshot
- `base_path`: Path to repository data within the container
- `snapshot_name`: Name of the snapshot to restore
- `compress`, `chunk_size`, `endpoint_suffix`: Optional. Additional configuration
settings
- `indices`: Optional. Comma-separated list of index patterns to restore specific
indices. If no patterns are provided, all indices are restored by default

- **S3-compatible services**

- `endpoint`: Service endpoint for S3-compatible services.
- `access_key`: Access key for the S3-compatible service.
- `secret_key`**: Secret key for the S3-compatible service.
- `region`: Region or endpoint-specific region.
- `bucket`, `base_path`, `snapshot_name`: Bucket details and snapshot name.
- `server_side_encryption`, `compress`, `chunk_size`: (Optional) Settings
for encryption, compression, and file chunking.
- `indices`: (Optional) A comma-separated list of index patterns to restore specific
- `endpoint`: Service endpoint for S3-compatible services
- `access_key`: Access key for the S3-compatible service
- `secret_key`**: Secret key for the S3-compatible service
- `region`: Region or endpoint-specific region
- `bucket`, `base_path`, `snapshot_name`: Bucket details and snapshot name
- `server_side_encryption`, `compress`, `chunk_size`: Optional. Settings
for encryption, compression, and file chunking
- `indices`: Optional. Comma-separated list of index patterns to restore specific
indices. If no patterns are provided, all indices are restored by default

## Configure snapshot migration settings
Expand All @@ -103,6 +102,10 @@ field within the `user-config` object during configuration. If no index patterns
provided, Aiven restores all indices by default. For example, to restore indices starting
with `logs-` and `metrics-`, use: `indices: "logs-*,metrics-*"`.

:::note
Exclude the `.opendistro_security` index pattern from your snapshot restore process.
:::

:::warning
Aiven for OpenSearch only allows one migration to progress at a time. Ensure the current
migration finishes before starting a new one to avoid interruptions.
Expand Down

0 comments on commit 636a71e

Please sign in to comment.