diff --git a/json-schema/platforms/custom-s3.json b/json-schema/platforms/custom-s3.json index 25a89b7..632ae12 100644 --- a/json-schema/platforms/custom-s3.json +++ b/json-schema/platforms/custom-s3.json @@ -11,6 +11,11 @@ } }, "then": { - "$comment": "No specific validation rules apply" + "bucket": { + "type": "string" + }, + "region": { + "type": "string" + } } } \ No newline at end of file diff --git a/platforms/aws-s3.md b/platforms/aws-s3.md index 93fde52..08751fd 100644 --- a/platforms/aws-s3.md +++ b/platforms/aws-s3.md @@ -2,6 +2,10 @@ This defines the Amazon Web Services (AWS) S3 interface. -- `platform`: `https://{bucket}.s3.{region}.amazonaws.com` -- `bucket`: The bucket name -- `region`: One of the S3 regions (lowercase) +- `platform`: `https://{bucket}.s3.{region}.amazonaws.com`, + which is the endpoint URL after replacing all variables in the URL. +- `bucket`: The bucket name. +- `region`: One of the S3 regions (lowercase). + +**Note:** If the `s3` authentication scheme (i.e. "Simple S3 authentication") is referred to through `auth:refs`, you should disable signing requests, +e.g. using the AWS CLI parameter `--no-sign-request`. diff --git a/platforms/custom-s3.md b/platforms/custom-s3.md index c6c5ebd..3b15426 100644 --- a/platforms/custom-s3.md +++ b/platforms/custom-s3.md @@ -2,4 +2,40 @@ This defines the S3 interface for providers other than AWS (e.g. minio-based). -- `platform`: The API URL, must be the endpoint URL that can be used for the AWS CLI for example. +- `platform`: The API URL (template), must be the endpoint URL that can be used for the AWS CLI for example, e.g. `https://{bucket}.example.com` or `http://example.com:9000`. +- `bucket`: The bucket name, if applicable. +- `region`: The region, if applicable. + +## Mapping to S3 tooling + +### GDAL (`/vsis3/`) + +GDAL documentation: + +- `platform`: Some options for S3 can be inferred from the given URL (template): + - `AWS_HTTPS` can be retrieved by parsing the scheme part of the URL. `https` = `ON`, `http` = `OFF`. + - `AWS_S3_ENDPOINT` is the authority part of the URL after replacing all variables in the URL, + e.g. `us-west.mycloud.com` without `https://` or `s3://` as prefix. + - `AWS_VIRTUAL_HOSTING` must be set to `FALSE` if there's no `{bucket}` placeholder in the URL template, otherwise `TRUE` (default value). +- The `region` property corresponds to the `AWS_REGION` option. +- The `requester_pays` property corresponds to the `AWS_REQUEST_PAYER` option. If `requester_pays` is `true`, set `AWS_REQUEST_PAYER` to `requester`. +- If the `s3` authentication scheme (i.e. "Simple S3 authentication") is referred to through `auth:refs`, + you should set `AWS_NO_SIGN_REQUEST` to `NO`. Otherwise it should be `YES`. + +### AWS CLI + +AWS CLI documentation: + +- `platform` corresponds to `--endpoint-url` after replacing all variables in the URL. +- `region` corresponds to `--region`. +- If `s3` is **missing** from `auth:refs`, you should use `--no-sign-request`. + +### s3cmd + +s3cmd documentation: + +- `platform` corresponds to `--host` / `--host-bucket` after replacing all variables in the URL (?). +- `region` corresponds to `--region`. +- `requester_pays` corresponds to `--requester-pays`. +- If the `s3` authentication scheme (i.e. "Simple S3 authentication") is referred to through `auth:refs`, + you should provide an secret access key and an access key id through environment variables, a profile or the `s3cmd sign` command.