[BUG]: Non-Dynamic S3 Configuration in Single-Node Opensearch Setup Fails to Restart in GitLab CI/CD

[IvanZenger]

Describe the bug
I am setting up Opensearch as a Single-Node within a GitLab CI/CD Service and need to configure the following: s3.client.backup.endpoint: s3-endpoint.foo.ch and s3.client.backup.region: foobaar.

The problem is that these configurations are not dynamic, requiring a restart of the node. However, such a restart is not feasible within a GitLab Service.

Using environment variables (ENV) is not a solution in this case, as the opensearch-docker-entrypoint.sh script (see opensearch-project/opensearch-build) only processes two-level configurations.

To Reproduce
Steps to reproduce the behavior:

1. Start GitLab Job (Simplified):

integration-test:
  stage: pre-test
  variables:
    FF_NETWORK_PER_BUILD: 1
    OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m"
    CI_DEBUG_SERVICES: "true"
  services:
    - name: opensearchproject/opensearch:2.9.0
      alias: opensearch
      entrypoint: ["bash", "-c", "env 's3.client.backup.endpoint=s3-endpint.foo.ch' 's3.client.backup.region=foobar' 'discovery.type=single-node' 'cluster.name=opensearch' ./opensearch-docker-entrypoint.sh"]
  script:
    - > #https://gitlab.com/gitlab-org/gitlab/-/issues/24197
      until curl -o /dev/null -sSf -w '%{http_code}' https://opensearch:9200/_cat/health --insecure -u "admin:admin" | grep -q '200'; do
        echo 'Waiting for OpenSearch to be ready...'; sleep 10;
      done

Or alternatively, a corresponding docker run command

Expected behavior
The configuration I have set as an environment variable (ENV) is being applied as settings (like discovery.type=single-node).

Plugins
repository-s3

Host/Environment (please complete the following information):

• OS: [RHEL8]
• Opensearch Version 22.9.0

[peternied]

[Triage - attendees 1 2 3 4]
@IvanZenger Thanks for filing

[peternied]

@reta Did you have thoughts about this issue?

[gbbafna]

@IvanZenger , have you tried out reloadable repository introduced in 2.11 ? Please refer to this changes .

[ashking94]

[Triage - attendees 1 2 3 4 5 6 7 8 9 10 11 12]
We are closing this issue. Please do reopen if you have any questions.

[peternied]

@ashking94 Could you include why this issue was closed? There are templates [1] for typical responses you can reuse

• [1] https://github.com/opensearch-project/OpenSearch/blob/main/TRIAGING.md#what-are-the-typical-outcomes-of-a-triaged-issue