Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comment out hard coded opensearch.yml #504

Closed

Conversation

derek-ho
Copy link

@derek-ho derek-ho commented Dec 7, 2023

Description

This PR stops feeding in a hard coded opensearch.yml into the pods. We are expecting to generate this output from running the install demo configuration script from the security plugin side.

Issues Resolved

[List any issues this PR will resolve. You should likely open an issue if one does not already exist.]

Check List

  • Commits are signed per the DCO using --signoff

For any changes to files within Helm chart directories:

  • Helm chart version bumped
  • Helm chart CHANGELOG.md updated to reflect change

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Comment on lines -34 to -60
config:
# Values must be YAML literal style scalar / YAML multiline string.
# <filename>: |
# <formatted-value(s)>
# log4j2.properties: |
# status = error
#
# appender.console.type = Console
# appender.console.name = console
# appender.console.layout.type = PatternLayout
# appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %m%n
#
# rootLogger.level = info
# rootLogger.appenderRef.console.ref = console
opensearch.yml: |
cluster.name: opensearch-cluster

# Bind to all interfaces because we don't know what IP address Docker will assign to us.
network.host: 0.0.0.0

# Setting network.host to a non-loopback address enables the annoying bootstrap checks. "Single-node" mode disables them again.
# Implicitly done if ".singleNode" is set to "true".
# discovery.type: single-node

# Start OpenSearch Security Demo Configuration
# WARNING: revise all the lines below before you go into production
plugins:
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file should be generated via the demo configuration script, if it is hard coded we will need to maintain this in two places.

Copy link
Member

@TheAlgo TheAlgo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this sample yaml is very helpful for anyone starting opensearch right?

@smlx
Copy link
Contributor

smlx commented Jan 13, 2024

I don't understand this change. This is a helm chart, they are configured via values files.. how are you generating your configuration otherwise?

@DarshitChanpura
Copy link
Member

DarshitChanpura commented Jan 18, 2024

@smlx @TheAlgo When an OpenSearch image is downloaded and cluster is spun up, the Security plugin runs install demo configuration by default as part of install. During this phase, opensearch.yml is loaded with security related values. Hence we do not need to hardcode opensearch.yml. Feel free to run this locally and lmk if otherwise.

charts/opensearch/CHANGELOG.md Outdated Show resolved Hide resolved
charts/opensearch/Chart.yaml Outdated Show resolved Hide resolved
charts/opensearch/CHANGELOG.md Outdated Show resolved Hide resolved
charts/opensearch/CHANGELOG.md Outdated Show resolved Hide resolved
@derek-ho derek-ho changed the title Remove hard coded opensearch.yml Comment out hard coded opensearch.yml Jan 29, 2024
@DarshitChanpura
Copy link
Member

@prudhvigodithi Could you please review this?

@DarshitChanpura
Copy link
Member

@prudhvigodithi Could you please add 1 more review?

@prudhvigodithi
Copy link
Member

prudhvigodithi commented Feb 6, 2024

Hey Just tested this and works fine with removing the .Values.config , basically when user adds the opensearch.yml through .Values.config, it gets added to /usr/share/opensearch/config/opensearch.yml overriding the default. Also one advantage is next if there is any change in opensearch.yml and added through .Values.config, the pods get auto restarted and loads the new values from opensearch.yml. So @DarshitChanpura and @derek-ho any specific reason we are removing, we can leave it as well right ?

Adding @TheAlgo @smlx @bbarani @peterzhuamazon

@DarshitChanpura
Copy link
Member

So @DarshitChanpura and @derek-ho any specific reason we are removing, we can leave it as well right ?

From what I understand, the opensearch.yml always gets created when demo configuration script is run (which is default). If in case the user decides to pass in custom configuration, they can always pass it in via .values.yml. In either of these two cases, the hardcoded opensearch.yml serves no purpose. Which is why we are removing this? @derek-ho Please add if I missed anything here.

@smlx
Copy link
Contributor

smlx commented Feb 7, 2024

I don't know if it is on the roadmap of the Opensearch image to avoid generating the demo config at runtime but IMO it should be. Generating config at runtime has the following consequences for docker and k8s:

  • it slows down container startup
  • root filesystem can't be read only
  • can't run the container as an arbitrary user (has to have permissions to write the config file)

IMO it makes more sense to aim for immutable images with defined injection points for configuration rather than generating config at runtime. Therefore I think this PR is a move in the wrong direction.

@prudhvigodithi
Copy link
Member

Instead of commenting out opensearch.yml, I have raised a PR #516, that address configMap Read-only file system error when passed via .Values.config, with this the user can either use the default opensearch.yml or pass the opensearch.yml via .Values.config. The file will be created as -rw-r--r-- 1 opensearch opensearch 3646 Feb 7 03:22 opensearch.yml, having RW access within the cluster, please check.

@TheAlgo @DarshitChanpura @peterzhuamazon @bbarani @derek-ho @smlx

@derek-ho
Copy link
Author

derek-ho commented Feb 7, 2024

I can close this out, since I tested out the alternative also works and it seems the direction of the community is not towards removing this.

@derek-ho derek-ho closed this Feb 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants