fix openshift values to default with in-cluster prometheus

[mittal-ishaan]

What does this PR change?

• This PR adds an example values file to have default properties set to deploy kubecost with in-cluster prometheus. (As this is what most users want)
• Add the general configurations to default values file and remove unnecessary configurations and comments from openshift and openshift prometheus example values file.
• Adds clarifying comment in the template while binding the cluster role to the kubecost service account
• Moves in-cluster prometheus service account namespace name to values to make it more generalized and to cover the case where it can be of any other name.

Does this PR rely on any other PRs?

No

How does this PR impact users? (This is the kind of thing that goes in release notes!)

Added standard configuration template for openshift to use the openshift-monitoring prometheus and cleaned up openshift example values file.

Links to Issues or tickets this PR addresses or fixes

What risks are associated with merging this PR? What is required to fully test this PR?

No risk, Testing that it gets deployed on openshift cluster easily and successfully need to be done.

How was this PR tested?

Installing kubecost in openshift environment with network cost enabled without helm hook:

Installing with pre-install helm hook

Resulting kubecost diagnostics page

Have you made an update to documentation? If so, please provide the corresponding PR.

[chipzoller]

I'm wondering if this is the right approach (on by default) as it deviates from our standard architecture. How extensively has this been tested? Do you have any concerns here, @jessegoodier and @kwombach12 ?

[jessegoodier]

I'm wondering if this is the right approach (on by default) as it deviates from our standard architecture. How extensively has this been tested? Do you have any concerns here, @jessegoodier and @kwombach12 ?

This is what users are asking for, and I do think it is the best architecture.

A consistent configuration will reduce support concerns with openshift- in comparison to what we have seen with the various custom configs attempted.

Historically, issues with prometheus are due to misconfiguration, not version.

[thomasvn]

Also any new configurations that you've added into values-openshift.yaml should also be in values.yaml. Helm will always use the values.yaml as the default configuration for the app (docs ref).

For example the following install command will use all default values from the values.yaml in the chart:

helm install kubecost cost-analyzer \
--repo https://kubecost.github.io/cost-analyzer/ \
--namespace kubecost --create-namespace

[thomasvn]

@mittal-ishaan and I discussed this PR on a call. We made the following changes together:

• The values.yaml should be the source of truth, and should capture all possible configurations. It should also contain detailed documentation on what each of the configurations do.
• The values-openshift.yaml is an example of the minimal set of configurations needed to deploy Kubecost on OpenShift assuming you are also deploying Kubecost's bundled Prometheus
• The values-openshift-cluster-prometheus.yaml is an example of the minimal set of configurations needed to deploy Kubecost on OpenShift assuming you are leveraging a Prometheus instance which already exists in the cluster.

[thomasvn]

@mittal-ishaan @jessegoodier I think we should cherry-pick this into the v2.5 branch so that it gets released with v2.5.0? Thoughts?

[jessegoodier]

100%

[mittal-ishaan]

/cherry-pick v2.5