Add settings to configure reaper HTTP management interface

[Miles-Garnsey]

What this PR does:

Adds settings to use the Reaper HTTP management API to k8ssandra-operator.

Which issue(s) this PR fixes:
#1072

Checklist

• Changes manually tested
• Automated Tests added/updated
• Documentation added/updated
• CHANGELOG.md updated (not required for documentation PRs)
• CLA Signed: DataStax CLA

[github-actions]

No linked issues found. Please add the corresponding issues in the pull request description.
Use GitHub automation to close the issue when a PR is merged

[codecov]

Codecov Report

Merging #1082 (a05fed1) into main (08a62b3) will increase coverage by 0.07%.
Report is 4 commits behind head on main.
The diff coverage is 100.00%.

❗ Current head a05fed1 differs from pull request most recent head 785fc1b. Consider uploading reports for the commit 785fc1b to get more accurate results

@@            Coverage Diff             @@
##             main    #1082      +/-   ##
==========================================
+ Coverage   57.28%   57.35%   +0.07%     
==========================================
  Files         100      100              
  Lines       10165    10175      +10     
==========================================
+ Hits         5823     5836      +13     
+ Misses       3832     3830       -2     
+ Partials      510      509       -1     

Files	Coverage Δ
controllers/reaper/reaper_controller.go	52.68% <100.00%> (+0.31%)	⬆️
pkg/reaper/deployment.go	93.77% <100.00%> (+0.16%)	⬆️

... and 1 file with indirect coverage changes

[Miles-Garnsey]

Results of manual testing:

Using the following manifest:

apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraCluster
metadata:
  name: test
  namespace: k8ssandra-operator
spec:
  reaper:
    containerImage:
      name: cassandra-reaper
      registry: australia-southeast1-docker.pkg.dev
      repository: k8ssandra/miles
      tag: http-management
    httpManagement:
      enabled: true
  cassandra:
    datacenters:
      - metadata:
          name: dc1
        size: 1
        serverVersion: 4.0.6
        storageConfig:
          cassandraDataVolumeClaimSpec:
            storageClassName: standard
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 5Gi

And the following sequence of steps:

gcloud --project k8ssandra container clusters create  \
	miles-test \
	--zone australia-southeast1-a \
	--num-nodes=2 \
	--max-nodes=6 \
	--enable-autoscaling \
	--machine-type=e2-standard-4
  make docker-build && \
  docker tag k8ssandra/k8ssandra-operator:latest australia-southeast1-docker.pkg.dev/k8ssandra/miles/k8ssandra-operator:latest
  docker push australia-southeast1-docker.pkg.dev/k8ssandra/miles/k8ssandra-operator:latest

docker push australia-southeast1-docker.pkg.dev/k8ssandra/miles/cassandra-reaper:http-management && \
  kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.1/cert-manager.yaml && \
  until kustomize build .miles/build/ | kubectl apply --server-side --force-conflicts -f -; do sleep 10; done

I get a DC up with both Reaper and STS-1 pods.

I then go to the Reaper logs and see

io.cassandrareaper.ReaperException: no host could be reached through HTTP
        at io.cassandrareaper.management.http.HttpManagementConnectionFactory.connectAny(HttpManagementConnectionFactory.java:101)
        at io.cassandrareaper.management.ClusterFacade.connectImpl(ClusterFacade.java:896)
        at io.cassandrareaper.management.ClusterFacade.connect(ClusterFacade.java:878)
        at io.cassandrareaper.management.ClusterFacade.getClusterName(ClusterFacade.java:219)
        at io.cassandrareaper.resources.ClusterResource.findClusterWithSeedHost(ClusterResource.java:376)
        at io.cassandrareaper.resources.ClusterResource.addOrUpdateCluster(ClusterResource.java:300)
        at io.cassandrareaper.resources.ClusterResource.addOrUpdateCluster(ClusterResource.java:251)

So this is positive, because it means that the HTTP management proxy is indeed being used. However, it is negative because for some reason, the seeds are not being properly discovered:

INFO   [2023-10-12 08:09:49,205] [dw-35 - PUT /cluster/test?seedHost=test-seed-service] i.c.r.ClusterResource - PUT addOrUpdateCluster called with: cluster_name = test, seedHost = test-seed-service

This appears to be some sort of DNS issue if I'm not mistaken. I think what I'm seeing is probably that we don't have a docker image built yet that has the new management API stuff in it. Maybe I will merge the integration branch there so we can get our hands on one.

[sonarqubecloud]

SonarCloud Quality Gate failed.   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
4 Code Smells

No Coverage information
5.1% Duplication

Catch issues before they fail your Quality Gate with our IDE extension SonarLint