Backport 16907 to 8.17: Use --qualifier in release manager (#16907)

[dliappis]

(cherry picked from commit 9385cfa)

Automated backport had conflicts in .buildkite/scripts/dra/build_docker.sh where before main we also build ubi8.

---

Release notes

[rn:skip]

What does this PR do?

This commit uses the new --qualifier parameter in the release manager for publishing dra artifacts. Additionally, simplifies the expected variables to rely on a simple VERSION_QUALIFIER.

Finally, we skip snapshot builds when VERSION_QUALIFIER is set.

Why is it important/What is the impact to the user?

Enables prerelease staging builds

How to test this PR

To test via this PR supply the following BK options to the staging pipeline:

DRA_VERSION="main"
VERSION_QUALIFIER="alpha1"

which results in a successful build -> https://buildkite.com/elastic/logstash-dra-staging-pipeline/builds/177

Once it's merged, manual staging builds need to be triggered using only VERSION_QUALIFIER; docs will be updated for https://docs.elastic.dev/ingest-dev-docs/logstash/dra#usage-of-version_qualifier_opt via a follow up PR.

Also tested a snapshot build to ensure nothing's broken (this just requires DRA_BRANCH="main" as params), it's successful -> https://buildkite.com/elastic/logstash-dra-snapshot-pipeline/builds/2052

Related issues

Closes https://github.com/elastic/ingest-dev/issues/4856

Screenshots

[dliappis]

Note for reviewers

This is identical to #16913, but targeted to 8.17 instead.

Automation couldn't be used due to conflicts in the build_docker script (we still build ubi8 in this branch).

Comparing this PR against 8.x

$ TARGET_BRANCH="8.x"; for fname in .buildkite/dra_pipeline.yml .buildkite/scripts/dra/build_docker.sh .buildkite/scripts/dra/build_packages.sh .buildkite/scripts/dra/generatesteps.py .buildkite/scripts/dra/publish.sh; do echo ">>> Comparing $fname against $TARGET_BRANCH"; git diff $TARGET_BRANCH -- $fname ; done
>>> Comparing .buildkite/dra_pipeline.yml against 8.x
>>> Comparing .buildkite/scripts/dra/build_docker.sh against 8.x
>>> Comparing .buildkite/scripts/dra/build_packages.sh against 8.x
>>> Comparing .buildkite/scripts/dra/generatesteps.py against 8.x
>>> Comparing .buildkite/scripts/dra/publish.sh against 8.x

Comparing this PR against main

$ TARGET_BRANCH="main"; for fname in .buildkite/dra_pipeline.yml .buildkite/scripts/dra/build_docker.sh .buildkite/scripts/dra/build_packages.sh .buildkite/scripts/dra/generatesteps.py .buildkite/scripts/dra/publish.sh; do echo ">>> Comparing $fname against $TARGET_BRANCH"; git diff $TARGET_BRANCH -- $fname ; done
>>> Comparing .buildkite/dra_pipeline.yml against main
>>> Comparing .buildkite/scripts/dra/build_docker.sh against main
diff --git a/.buildkite/scripts/dra/build_docker.sh b/.buildkite/scripts/dra/build_docker.sh
index 938922782..891a70ce2 100755
--- a/.buildkite/scripts/dra/build_docker.sh
+++ b/.buildkite/scripts/dra/build_docker.sh
@@ -26,6 +26,10 @@ rake artifact:docker_oss || error "artifact:docker_oss build failed."
 rake artifact:docker_wolfi || error "artifact:docker_wolfi build failed."
 rake artifact:dockerfiles || error "artifact:dockerfiles build failed."
 
+if [[ "$ARCH" != "aarch64" ]]; then
+  rake artifact:docker_ubi8 || error "artifact:docker_ubi8 build failed."
+fi
+
 if [[ "$WORKFLOW_TYPE" == "staging" ]] && [[ -n "$VERSION_QUALIFIER" ]]; then
     # Qualifier is passed from CI as optional field and specify the version postfix
     # in case of alpha or beta releases for staging builds only:
@@ -48,6 +52,10 @@ for file in build/logstash-*; do shasum $file;done
 info "Uploading DRA artifacts in buildkite's artifact store ..."
 # Note the deb, rpm tar.gz AARCH64 files generated has already been loaded by the build_packages.sh
 images="logstash logstash-oss logstash-wolfi"
+if [ "$ARCH" != "aarch64" ]; then
+    # No logstash-ubi8 for AARCH64
+    images="logstash logstash-oss logstash-wolfi logstash-ubi8"
+fi
 for image in ${images}; do
     buildkite-agent artifact upload "build/$image-${STACK_VERSION}-docker-image-${ARCH}.tar.gz"
 done
@@ -55,7 +63,7 @@ done
 # Upload 'docker-build-context.tar.gz' files only when build x86_64, otherwise they will be
 # overwritten when building aarch64 (or viceversa).
 if [ "$ARCH" != "aarch64" ]; then
-    for image in logstash logstash-oss logstash-wolfi logstash-ironbank; do
+    for image in logstash logstash-oss logstash-wolfi logstash-ubi8 logstash-ironbank; do
         buildkite-agent artifact upload "build/${image}-${STACK_VERSION}-docker-build-context.tar.gz"
     done
 fi
>>> Comparing .buildkite/scripts/dra/build_packages.sh against main
>>> Comparing .buildkite/scripts/dra/generatesteps.py against main
>>> Comparing .buildkite/scripts/dra/publish.sh against main
diff --git a/.buildkite/scripts/dra/publish.sh b/.buildkite/scripts/dra/publish.sh
index 09ef1a2ac..c56fbaedf 100755
--- a/.buildkite/scripts/dra/publish.sh
+++ b/.buildkite/scripts/dra/publish.sh
@@ -42,6 +42,13 @@ if [ "$RELEASE_VER" != "7.17" ]; then
   :
 fi
 
+# Deleting ubi8 for aarch64 for the time being. This image itself is not being built, and it is not expected
+# by the release manager.
+# See https://github.com/elastic/infra/blob/master/cd/release/release-manager/project-configs/8.5/logstash.gradle
+# for more details.
+# TODO filter it out when uploading artifacts instead
+rm -f build/logstash-ubi8-${STACK_VERSION}-docker-image-aarch64.tar.gz
+
 info "Downloaded ARTIFACTS sha report"
 for file in build/logstash-*; do shasum $file;done

[andsel]

LGTM once CI is green.

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 6cae961

cc @dliappis

[dliappis]

Successful snapshot build using this PR: https://buildkite.com/elastic/logstash-dra-snapshot-pipeline/builds/2077