Simplify the Galasa contribution + build process : 17-> 10 code repositories

[techcobweb]

Story

As a Galasa code developer I want a simpler build process, so it's easier to contribute changes.

Background

Currently we have many repositories which have separate build processes. These

1. cli
2. githubapp-copyright
3. isolated
4. obr
5. extensions
6. gradle
7. managers
8. webui
9. automation
10. framework
11. helm
12. maven
13. wrapping
14. buildutils
15. galasa.dev
16. simplatform
17. homebrew-tap

Lets change things to have fewer, like this:

1. githubapp-copyright
2. automation
3. buildutils, wrapping, gradle, maven, framework, extensions, managers, obr
4. cli
5. webui
6. simplatform
7. isolated
8. helm
9. galasa.dev
10. homebrew-tap

+ves: Easier to commit atomic changes across framework/extensions/managers/obr
-ves: Slightly longer build time, as there is more code. We could speed things up with a build cache.

Question: Can VScode cope with this ? You can always open vscode one folder down in the source tree.

Tasks

• Analyse the existing github workflows to see how they can be combined into one, or called from one, with information flowing between them which does not need docker images to be stood-up on infrastructure.

• Create a top-level prototype of a github workflow which builds the gradle and maven modules. Demonstrating how information is shared between them.

• Create a PR to the mono-repo-creator containing this top-level gh actions wizardry.

  • Master repo calls the sub-flows

• When a PR build occurs it shouldn't have to build everything if it didn't touch it.

• Test with different changes in different modules, to check that the logic builds the correct things

• Release processed needs changing + release documentation.

[jlolivos]

Tutorial to build Galasa on local:

1.- Disable GPG:
pass flag: -Dgpg.skip=true

galasa/modules/wrapping/.github/workflows/pr-build.yaml:78: -Dgpg.skip=true
galasa/modules/wrapping/build-locally.sh:182:#cmd="mvn clean install -Dgpg.passphrase=${gpg_passphrase} -Dgpg.skip=true"
galasa/modules/wrapping/build-locally.sh:183:cmd="mvn clean install -Dgpg.skip=true"
galasa/modules/maven/.github/workflows/pr-build.yaml:72: mvn -f galasa-maven-plugin/pom.xml -X deploy -Dgpg.skip=true
galasa/modules/obr/.github/workflows/pr-build.yaml:101: -Dgpg.skip=true
galasa/modules/obr/.github/workflows/pr-build.yaml:128: -Dgpg.skip=true
galasa/modules/obr/.github/workflows/pr-build.yaml:236: -Dgpg.skip=true
galasa/modules/obr/.github/workflows/pr-build.yaml:319: -Dgpg.skip=true
galasa/modules/obr/build-locally.sh:396: -Dgpg.skip=true \

2.- Disable check_secrets in build-locally.sh scripts
Example_:
sed -zi 's/check_secrets/#check_secrets/2' ./galasa/modules/wrapping/build-locally.sh

3.- Create a GPG key on your local
https://docs.github.com/en/authentication/managing-commit-signature-verification/generating-a-new-gpg-key

Export the GPG key to: /home/jenkins/.gnupg/secring.gpg

4.- Add next values to your gradle.properties file:
Ensure this variable is false when it is not a release.
isMainOrRelease=false

signingPassword=xxxxx
signingKeyId=2***********D
signingKey=/home/jenkins/.gnupg/secring.gpg

5.- Update script: galasa/modules/obr/build-locally.sh
In my case was trying to run galasabld-darwin, instead of linux version.
Update accordingly to your architecture.

175 #export GALASA_BUILD_TOOL_NAME=galasabld-darwin-${ARCHITECTURE}
176 export GALASA_BUILD_TOOL_NAME=galasabld-linux-${ARCHITECTURE}

6.- Run script: ./galasa/tools/build-locally.sh

[INFO] Installing /home/jenkins/repo/galasa/galasa/modules/cli/temp/dev.galasa.example.banking/dev.galasa.example.banking.obr/target/testcatalog.json to /home/jenkins/.m2/repository/dev/galasa/example/banking/dev.galasa.example.banking.obr/0.0.1-SNAPSHOT/dev.galasa.example.banking.obr-0.0.1-SNAPSHOT-testcatalog.json
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for dev.galasa.example.banking 0.0.1-SNAPSHOT:
[INFO]
[INFO] dev.galasa.example.banking ......................... SUCCESS [ 0.364 s]
[INFO] dev.galasa.example.banking.account ................. SUCCESS [ 3.327 s]
[INFO] dev.galasa.example.banking.payee ................... SUCCESS [ 0.817 s]
[INFO] dev.galasa.example.banking.obr ..................... SUCCESS [ 0.350 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.450 s
[INFO] Finished at: 2024-09-03T09:11:02-07:00
[INFO] ------------------------------------------------------------------------
✔ OK

Use the results..
➜ Binary executable programs are found in the 'bin' folder.
galasactl-darwin-arm64
galasactl-darwin-x86_64
galasactl-linux-arm64
galasactl-linux-s390x
galasactl-linux-x86_64
galasactl-windows-x86_64.exe
[✔] Built module cli OK

[jlolivos]

Working on this task:
[ ] Analyse the existing github workflows to see how they can be combined into one, or called from one, with information flowing between them which does not need docker images to be stood-up on infrastructure.

[jadecarino]

Release process/automation updates in this PR: galasa-dev/automation#625

[jadecarino]

Updates to mono repo creator tool in this PR: galasa-dev/mono-repo-creator#1

[jadecarino]

Updates to the Isolated/MVP build workflows to support the new mono repo structure in this PR: galasa-dev/isolated#73

[jadecarino]

Updates to the CLI build workflows to support the new mono repo changes in this PR: galasa-dev/cli#299

[techcobweb]

This works. Marking as done.