breaking: use a ltsc2019 base image for windowsservercore-ltsc2019 agent images

[lemeurherve]

This PR initially added nanoserver-ltsc2019 images by refactoring the build and the docker compose file to resolve #451.

Doing so I've identified #475, which made me focus in this PR to fix this issue by using mcr.microsoft.com/windows/servercore:ltsc2019 image instead of mcr.microsoft.com/powershell:windowsservercore-1809 for windowsservercore-ltsc2019 images as there is not mcr.microsoft.com/windows/servercore:ltsc2019 tag according to https://mcr.microsoft.com/v2/powershell/tags/list. (Same for nanoserver images).

This PR also adds a "proper" set windowsservercore-1809 images which should be a transparent switch from the previous windowsservercore-ltsc2019 one.

In addition to fixing issue #475, the goal of this build process refactoring is to facilitate the introduction of additional nanoserver-ltsc2019 images (will be done in a following PR).

To be mentioned in the release note:

For those wanting to stay as close as possible to the previous windowsservercore-ltsc2019 image, they will be able to use the new windowsservercore-1809 docker agent image.
Otherwise they can switch to the new "proper" windowsservercore-ltsc2019 image.

Closes #475

Testing done

https://ci.jenkins.io/job/Packaging/job/docker-agent/job/PR-459/63/

Submitter checklist

Beta Give feedback

1. Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
2. Ensure that the pull request title represents the desired changelog entry
3. Please describe what you did
4. Link to relevant issues in GitHub or Jira
5. Link to relevant pull requests, esp. upstream and downstream changes
6. Ensure you have provided tests - that demonstrates feature works or fixes the issue

[lemeurherve]

Last logs from https://ci.jenkins.io/blue/rest/organizations/jenkins/pipelines/Packaging/pipelines/docker-agent/branches/PR-459/runs/15/nodes/67/steps/71/log/?start=0:

output

[2023-07-24T13:57:31.221Z] = PREPARE: List of images and tags to be processed:
[2023-07-24T13:57:31.221Z] {
[2023-07-24T13:57:31.221Z]     "jdk11-windowsservercore-ltsc2019":  {
[2023-07-24T13:57:31.221Z]                                              "Tags":  [
[2023-07-24T13:57:31.221Z]                                                           "jdk11-windowsservercore-ltsc2019",
[2023-07-24T13:57:31.221Z]                                                           "3131.vf2b_b_798b_ce99-1-jdk11-windowsservercore-ltsc2019",
[2023-07-24T13:57:31.221Z]                                                           "windowsservercore-ltsc2019"
[2023-07-24T13:57:31.221Z]                                                       ]
[2023-07-24T13:57:31.221Z]                                          },
[2023-07-24T13:57:31.221Z]     "jdk17-nanoserver-1809":  {
[2023-07-24T13:57:31.222Z]                                   "Tags":  [
[2023-07-24T13:57:31.222Z]                                                "jdk17-nanoserver-1809",
[2023-07-24T13:57:31.222Z]                                                "3131.vf2b_b_798b_ce99-1-jdk17-nanoserver-1809",
[2023-07-24T13:57:31.222Z]                                                "jdk17-nanoserver-ltsc2019",
[2023-07-24T13:57:31.222Z]                                                "3131.vf2b_b_798b_ce99-1-jdk17-nanoserver-ltsc2019"
[2023-07-24T13:57:31.222Z]                                            ]
[2023-07-24T13:57:31.222Z]                               },
[2023-07-24T13:57:31.222Z]     "jdk17-windowsservercore-ltsc2019":  {
[2023-07-24T13:57:31.222Z]                                              "Tags":  [
[2023-07-24T13:57:31.222Z]                                                           "jdk17-windowsservercore-ltsc2019",
[2023-07-24T13:57:31.222Z]                                                           "3131.vf2b_b_798b_ce99-1-jdk17-windowsservercore-ltsc2019"
[2023-07-24T13:57:31.222Z]                                                       ]
[2023-07-24T13:57:31.222Z]                                          },
[2023-07-24T13:57:31.222Z]     "jdk11-nanoserver-1809":  {
[2023-07-24T13:57:31.223Z]                                   "Tags":  [
[2023-07-24T13:57:31.223Z]                                                "jdk11-nanoserver-1809",
[2023-07-24T13:57:31.223Z]                                                "3131.vf2b_b_798b_ce99-1-jdk11-nanoserver-1809",
[2023-07-24T13:57:31.223Z]                                                "nanoserver-1809",
[2023-07-24T13:57:31.223Z]                                                "jdk11-nanoserver-ltsc2019",
[2023-07-24T13:57:31.223Z]                                                "3131.vf2b_b_798b_ce99-1-jdk11-nanoserver-ltsc2019",
[2023-07-24T13:57:31.223Z]                                                "nanoserver-ltsc2019"
[2023-07-24T13:57:31.223Z]                                            ]
[2023-07-24T13:57:31.223Z]                               }
[2023-07-24T13:57:31.223Z] }

[lemeurherve]

I initially though the nanoserver-1809 and nanoserver-ltsc2019 were the same, they aren't (the ltsc2019 is ensured to get updates)

Fixed in 580ef8b

new output:

[2023-07-28T15:59:26.539Z] = PREPARE: List of images and tags to be processed:
[2023-07-28T15:59:27.106Z] {
[2023-07-28T15:59:27.107Z]     "jdk17-nanoserver-ltsc2019":  {
[2023-07-28T15:59:27.107Z]                                       "Tags":  [
[2023-07-28T15:59:27.107Z]                                                    "jdk17-nanoserver-ltsc2019",
[2023-07-28T15:59:27.107Z]                                                    "3131.vf2b_b_798b_ce99-1-jdk17-nanoserver-ltsc2019"
[2023-07-28T15:59:27.107Z]                                                ]
[2023-07-28T15:59:27.107Z]                                   },
[2023-07-28T15:59:27.107Z]     "jdk11-nanoserver-ltsc2019":  {
[2023-07-28T15:59:27.107Z]                                       "Tags":  [
[2023-07-28T15:59:27.107Z]                                                    "jdk11-nanoserver-ltsc2019",
[2023-07-28T15:59:27.107Z]                                                    "3131.vf2b_b_798b_ce99-1-jdk11-nanoserver-ltsc2019",
[2023-07-28T15:59:27.107Z]                                                    "nanoserver-ltsc2019"
[2023-07-28T15:59:27.107Z]                                                ]
[2023-07-28T15:59:27.107Z]                                   },
[2023-07-28T15:59:27.107Z]     "jdk17-nanoserver-1809":  {
[2023-07-28T15:59:27.107Z]                                   "Tags":  [
[2023-07-28T15:59:27.107Z]                                                "jdk17-nanoserver-1809",
[2023-07-28T15:59:27.107Z]                                                "3131.vf2b_b_798b_ce99-1-jdk17-nanoserver-1809"
[2023-07-28T15:59:27.108Z]                                            ]
[2023-07-28T15:59:27.108Z]                               },
[2023-07-28T15:59:27.108Z]     "jdk11-windowsservercore-ltsc2019":  {
[2023-07-28T15:59:27.108Z]                                              "Tags":  [
[2023-07-28T15:59:27.108Z]                                                           "jdk11-windowsservercore-ltsc2019",
[2023-07-28T15:59:27.108Z]                                                           "3131.vf2b_b_798b_ce99-1-jdk11-windowsservercore-ltsc2019",
[2023-07-28T15:59:27.108Z]                                                           "windowsservercore-ltsc2019"
[2023-07-28T15:59:27.108Z]                                                       ]
[2023-07-28T15:59:27.108Z]                                          },
[2023-07-28T15:59:27.108Z]     "jdk11-nanoserver-1809":  {
[2023-07-28T15:59:27.108Z]                                   "Tags":  [
[2023-07-28T15:59:27.108Z]                                                "jdk11-nanoserver-1809",
[2023-07-28T15:59:27.108Z]                                                "3131.vf2b_b_798b_ce99-1-jdk11-nanoserver-1809",
[2023-07-28T15:59:27.108Z]                                                "nanoserver-1809"
[2023-07-28T15:59:27.108Z]                                            ]
[2023-07-28T15:59:27.108Z]                               },
[2023-07-28T15:59:27.108Z]     "jdk17-windowsservercore-ltsc2019":  {
[2023-07-28T15:59:27.108Z]                                              "Tags":  [
[2023-07-28T15:59:27.108Z]                                                           "jdk17-windowsservercore-ltsc2019",
[2023-07-28T15:59:27.108Z]                                                           "3131.vf2b_b_798b_ce99-1-jdk17-windowsservercore-ltsc2019"
[2023-07-28T15:59:27.108Z]                                                       ]
[2023-07-28T15:59:27.108Z]                                          }
[2023-07-28T15:59:27.109Z] }

[lemeurherve]

Unfortunately the command docker-compose --file=build-windows.yaml build --parallel --pull called at

docker-agent/build.ps1

Lines 118 to 120 in b49cad5

	Write-Host "= BUILD: Building all images..."
	Invoke-Expression $baseDockerBuildCmd
	Write-Host "= BUILD: Finished building all image"

for building all images at once is problematic, it doesn't take in account the $builds array of images and their tags and thus doesn't have any idea of the additional nanoserver-ltsc2019 images 😓

To fix that without a bigger refactoring would mean iterate on every $builds items to call the build like

docker-agent/build.ps1

Lines 113 to 116 in b49cad5

	Write-Host "= BUILD: Building image ${Build}..."
	$dockerBuildCmd = '{0} {1}' -f $baseDockerBuildCmd, $Build
	Invoke-Expression $dockerBuildCmd
	Write-Host "= BUILD: Finished building image ${Build}"

[dduportal]

Unfortunately the command docker-compose --file=build-windows.yaml build --parallel --pull called at

docker-agent/build.ps1

Lines 118 to 120 in b49cad5

	Write-Host "= BUILD: Building all images..."
	Invoke-Expression $baseDockerBuildCmd
	Write-Host "= BUILD: Finished building all image"

for building all images at once is problematic, it doesn't take in account the $builds array of images and their tags and thus doesn't have any idea of the additional nanoserver-ltsc2019 images 😓

To fix that without a bigger refactoring would mean iterate on every $builds items to call the build like

docker-agent/build.ps1

Lines 113 to 116 in b49cad5

	Write-Host "= BUILD: Building image ${Build}..."
	$dockerBuildCmd = '{0} {1}' -f $baseDockerBuildCmd, $Build
	Invoke-Expression $dockerBuildCmd
	Write-Host "= BUILD: Finished building image ${Build}"

Could we run LTSC2019 and 1809 in parallel in two 2019 agents?

[lemeurherve]

Unfortunately the command docker-compose --file=build-windows.yaml build --parallel --pull called at

docker-agent/build.ps1

Lines 118 to 120 in b49cad5

	Write-Host "= BUILD: Building all images..."
	Invoke-Expression $baseDockerBuildCmd
	Write-Host "= BUILD: Finished building all image"

for building all images at once is problematic, it doesn't take in account the $builds array of images and their tags and thus doesn't have any idea of the additional nanoserver-ltsc2019 images 😓
To fix that without a bigger refactoring would mean iterate on every $builds items to call the build like

docker-agent/build.ps1

Lines 113 to 116 in b49cad5

	Write-Host "= BUILD: Building image ${Build}..."
	$dockerBuildCmd = '{0} {1}' -f $baseDockerBuildCmd, $Build
	Invoke-Expression $dockerBuildCmd
	Write-Host "= BUILD: Finished building image ${Build}"

Could we run LTSC2019 and 1809 in parallel in two 2019 agents?

This should work but then we'll need to add a windows-1809 label on (trusted.)ci.jenkins.io agents so I can use the agent label to generate the corresponding images and tags.

EDIT: will set agent label dynamically instead.

[dduportal]

EDIT: will set agent label dynamically instead.

Yup that was the (not really explicit :D) idea: `if endsWith('1809') then set to 2019)

[lemeurherve]

FTR, some updatecli manifests are now failing, will look into it later.