Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] Add Alma Linux 9 to matrix in packaging and platform jobs #118331

Merged
merged 5 commits into from
Dec 12, 2024
Merged

[ci] Add Alma Linux 9 to matrix in packaging and platform jobs #118331

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5462e00
[ci] Add debian-12 to matrix in packaging and platform jobs
jozala Dec 5, 2024
0e4ce20
Upgrade SmbTestContainer base image
jozala Dec 10, 2024
fef99c7
Improve waiting strategy for SMB test container
jozala Dec 10, 2024
6821d99
Spotless fix
jozala Dec 10, 2024
2035fb0
Longer timeout for starting container to avoid flakiness
jozala Dec 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .buildkite/pipelines/periodic-packaging.template.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ steps:
- rhel-8
- rhel-9
- almalinux-8
- almalinux-9
agents:
provider: gcp
image: family/elasticsearch-{{matrix.image}}
Expand Down
1 change: 1 addition & 0 deletions .buildkite/pipelines/periodic-packaging.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ steps:
- rhel-8
- rhel-9
- almalinux-8
- almalinux-9
agents:
provider: gcp
image: family/elasticsearch-{{matrix.image}}
Expand Down
1 change: 1 addition & 0 deletions .buildkite/pipelines/periodic-platform-support.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ steps:
- rhel-8
- rhel-9
- almalinux-8
- almalinux-9
agents:
provider: gcp
image: family/elasticsearch-{{matrix.image}}
Expand Down
1 change: 1 addition & 0 deletions .buildkite/pipelines/pull-request/packaging-tests-unix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ steps:
- rhel-8
- rhel-9
- almalinux-8
- almalinux-9
PACKAGING_TASK:
- docker
- docker-cloud-ess
Expand Down
2 changes: 2 additions & 0 deletions x-pack/test/smb-fixture/build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ apply plugin: 'elasticsearch.java'
apply plugin: 'elasticsearch.cache-test-fixtures'

dependencies {
implementation "com.fasterxml.jackson.core:jackson-annotations:${versions.jackson}"

api project(':test:fixtures:testcontainer-utils')
api "junit:junit:${versions.junit}"
api "org.testcontainers:testcontainers:${versions.testcontainer}"
Expand Down
28 changes: 22 additions & 6 deletions .../test/smb-fixture/src/main/java/org/elasticsearch/test/fixtures/smb/SmbTestContainer.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,28 +7,34 @@

package org.elasticsearch.test.fixtures.smb;

import com.github.dockerjava.api.model.Capability;

import org.elasticsearch.test.fixtures.testcontainers.DockerEnvironmentAwareTestContainer;
import org.testcontainers.containers.wait.strategy.Wait;
import org.testcontainers.containers.wait.strategy.WaitAllStrategy;
import org.testcontainers.images.builder.ImageFromDockerfile;

import java.time.Duration;

public final class SmbTestContainer extends DockerEnvironmentAwareTestContainer {

private static final String DOCKER_BASE_IMAGE = "ubuntu:16.04";
private static final String DOCKER_BASE_IMAGE = "ubuntu:24.04";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a problem with installation of Samba in Ubuntu 16.04 on specific Docker configuration. The installation hangs on some Python modules compilation.
Python modules compilation hangs because in RHEL9-based distros, during Docker build, ulimit set to unlimited. This is hard to change so instead I've decided to upgrade Ubuntu image to avoid this problem.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome!

public static final int AD_LDAP_PORT = 636;
public static final int AD_LDAP_GC_PORT = 3269;

public SmbTestContainer() {
super(
new ImageFromDockerfile("es-smb-fixture").withDockerfileFromBuilder(
builder -> builder.from(DOCKER_BASE_IMAGE)
.run("apt-get update -qqy && apt-get install -qqy samba ldap-utils")
.env("TZ", "Etc/UTC")
.run("DEBIAN_FRONTEND=noninteractive apt-get update -qqy && apt-get install -qqy tzdata winbind samba ldap-utils")
.copy("fixture/provision/installsmb.sh", "/fixture/provision/installsmb.sh")
.copy("fixture/certs/ca.key", "/fixture/certs/ca.key")
.copy("fixture/certs/ca.pem", "/fixture/certs/ca.pem")
.copy("fixture/certs/cert.pem", "/fixture/certs/cert.pem")
.copy("fixture/certs/key.pem", "/fixture/certs/key.pem")
.run("chmod +x /fixture/provision/installsmb.sh")
.run("/fixture/provision/installsmb.sh")
.cmd("service samba-ad-dc restart && sleep infinity")
.cmd("/fixture/provision/installsmb.sh && service samba-ad-dc restart && echo Samba started && sleep infinity")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Samba configuration script has been moved from building to starting the container, because it requires SYS_ADMIN capabilities to run in Ubuntu:24.04 image (capability has been added below). It's not possible to add the capability during the build phase.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's unfortunate

.build()
)
.withFileFromClasspath("fixture/provision/installsmb.sh", "/smb/provision/installsmb.sh")
Expand All @@ -37,10 +43,20 @@ public SmbTestContainer() {
.withFileFromClasspath("fixture/certs/cert.pem", "/smb/certs/cert.pem")
.withFileFromClasspath("fixture/certs/key.pem", "/smb/certs/key.pem")
);
// addExposedPort(389);
// addExposedPort(3268);

addExposedPort(AD_LDAP_PORT);
addExposedPort(AD_LDAP_GC_PORT);

setWaitStrategy(
new WaitAllStrategy().withStartupTimeout(Duration.ofSeconds(120))
.withStrategy(Wait.forLogMessage(".*Samba started.*", 1))
.withStrategy(Wait.forListeningPort())
);
Comment on lines +50 to +54
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait strategy needed now since the Samba configuration script has been moved from building the container to starting it.


getCreateContainerCmdModifiers().add(createContainerCmd -> {
createContainerCmd.getHostConfig().withCapAdd(Capability.SYS_ADMIN);
return createContainerCmd;
});
}

public String getAdLdapUrl() {
Expand Down
2 changes: 1 addition & 1 deletion x-pack/test/smb-fixture/src/main/resources/smb/provision/installsmb.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ cat $SSL_DIR/ca.pem >> /etc/ssl/certs/ca-certificates.crt

mv /etc/samba/smb.conf /etc/samba/smb.conf.orig

samba-tool domain provision --server-role=dc --use-rfc2307 --dns-backend=SAMBA_INTERNAL --realm=AD.TEST.ELASTICSEARCH.COM --domain=ADES --adminpass=Passw0rd --use-ntvfs
samba-tool domain provision --server-role=dc --use-rfc2307 --dns-backend=SAMBA_INTERNAL --realm=AD.TEST.ELASTICSEARCH.COM --domain=ADES --adminpass=Passw0rd
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ntvfs is no longer available in newer versions of Samba (info)


cp /var/lib/samba/private/krb5.conf /etc/krb5.conf

Expand Down
Loading