RHOAIENG-17695: chore(ci): create a test for calling `oc version` in the test, which can be run with ci testing #829

jiridanek · 2025-01-08T16:43:50Z

https://issues.redhat.com/browse/RHOAIENG-17695

Description

Pytest! Yay!

How Has This Been Tested?

poetry run pytest tests/containers \
  --image ghcr.io/jiridanek/notebooks/workbench-images:runtime-minimal-ubi9-python-3.9-main_fe3641a01ac9e1e8f694f1a8bce3b34cacf9ae57 \
  --image quay.io/modh/odh-minimal-notebook-container@sha256:eec50e5518176d5a31da739596a7ddae032d73851f9107846a587442ebd10a82 \
  --image=ghcr.io/jiridanek/notebooks/workbench-images:base-ubi9-python-3.11-jd_try_oc_command_cleanup_b1b8d1f10a329317d85f0ea54580e170b002cae8

This should fail one test, because one of the images is the one that was seen failing on FIPS systems.

Merge criteria:

The commits are squashed in a cohesive manner and have meaningful messages.
Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
The developer has manually tested the changes and verified that the changes work

…the test, which can be run with ci testing

jiridanek

Design issues

suite should support local run
github run
and on openshift run

Suite should handle reporting of failing tests

junit, or markdown table, something that's clear at a glanc

Makefile tests should be rewritten into this

the source of truth should be the manifest files
need to figure out which manifest applies for which image

jiridanek · 2025-01-08T18:15:40Z

.github/workflows/build-notebooks-TEMPLATE.yaml

+        env:
+          DOCKER_HOST: "unix:///var/run/podman/podman.sock"
+          TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE: "/var/run/podman/podman.sock"
+          RYUK_CONTAINER_IMAGE: "testcontainers/ryuk:0.8.1"


@jstourac suggested disabling ryuk for gha runs

@jstourac suggested disabling ryuk for gha runs #846

jiridanek · 2025-01-08T18:16:47Z

tests/containers/base_image_test.py

+    """Tests that are applicable for all images we have in this repository."""
+
+    def test_oc_command_runs(self, image: str):
+        container = testcontainers.core.container.DockerContainer(image=image, user=123456, group_add=[0])


@jstourac suggested abstracting this object away, so the same test can run against local docker as well as remote kubernetes, depending on some cli switch or some other configuration mechanism

@jstourac suggested abstracting this object away, so the same test can run against local docker as well as remote kubernetes, depending on some cli switch or some other configuration mechanism #847

jiridanek · 2025-01-08T18:17:23Z

tests/containers/base_image_test.py

+        logging.debug(output.decode())
+        assert ecode == 0
+
+    def test_oc_command_runs_fake_fips(self, image: str, subtests: pytest_subtests.SubTests):


@jstourac suggested skipping this test when running suite on openshift, openshift will test real configurations, not these fake ones

@jstourac suggested skipping this test when running suite on openshift, openshift will test real configurations, not these fake ones #848

jiridanek · 2025-01-08T18:18:02Z

tests/containers/conftest.py

+        metafunc.parametrize(image.__name__, metafunc.config.getoption("--image"))
+
+
+# https://docs.pytest.org/en/stable/how-to/fixtures.html#parametrizing-fixtures


@jstourac was discouraged by this long doc page and left the office for today shortly after opening it

jiridanek · 2025-01-08T18:18:49Z

tests/containers/base_image_test.py

+    import pytest_subtests
+
+
+class TestBaseImage:


@jstourac wondered how to structure the tests when more are to be added, like when the version tests are rewritten from makefile to here

andyatmiami · 2025-01-20T20:25:17Z

/lgtm

Reviewed code and was able to run locally to observe intended behavior.

@jstourac 's insights all certainly good/valid - but current form of this PR seems like a great starting point to enhance in the future.

jiridanek · 2025-01-21T08:43:25Z

/override ci/prow/images
/approve

openshift-ci · 2025-01-21T08:43:33Z

@jiridanek: Overrode contexts on behalf of jiridanek: ci/prow/images

In response to this:

/override ci/prow/images
/approve

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci · 2025-01-21T08:43:34Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jiridanek

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jiridanek]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

RHOAIENG-17695: chore(ci): create a test for calling oc version in …

b7c937e

…the test, which can be run with ci testing

openshift-ci bot requested review from andyatmiami and caponetto January 8, 2025 16:43

jiridanek commented Jan 8, 2025

View reviewed changes

This was referenced Jan 9, 2025

[WIP] RHOAIENG-9707 ci: dynamic testing of container images with pytest #629

Open

RHOAIENG-17305: chore(tests): IPv4 compatibility test(s), checking that we did not break the single-stack IPv4 case #835

Merged

openshift-ci bot assigned andyatmiami Jan 20, 2025

openshift-ci bot added the lgtm label Jan 20, 2025

openshift-ci bot added the approved label Jan 21, 2025

openshift-merge-bot bot merged commit ff4ee73 into opendatahub-io:main Jan 21, 2025
7 checks passed

jiridanek deleted the jd_try_oc_command_cleanup branch January 21, 2025 08:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RHOAIENG-17695: chore(ci): create a test for calling `oc version` in the test, which can be run with ci testing #829

RHOAIENG-17695: chore(ci): create a test for calling `oc version` in the test, which can be run with ci testing #829

jiridanek commented Jan 8, 2025

jiridanek left a comment

jiridanek Jan 8, 2025

jiridanek Jan 21, 2025

jiridanek Jan 8, 2025

jiridanek Jan 21, 2025

jiridanek Jan 21, 2025 •

edited

Loading

jiridanek Jan 8, 2025

jiridanek Jan 21, 2025

jiridanek Jan 8, 2025

jiridanek Jan 8, 2025

andyatmiami commented Jan 20, 2025

jiridanek commented Jan 21, 2025

openshift-ci bot commented Jan 21, 2025

openshift-ci bot commented Jan 21, 2025

		metafunc.parametrize(image.__name__, metafunc.config.getoption("--image"))


		# https://docs.pytest.org/en/stable/how-to/fixtures.html#parametrizing-fixtures

RHOAIENG-17695: chore(ci): create a test for calling oc version in the test, which can be run with ci testing #829

RHOAIENG-17695: chore(ci): create a test for calling oc version in the test, which can be run with ci testing #829

Conversation

jiridanek commented Jan 8, 2025

Description

How Has This Been Tested?

Merge criteria:

jiridanek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jiridanek Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andyatmiami commented Jan 20, 2025

jiridanek commented Jan 21, 2025

openshift-ci bot commented Jan 21, 2025

openshift-ci bot commented Jan 21, 2025

RHOAIENG-17695: chore(ci): create a test for calling `oc version` in the test, which can be run with ci testing #829

RHOAIENG-17695: chore(ci): create a test for calling `oc version` in the test, which can be run with ci testing #829

jiridanek Jan 21, 2025 •

edited

Loading