Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[HOLD ON MERGE] Virtualization MU SLES OpenQA part changes for medium based triggering with new flavors #19864

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[HOLD ON MERGE] Virtualization MU SLES OpenQA part changes for medium based triggering with new flavors #19864

wants to merge 4 commits into from
+74 −38

Conversation

alice-suse
Copy link
Contributor

@alice-suse alice-suse commented Aug 2, 2024 •
edited
Loading

This PR mainly achieves https://progress.opensuse.org/issues/160868. Please refer to ticket's description for what are required.
Note: This shall be merged TOGETHER WITH https://progress.opensuse.org/issues/160865. Now need to wait for it.

Explanations for some changes:

  • bind all job groups to one unified job group, for reuse, more simplified contents, and wildcard match for products per flavor but different versions
  • 5 new flavors created for both TERADATA, ES, and regular products, and bind each of them with specific tests, see https://openqa.suse.de/admin/job_templates/426? and https://openqa.suse.de/admin/job_templates/426?.
  • move more product related settings from medium setting to test code, so that we can share the same medium among all products for the same flavor, example:
    image
  • reg code for ltss products are all set in job group yaml LTSS_REGCODES_SECRET (new products entering LTSS will need to add the setting into the var)
  • reg code and reg email for all non-ltss products is aligned to be SCC_EMAIL([email protected]) and SCC_REGCODE, which is set to be the same and put in medium (it shall be fine for future products too, because each new SLES product will enable the same SCC_REGCODE during development phase by QA)
  • exclude 12sp3 xen tests completely
  • for 12sp3/12sp5, on those flavors which have test qam-[kvm|xen]-install-and-features-test
    , remove qam-[kvm|xen]-install-and-sriov-test tests, because sriov is not supported on sle12, so the test mainly does host and guest installation/patches which is already covered in the feature test (still keeps for kernel flavors)
  • create a new machine type 64bit-ipmi-12sp5 to run 12sp5 xen tests (because not all machines can run the test successfully on host installation part, the selected machines run well from historical jobs), and bind it to 2 pairs of machines -- worker class live_migrat and live_migrat_5
  • to distribute migration tests to multiple pairs of machines for more balanced tests on SUT, assign live_migrat_4(worker class) in tests
  • the command to trigger tests in schedule_issue.py, after merging the PR, will be like this(an example): openqa-cli api -X POST isos ARCH="x86_64" DISTRI="sle" VERSION="12-SP3" FLAVOR=Server-DVD-Incidents-VIRT-Kernel-TERADATA BUILD=":888888:kernel-debug" INCIDENT_REPO=xxxx, nothing else. But for personal triggering purpose, of course, you can specify more settings to customize.
  • fix guest_management test failure in 15sp4 non-TD , see failure example http://openqa.qam.suse.cz/tests/78699

Needles:

https://gitlab.suse.de/openqa/os-autoinst-needles-sles/-/merge_requests/1671

Verification run:

On openqa.qam.suse.cz,

On openqa.suse.de (vmware & hyperv tests),

@github-actions GitHub Actions
Copy link

github-actions bot commented Aug 2, 2024

Great PR! Please pay attention to the following items before merging:

Files matching lib/**.pm:

  • Consider adding or extending unit tests in t/

This is an automatically generated QA checklist based on modified files.

@alice-suse alice-suse changed the title [WIP] Alice/adopt ci tools a5 [HOLD ON MERGE] Virtualization MU SLES OpenQA part changes for medium based triggering with new flavors Aug 9, 2024
@alice-suse
Copy link
Contributor Author

@RoyCai7 @tbaev @martinsmarcelo @Julie-CAO Welcome to review!

@alice-suse alice-suse changed the title [HOLD ON MERGE] Virtualization MU SLES OpenQA part changes for medium based triggering with new flavors [WIP] Virtualization MU SLES OpenQA part changes for medium based triggering with new flavors Aug 9, 2024
@alice-suse alice-suse changed the title [WIP] Virtualization MU SLES OpenQA part changes for medium based triggering with new flavors [HOLD ON MERGE] Virtualization MU SLES OpenQA part changes for medium based triggering with new flavors Aug 9, 2024
@alice-suse alice-suse force-pushed the alice/adopt-ci-tools-A5 branch 2 times, most recently from 65b8e9e to 35066e6 Compare August 9, 2024 12:26
@RoyCai7
Copy link
Contributor

RoyCai7 commented Sep 4, 2024 •
edited
Loading

Regarding the test suites on http://openqa.qam.suse.cz/group_overview/174, if the test suite is either qam-kvm-install-and-features-test or qam-xen-install-and-features-test, and it's not for 12sp5, it would be advisable to create a new machine specifically for these tests. name this machine 64bit-ipmi-vft(virtualization feature test).
it is convenient for me to schedule job. HDYT?

if using 64bit-ipmi, it is not reasonable, because there are a lot of workers using such machine name.

@alice-suse
Copy link
Contributor Author

alice-suse commented Sep 4, 2024 •
edited
Loading

Regarding the test suites on http://openqa.qam.suse.cz/group_overview/174, if the test suite is either qam-kvm-install-and-features-test or qam-xen-install-and-features-test, and it's not for 12sp5, it would be advisable to create a new machine specifically for these tests. name this machine 64bit-ipmi-vft(virtualization feature test). it is convenient for me to schedule job. HDYT?

This job group is for development test at the moment. However it is going to be official group after PRs and reviews done and agreed.

The group is where defines tests for all products (including 12sp5) and all mediums, and all non-vmware/hyperv testsuites.

if using 64bit-ipmi, it is not reasonable, because there are a lot of workers using such machine name.

About on which machine(class) to run the tests, the applied principle is that if there is no known problem to run specific test on any single machine by bugs, the tests are good to bind to machine type as general as possible, so that they have chance to test on any machine. This is widely used principle.

If you worry about the interrupt to official test because it uses potentially any machine to test, actually the devel tests were done only on my reserved machine there. To test if it is fine to make it final binded to the type of machine in official job group, I had to do this. And I think it did not interrupt any scheduled tests, because openqa can tell which worker is busy running jobs and only schedule once job finishes.

I am not sure if I get your point fully correctly. If needed, we can have offline talk to discuss.

@RoyCai7
Copy link
Contributor

RoyCai7 commented Sep 4, 2024

Regarding the test suites on http://openqa.qam.suse.cz/group_overview/174, if the test suite is either qam-kvm-install-and-features-test or qam-xen-install-and-features-test, and it's not for 12sp5, it would be advisable to create a new machine specifically for these tests. name this machine 64bit-ipmi-vft(virtualization feature test). it is convenient for me to schedule job. HDYT?

This job group is for development test at the moment. However it is going to be official group after PRs and reviews done and agreed.

The group is where defines tests for all products (including 12sp5) and all mediums, and all non-vmware/hyperv testsuites.

if using 64bit-ipmi, it is not reasonable, because there are a lot of workers using such machine name.

About on which machine(class) to run the tests, the applied principle is that if there is no known problem to run specific test on any single machine by bugs, the tests are good to bind to machine type as general as possible, so that they have chance to test on any machine. This is widely used principle.

If you worry about the interrupt to official test because it uses potentially any machine to test, actually the devel tests were done only on my reserved machine there. To test if it is fine to make it final binded to the type of machine in official job group, I had to do this. And I think it did not interrupt any scheduled tests, because openqa can tell which worker is busy running jobs and only schedule once job finishes.

I am not sure if I get your point fully correctly. If needed, we can have offline talk to discuss.

if hit issue, can make change. I will use 64bit-ipmi temporarily.

@alice-suse
Copy link
Contributor Author

Regarding the test suites on http://openqa.qam.suse.cz/group_overview/174, if the test suite is either qam-kvm-install-and-features-test or qam-xen-install-and-features-test, and it's not for 12sp5, it would be advisable to create a new machine specifically for these tests. name this machine 64bit-ipmi-vft(virtualization feature test). it is convenient for me to schedule job. HDYT?

This job group is for development test at the moment. However it is going to be official group after PRs and reviews done and agreed.
The group is where defines tests for all products (including 12sp5) and all mediums, and all non-vmware/hyperv testsuites.

if using 64bit-ipmi, it is not reasonable, because there are a lot of workers using such machine name.

About on which machine(class) to run the tests, the applied principle is that if there is no known problem to run specific test on any single machine by bugs, the tests are good to bind to machine type as general as possible, so that they have chance to test on any machine. This is widely used principle.
If you worry about the interrupt to official test because it uses potentially any machine to test, actually the devel tests were done only on my reserved machine there. To test if it is fine to make it final binded to the type of machine in official job group, I had to do this. And I think it did not interrupt any scheduled tests, because openqa can tell which worker is busy running jobs and only schedule once job finishes.
I am not sure if I get your point fully correctly. If needed, we can have offline talk to discuss.

if hit issue, can make change. I will use 64bit-ipmi temporarily.

Sorry, I kind of lost for your situation. You use 64bit-ipmi temporarily for what? If you mean schedule_issue.py change for A5, you do not need to specify machine info at all, in triggering cmd, just as ticket description gives. If you mean official incident triggering, you can continue with before.

@alice-suse alice-suse force-pushed the alice/adopt-ci-tools-A5 branch 9 times, most recently from c749da3 to a4687ea Compare September 10, 2024 01:18
@alice-suse
Copy link
Contributor Author

alice-suse commented Sep 10, 2024 •
edited
Loading

Latest validation jobs are updated into description. Ready for review.

@alice-suse
Copy link
Contributor Author

I rebased code to latest master and extend code to support 12SP5 LTSS-ES, updated mediums and job group settings, and newly triggered validation runs:

openqa.qam.suse.cz:

OSD:

@alice-suse
Copy link
Contributor Author

I rebased code to latest master and extend code to support 12SP5 LTSS-ES, updated mediums and job group settings, and newly triggered validation runs:

openqa.qam.suse.cz:

OSD:

All these verification jobs finished and checked. All expected.

@RoyCai7 @tbaev @martinsmarcelo Welcome to do final review! Now it supports new flavor for 12sp5 LTSS ES.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@waynechen55 waynechen55 waynechen55 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@alice-suse @RoyCai7 @waynechen55