Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run integration tests against OpenSearch 2.16. #409

Merged
merged 2 commits into from
Jul 11, 2024

Conversation

dblock
Copy link
Member

@dblock dblock commented Jul 11, 2024

Description

  1. Run integration tests against multiple versions of OpenSearch (latest and next).
  2. Actively wait for OpenSearch docker container to come up instead of a fixed 60s.
  3. Display OpenSearch version tests are being run against.
Screenshot 2024-07-11 at 9 41 46 AM

Issues Resolved

List any issues this PR will resolve, e.g. Closes [...].

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

github-actions bot commented Jul 11, 2024

Changes Analysis

Commit SHA: 8b26266
Comparing To SHA: ece6d96

API Changes

Summary

NO CHANGES

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/9896580766/artifacts/1692313234

API Coverage

Before After Δ
Covered (%) 483 (47.31 %) 483 (47.31 %) 0 (0 %)
Uncovered (%) 538 (52.69 %) 538 (52.69 %) 0 (0 %)
Unknown 24 24 0

@dblock dblock force-pushed the opensearch-2.16 branch 2 times, most recently from f7178b1 to cdcbab8 Compare July 11, 2024 13:45
@dblock dblock marked this pull request as ready for review July 11, 2024 13:46
@dblock dblock force-pushed the opensearch-2.16 branch 3 times, most recently from 26260ee to 5ba175e Compare July 11, 2024 14:50
matrix:
entry:
- {version: 2.15.0, hub: 'opensearchproject'}
- {version: 2.16.0, hub: 'opensearchstaging'}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great. My concern is whether we should test multiple versions of OS now when the test framework cant differentiate which features in the spec can be tested against which version of OS. Say if we add a new query param in 2.17 and update the spec accordingly, then all matrix entries before 2.17 will likely fail stories that test the affected endpoint.

Potential solution (in a different PR): Add min_cluster_version and max_cluster_version properties for test stories and the test framework will only test the story if the it's being tested against a valid cluster version.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm glad you asked, I got better in #410.

@dblock dblock force-pushed the opensearch-2.16 branch from 5ba175e to 8b26266 Compare July 11, 2024 18:23
@dblock dblock requested a review from nhtruong July 11, 2024 18:42
@nhtruong nhtruong merged commit f8788da into opensearch-project:main Jul 11, 2024
11 checks passed
@dblock dblock deleted the opensearch-2.16 branch July 11, 2024 19:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants