Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing tests for ML APIs (search and delete) #672

Merged
merged 10 commits into from
Nov 16, 2024

Conversation

nathaliellenaa
Copy link
Contributor

Description

Added missing tests for GET /_plugins/_ml/models/_search and DELETE /_plugins/_ml/tasks/{task_id}.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

github-actions bot commented Nov 14, 2024

Changes Analysis

Commit SHA: c7569ac
Comparing To SHA: 398481e

API Changes

Summary

└─┬Components
  ├──[➕] schemas (52227:7)
  ├──[➕] schemas (52079:7)
  ├─┬ml._common___HitsTotal
  │ └─┬value
  │   └──[➕] format (52058:19)❌ 
  ├─┬ml._common___SearchModelsResponse
  │ ├──[➕] properties (52220:9)
  │ ├──[➕] properties (52217:9)
  │ └──[➕] properties (52214:9)
  ├─┬ml._common___SearchModelsHitsHit
  │ ├──[➕] required (52198:11)❌ 
  │ ├──[➖] required (52167:11)❌ 
  │ ├──[➖] required (52168:11)❌ 
  │ ├──[➕] properties (52182:9)
  │ ├──[➕] properties (52189:9)
  │ ├──[➕] properties (52180:9)
  │ ├──[➕] properties (52193:9)
  │ ├──[➕] properties (52178:9)
  │ ├─┬model_id
  │ │ └──[🔀] $ref (32155:13)❌ 
  │ ├─┬_id
  │ │ └──[🔀] $ref (31892:13)❌ 
  │ └─┬_index
  │   └──[🔀] $ref (31949:13)❌ 
  └─┬ml._common___SearchModelsHits
    └──[➕] properties (52168:9)

Document Element Total Changes Breaking Changes
components 18 7
  • BREAKING Changes: 7 out of 18
  • Modifications: 3
  • Removals: 2
  • Additions: 13
  • Breaking Removals: 2
  • Breaking Modifications: 3
  • Breaking Additions: 2

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/11865305204/artifacts/2196583379

API Coverage

Before After Δ
Covered (%) 588 (57.59 %) 588 (57.59 %) 0 (0 %)
Uncovered (%) 433 (42.41 %) 433 (42.41 %) 0 (0 %)
Unknown 42 42 0

Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, needs a bit of work, iterate to green.

  1. Fix validation/CI failures.
  2. Sign DCO, git commit --amend -s for existing commits.

spec/schemas/ml._common.yaml Outdated Show resolved Hide resolved
spec/schemas/ml._common.yaml Show resolved Hide resolved
tests/plugins/ml/ml/models.yaml Outdated Show resolved Hide resolved
Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The types for many of these things are too broad, I noted some below, but do look for existing fields with the same name for everything and see how they are declared with a ref.

Update CHANGELOG.

spec/schemas/ml._common.yaml Outdated Show resolved Hide resolved
spec/schemas/ml._common.yaml Outdated Show resolved Hide resolved
spec/schemas/ml._common.yaml Outdated Show resolved Hide resolved
spec/schemas/ml._common.yaml Show resolved Hide resolved
spec/schemas/ml._common.yaml Outdated Show resolved Hide resolved
spec/schemas/ml._common.yaml Outdated Show resolved Hide resolved
spec/schemas/ml._common.yaml Outdated Show resolved Hide resolved
spec/schemas/ml._common.yaml Show resolved Hide resolved
spec/schemas/ml._common.yaml Show resolved Hide resolved
spec/schemas/ml._common.yaml Show resolved Hide resolved
@dblock
Copy link
Member

dblock commented Nov 14, 2024

The test failure looks legit, probably needs a refresh or maybe something more.

ERROR   ml/models.yaml (tests/plugins/ml/ml/models.yaml)
    ERROR   CHAPTERS
        PASSED  Register model.
            PASSED  REQUEST BODY
            PASSED  RESPONSE STATUS
            PASSED  RESPONSE PAYLOAD BODY
            PASSED  RESPONSE PAYLOAD SCHEMA
            PASSED  RESPONSE OUTPUT VALUES
        ERROR   Search model.
            PASSED  REQUEST BODY
            PASSED  RESPONSE STATUS
            FAILED  RESPONSE PAYLOAD BODY (missing hits.hits[0]='[object Object]')
            PASSED  RESPONSE PAYLOAD SCHEMA
            ERROR   RESPONSE OUTPUT VALUES (Expected to find non undefined value at `payload.hits.hits[0]._id`.)
        SKIPPED Wait to get completed task.
        SKIPPED Delete model.

nathaliellenaa and others added 3 commits November 14, 2024 12:54
Signed-off-by: Nathalie Jonathan <[email protected]>
Signed-off-by: Nathalie Jonathan <[email protected]>
Signed-off-by: Daniel (dB.) Doubrovkine <[email protected]>
Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some tests are still failing.

spec/schemas/ml._common.yaml Show resolved Hide resolved
Signed-off-by: Nathalie Jonathan <[email protected]>
Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting there!

CHANGELOG.md Outdated Show resolved Hide resolved
tests/plugins/ml/ml/models.yaml Outdated Show resolved Hide resolved
tests/plugins/ml/ml/models.yaml Outdated Show resolved Hide resolved
@dblock
Copy link
Member

dblock commented Nov 15, 2024

            FAILED  RESPONSE PAYLOAD SCHEMA (data/hits/hits/27/_source/model_content_hash_value property is not defined in the spec, data/hits/hits/27/_source/last_registered_time property is not defined in the spec, data/hits/hits/27/_source/model_content_size_in_bytes property is not defined in the, ...)

Looks like more fields are missing, https://github.com/opensearch-project/ml-commons/blob/9c1b8a8c1e28ab8bdaf507cb71799c4809c3d8c6/plugin/src/test/resources/org/opensearch/ml/autoredeploy/ModelResult.json#L7.

Signed-off-by: Nathalie Jonathan <[email protected]>
Copy link
Contributor

Spec Test Coverage Analysis

Total Tested
517 351 (67.89 %)

Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing tests!

@dblock dblock merged commit bf35601 into opensearch-project:main Nov 16, 2024
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants