Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added ML Model APIs #733

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

nathaliellenaa
Copy link
Contributor

@nathaliellenaa nathaliellenaa commented Dec 13, 2024

Description

Added missing ML Model APIs.

Issues Resolved

Part of opensearch-project/opensearch-py#867.

ML Model APIs to add

  • GET /_plugins/_ml/models/{model_id}
  • POST /_plugins/_ml/models/meta
  • POST /_plugins/_ml/models/_register_meta
  • POST /_plugins/_ml/models/_search
  • POST /_plugins/_ml/models/_undeploy
  • POST /_plugins/_ml/models/_unload
  • POST /_plugins/_ml/models/{model_id}/_unload
  • POST /_plugins/_ml/models/_upload
  • POST /_plugins/_ml/models/{model_id}/_load
  • POST /_plugins/_ml/models/{model_id}/_predict
  • POST /_plugins/_ml/models/{model_id}/chunk/{chunk_number}
  • POST /_plugins/_ml/models/{model_id}/upload_chunk/{chunk_number}
  • POST /_plugins/_ml/models/{model_id}/{version}/_register
  • POST /_plugins/_ml/models/{model_id}/{version}/_upload
  • PUT /_plugins/_ml/models/{model_id}

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

github-actions bot commented Dec 13, 2024

Changes Analysis

Commit SHA: 33aab02
Comparing To SHA: a2afe56

API Changes

Summary

├─┬Paths
│ ├──[➕] path (4802:3)
│ ├──[➕] path (4968:3)
│ ├──[➕] path (4916:3)
│ ├──[➕] path (4818:3)
│ ├──[➕] path (4934:3)
│ ├──[➕] path (4755:3)
│ ├──[➕] path (5026:3)
│ ├──[➕] path (4988:3)
│ ├──[➕] path (5008:3)
│ ├──[➕] path (4837:3)
│ ├─┬/_plugins/_ml/models/{model_id}
│ │ ├──[➕] get (4872:7)
│ │ └──[➕] put (4886:7)
│ ├─┬/_plugins/_ml/models/_search
│ │ └──[➕] post (4788:7)
│ ├─┬/_plugins/_ml/models/_register
│ │ └─┬POST
│ │   └─┬Requestbody
│ │     └─┬application/json
│ │       └─┬Schema
│ │         ├──[➖] required (27143:17)❌ 
│ │         ├─┬model_format
│ │         │ └──[🔀] $ref (33660:13)❌ 
│ │         ├─┬model_group_id
│ │         │ └──[🔀] $ref (33450:13)❌ 
│ │         └─┬version
│ │           └──[🔀] $ref (34958:13)❌ 
│ └─┬/_plugins/_ml/models/{model_id}/_undeploy
│   └─┬POST
│     └──[➕] requestBody (27569:7)❌ 
└─┬Components
  ├──[➕] requestBodies (27585:7)
  ├──[➕] requestBodies (27642:7)
  ├──[➕] requestBodies (27569:7)
  ├──[➕] requestBodies (27517:7)
  ├──[➕] requestBodies (27424:7)
  ├──[➕] requestBodies (27601:7)
  ├──[➕] requestBodies (27387:7)
  ├──[➕] requestBodies (27344:7)
  ├──[➕] requestBodies (27631:7)
  ├──[➕] responses (30812:7)
  ├──[➕] responses (30725:7)
  ├──[➕] responses (30836:7)
  ├──[➕] responses (30772:7)
  ├──[➕] responses (30818:7)
  ├──[➕] responses (30751:7)
  ├──[➕] responses (30626:7)
  ├──[➕] responses (30824:7)
  ├──[➕] responses (30647:7)
  ├──[➕] responses (30713:7)
  ├──[➕] parameters (23147:7)
  ├──[➕] parameters (23123:7)
  ├──[➕] parameters (23057:7)
  ├──[➕] parameters (23154:7)
  ├──[➕] parameters (23141:7)
  ├──[➕] parameters (23050:7)
  ├──[➕] parameters (23135:7)
  ├──[➕] parameters (23105:7)
  ├──[➕] parameters (23117:7)
  ├──[➕] schemas (33660:7)
  ├──[➕] schemas (54271:7)
  ├──[➕] schemas (54422:7)
  ├──[➕] schemas (54436:7)
  ├──[➕] schemas (54231:7)
  ├──[➕] schemas (54429:7)
  ├──[➕] schemas (54677:7)
  ├──[➕] schemas (54680:7)
  ├──[➕] schemas (54161:7)
  ├──[➕] schemas (54685:7)
  ├──[➕] schemas (54200:7)
  ├──[➕] schemas (54672:7)
  ├──[➕] schemas (54386:7)
  └─┬ml._common___Source
    ├─┬model_format
    │ └──[🔀] $ref (33660:13)❌ 
    └─┬model_state
      └──[➕] enum (54552:15)

Document Element Total Changes Breaking Changes
paths 18 5
components 43 1
  • BREAKING Changes: 6 out of 61
  • Modifications: 4
  • Removals: 1
  • Additions: 56
  • Breaking Removals: 1
  • Breaking Modifications: 4
  • Breaking Additions: 1

Report

The full API changes report is available at: https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12437624737/artifacts/2350089724

API Coverage

Before After Δ
Covered (%) 606 (59.35 %) 619 (60.63 %) 13 (1.28 %)
Uncovered (%) 415 (40.65 %) 402 (39.37 %) -13 (-1.28 %)
Unknown 43 43 0

Copy link
Contributor

github-actions bot commented Dec 13, 2024

Spec Test Coverage Analysis

Total Tested
546 547 (100.18 %)

Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. I have some smallish nits. Iterate to green (validation is failing, etc.)

CHANGELOG.md Outdated Show resolved Hide resolved
tests/plugins/ml/ml/models.yaml Outdated Show resolved Hide resolved
tests/plugins/ml/ml/models.yaml Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
@dblock
Copy link
Member

dblock commented Dec 13, 2024

Btw, there's a whole bunch of others missing per #168. They don't all need to be done at the same time.

Screenshot 2024-12-13 at 5 07 47 PM

@nathaliellenaa
Copy link
Contributor Author

Yes, I'm planning to create some small PRs based on the APIs category (model, model groups, connector, agents, etc.) I will also add more missing Model APIs to this PR.

@nathaliellenaa nathaliellenaa changed the title Added missing ML Model APIs to the spec along with the tests Added ML Model APIs Dec 14, 2024
@dhrubo-os
Copy link

DCO is missing @nathaliellenaa

dblock and others added 10 commits December 18, 2024 12:12
…o models/search.yaml, resolved conflicts and updated CHANGELOG

Signed-off-by: Nathalie Jonathan <[email protected]>
Signed-off-by: Thomas Farr <[email protected]>
Signed-off-by: Nathalie Jonathan <[email protected]>
…project#732)

* Fixed /_search/scroll.

Signed-off-by: dblock <[email protected]>

* Added tests for GET and POST /_search.

Signed-off-by: dblock <[email protected]>

* Added a test for GET /_search/pipeline and DELETE /_search/pipeline/{id}.

Signed-off-by: dblock <[email protected]>

* Added missing _search/point_in_time tests.

Signed-off-by: dblock <[email protected]>

---------

Signed-off-by: dblock <[email protected]>
Signed-off-by: Nathalie Jonathan <[email protected]>
* Added tests for /_validate/query.

Signed-off-by: dblock <[email protected]>

* Added retry for opensearch-project#738.

Signed-off-by: dblock <[email protected]>

---------

Signed-off-by: dblock <[email protected]>
Signed-off-by: Nathalie Jonathan <[email protected]>
Signed-off-by: Nathalie Jonathan <[email protected]>
Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Some nits below.

In tests try to get rid of multiple-paths-detected: false as much as possible by moving setup into prologues and teardown into epilogues. You should only need that set if there's a need to call an unrelated API (e.g. wait on a task) to complete the test. Otherwise chapters should only have chapters for the API being tested.

CHANGELOG.md Outdated Show resolved Hide resolved
spec/namespaces/ml.yaml Outdated Show resolved Hide resolved
spec/namespaces/ml.yaml Outdated Show resolved Hide resolved
spec/namespaces/ml.yaml Outdated Show resolved Hide resolved
spec/namespaces/ml.yaml Outdated Show resolved Hide resolved
spec/namespaces/ml.yaml Outdated Show resolved Hide resolved
tests/plugins/ml/models/load.yaml Outdated Show resolved Hide resolved
tests/plugins/ml/models/load.yaml Outdated Show resolved Hide resolved
nathaliellenaa and others added 4 commits December 19, 2024 13:30
…own to epilogues in predict.yaml and load.yaml, updated CHANGELOG format, updated API description, 'model_group_id' ID type, 'version' parameter, and made 'model_format' a type of its own in ml.yaml.

Signed-off-by: Nathalie Jonathan <[email protected]>
…t for deprecated model metadata creation API.

Signed-off-by: Nathalie Jonathan <[email protected]>
…pload to create_metadata.yaml, updated CHANGELOG.

Signed-off-by: Nathalie Jonathan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants