Merge branch 'main' into constant_keyword

Signed-off-by: kkewwei <[email protected]>

.github/workflows/test-spec.yml

@@ -6,13 +6,15 @@ on:
       - .github/workflows/test-spec.yml
       - package*.json
       - spec/**
+      - tests/**
       - tools/src/tester/**
       - tsconfig.json
   pull_request:
     paths:
       - .github/workflows/test-spec.yml
       - package*.json
       - spec/**
+      - tests/**
       - tools/src/tester/**
       - tsconfig.json
 
@@ -21,6 +23,7 @@ jobs:
     strategy:
       matrix:
         entry:
+          - {version: 1.3.17, admin_password: admin}
           - {version: 2.0.0, admin_password: admin}
           - {version: 2.15.0}
           - {version: 2.16.0, hub: opensearchstaging}

CHANGELOG.md

@@ -26,7 +26,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - Added tests for response payload ([#347](https://github.com/opensearch-project/opensearch-api-specification/pull/347))
 - Added `cancel_after_time_interval` and `phase_took` in `_search` ([#353](https://github.com/opensearch-project/opensearch-api-specification/pull/353))
 - Added support for testing `application/x-ndjson` payloads ([#355](https://github.com/opensearch-project/opensearch-api-specification/pull/355))
-- Added SPECIFICATION_TESTING.md [#359](https://github.com/opensearch-project/opensearch-api-specification/pull/359)
+- Added TESTING_GUIDE.md [#359](https://github.com/opensearch-project/opensearch-api-specification/pull/359)
 - Added StoryValidator to validate stories before running them ([#354](https://github.com/opensearch-project/opensearch-api-specification/issues/354))
 - Added support for `text/plain` responses in `_cat` APIs ([#360](https://github.com/opensearch-project/opensearch-api-specification/pull/360))
 - Added support for `application/yaml` responses ([#363](https://github.com/opensearch-project/opensearch-api-specification/pull/363))
@@ -42,6 +42,10 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - Added `file` to `/_cache/clear` and `/{index}/_cache/clear` ([#396](https://github.com/opensearch-project/opensearch-api-specification/pull/396))
 - Added a workflow to run tests against the next version of OpenSearch ([#409](https://github.com/opensearch-project/opensearch-api-specification/pull/409))
 - Added support for skipping tests using semver range ([#410](https://github.com/opensearch-project/opensearch-api-specification/pull/410))
+- Added `cluster_manager_timeout` to `HEAD /{index}` ([#421](https://github.com/opensearch-project/opensearch-api-specification/pull/421))
+- Added missing fields to `/_nodes/stats` ([#415](https://github.com/opensearch-project/opensearch-api-specification/pull/415))
+- Added missing metrics options to `/_nodes/stats` ([#422](https://github.com/opensearch-project/opensearch-api-specification/pull/422))
+- Added tests against OpenSearch 1.3 ([#424](https://github.com/opensearch-project/opensearch-api-specification/pull/424))
 
 ### Changed
 
@@ -77,8 +81,10 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 - Fixed `/_data_stream` health status and required fields ([#401](https://github.com/opensearch-project/opensearch-api-specification/pull/401))
 - Fixed query DSL `match` that supports a field name and value ([#405](https://github.com/opensearch-project/opensearch-api-specification/pull/405)) 
 - Fixed `/_mapping` with `index` in query ([#385](https://github.com/opensearch-project/opensearch-api-specification/pull/385))
+- Fixed duplicate `/_nodes/{node_id}` path ([#416](https://github.com/opensearch-project/opensearch-api-specification/pull/416))
 - Fixed set `value` required in `constant_keyword` field type ([#419](https://github.com/opensearch-project/opensearch-api-specification/pull/419))
 
+
 ### Security
 
 [Unreleased]: https://github.com/opensearch-project/opensearch-api-specification/commits/main/

CLIENT_GENERATOR_GUIDE.md

@@ -1,3 +1,12 @@
+- [Generate Clients for OpenSearch using OpenAPI Specification](#generate-clients-for-opensearch-using-openapi-specification)
+  - [The Grouping of API Operations](#the-grouping-of-api-operations)
+  - [Overloaded Name](#overloaded-name)
+  - [Handling Bulk Operations](#handling-bulk-operations)
+  - [Parameter Validation](#parameter-validation)
+  - [Global Parameters](#global-parameters)
+  - [Default Parameter Values](#default-parameter-values)
+  - [Duplicate Paths](#duplicate-paths)
+
 # Generate Clients for OpenSearch using OpenAPI Specification
 
 OpenSearch Clients are available in multiple programming languages. The biggest challenge with this is keeping the clients up to date with the latest changes in OpenSearch. To solve this problem, we're automating the process of generating clients for OpenSearch using the OpenAPI specification. While OpenAPI comes with many well established off-the-shelf client generators for most languages, the OpenSearch APIs come with a lot of quirkiness that makes it near impossible to use these off-the-shelf generators. For this reason, we've opted to write our own client generators that is specifically tailored to the OpenSearch APIs. This document will walk you through the process of generating clients from [the published OpenSearch OpenAPI spec](https://github.com/opensearch-project/opensearch-api-specification/releases), more specifically client API methods.
@@ -58,4 +67,23 @@ Some clients also check for the validity of query string parameter names to guar
 All operations in the spec contain a set of parameters that are common across all operations. These parameters are denoted with `x-global: true` vendor extension. The generated clients should find a way to DRY these parameters in type definitions and method documentation.
 
 ## Default Parameter Values
-Parameters can have default values either through schema or the `x-default` vendor extension. When both are present, `x-default` will take precedence.
+Parameters can have default values either through schema or the `x-default` vendor extension. When both are present, `x-default` will take precedence.
+
+## Duplicate Paths
+OpenAPI does not allow duplicate paths, which makes it challenging to express certain OpenSearch APIs, such as `/_nodes/{node_id}` and `/_nodes/{metric}`. In this example the server expects either a `node_id` or a `metric`. The API specification handles this by defining a `/_nodes/{node_id_or_metric}` path, and decorating the `node_id_or_metric` type with a `title` field.
+
+```yaml
+name: node_id_or_metric
+schema:
+anyOf:
+  - title: node_id
+    $ref: '../schemas/_common.yaml#/components/schemas/NodeIds'
+  - title: metric
+    type: array
+    items:
+      $ref: '../schemas/nodes.info.yaml#/components/schemas/Metric'
+```
+
+See [#416](https://github.com/opensearch-project/opensearch-api-specification/pull/416) for a complete example.
+
+Clients should generate multiple methods or a method that accepts multiple parameters in this case.

DEVELOPER_GUIDE.md

@@ -149,7 +149,7 @@ This repository includes several OpenAPI Specification Extensions to fill in any
 
 ## Writing Spec Tests
 
-To assure the correctness of the spec, you must add tests for the spec, when making changes. Check [SPECIFICATION_TESTING.md](SPECIFICATION_TESTING.md) for more information.
+To assure the correctness of the spec, you must add tests for the spec, when making changes. Check [TESTING_GUIDE.md](TESTING_GUIDE.md) for more information.
 
 ## Tools
 

README.md

@@ -12,25 +12,25 @@
 - [Welcome!](#welcome)
 - [Project Resources](#project-resources)
 - [Code of Conduct](#code-of-conduct)
-- [Developer Guide](#developer-guide)
-- [Client Generator Guide](#client-generator-guide)
-- [Published Spec](#published-spec)
 - [Security](#security)
 - [License](#license)
 - [Copyright](#copyright)
 
 ## Welcome!
 
-`opensearch-api-specification` is an open source, community-driven collection of API model specifications for
-[OpenSearch](https://github.com/opensearch-project/OpenSearch) APIs. The API models are written in OpenAPI format and are used to generate client libraries and documentation.
+The `opensearch-api-specification` is an open source, community-driven collection of API model specifications for [OpenSearch](https://github.com/opensearch-project/OpenSearch) APIs. The API models are written in OpenAPI format and are used to generate client libraries and documentation. You can find the latest version of the API specification [here](https://github.com/opensearch-project/opensearch-api-specification/releases/download/main-latest/opensearch-openapi.yaml).
 
-To contribute to this project or to track the developments head over to [Projects](https://github.com/opensearch-project/opensearch-api-specification/projects)
-board. Follow the [Developer guide](DEVELOPER_GUIDE.md) and [Contributing guidelines](CONTRIBUTING.md) for instructions
+To contribute to this project or to track the developments head over to [Projects](https://github.com/opensearch-project/opensearch-api-specification/projects) board. Follow the [Developer guide](DEVELOPER_GUIDE.md) and [Contributing guidelines](CONTRIBUTING.md) for instructions
 on building and contributing to opensearch-api-specification.
 
 ## Project Resources
 
+* [Current Release](https://github.com/opensearch-project/opensearch-api-specification/releases/download/main-latest/opensearch-openapi.yaml)
+* [Developer Guide](DEVELOPER_GUIDE.md).
+* [Client Generator Guide](CLIENT_GENERATOR_GUIDE.md).
+* [Spec Publishing Guide](PUBLISHING_GUIDE.md).
 * [Project Website](https://opensearch.org/)
+* [API Playground](https://opensearch-project.github.io/opensearch-api-specification/)
 * [Downloads](https://opensearch.org/downloads.html)
 * [Documentation](https://opensearch.org/docs/)
 * Need help? Try [Forums](https://forum.opensearch.org/)
@@ -41,21 +41,6 @@ on building and contributing to opensearch-api-specification.
 
 This project has adopted the [Amazon Open Source Code of Conduct](CODE_OF_CONDUCT.md). For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq), or contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
 
-## Developer Guide
-
-See [DEVELOPER_GUIDE](DEVELOPER_GUIDE.md).
-
-## Client Generator Guide
-
-See [CLIENT_GENERATOR_GUIDE](CLIENT_GENERATOR_GUIDE.md).
-
-## Published Spec
-
-OpenSearch API Specs are hosted at https://opensearch-project.github.io/opensearch-api-specification/. See [PUBLISHING_SPECS](PUBLISHING_SPECS.md) for more information.
-
-Click [here](https://github.com/opensearch-project/opensearch-api-specification/releases/download/main-latest/opensearch-openapi.yaml) to download the latest OpenSearch OpenAPI yaml file.
-
-
 ## Security
 
 If you discover a potential security issue in this project we ask that you notify OpenSearch Security directly via email to [email protected]. Please do **not** create a public GitHub issue.

SPECIFICATION_TESTING.md → TESTING_GUIDE.md

@@ -1,13 +1,13 @@
 <!-- TOC -->
-- [Specification Testing](#specification-testing)
+- [Spec Testing Guide](#spec-testing-guide)
   - [Running Spec Tests Locally](#running-spec-tests-locally)
   - [Writing Spec Tests](#writing-spec-tests)
     - [Simple Test Story](#simple-test-story)
     - [Using Output from Previous Chapters](#using-output-from-previous-chapters)
     - [Managing Versions](#managing-versions)
 <!-- TOC -->
 
-# Specification Testing
+# Spec Testing Guide
 
 We have devised our own test framework to test the spec against an OpenSearch cluster. We're still adding more features to the framework as the needs arise, and this document will be updated accordingly. This test framework has also been integrated into the repo's CI/CD pipeline. Checkout the [test-spec](.github/workflows/test-spec.yml) workflow for more details.
 

spec/namespaces/indices.yaml

@@ -894,6 +894,7 @@ paths:
       parameters:
         - $ref: '#/components/parameters/indices.exists::path.index'
         - $ref: '#/components/parameters/indices.exists::query.allow_no_indices'
+        - $ref: '#/components/parameters/indices.exists::query.cluster_manager_timeout'
         - $ref: '#/components/parameters/indices.exists::query.expand_wildcards'
         - $ref: '#/components/parameters/indices.exists::query.flat_settings'
         - $ref: '#/components/parameters/indices.exists::query.ignore_unavailable'
@@ -3220,6 +3221,13 @@ components:
       schema:
         $ref: '../schemas/_common.yaml#/components/schemas/Indices'
       style: simple
+    indices.exists::query.cluster_manager_timeout:
+      name: cluster_manager_timeout
+      in: query
+      description: Operation timeout for connection to cluster-manager node.
+      schema:
+        $ref: '../schemas/_common.yaml#/components/schemas/Duration'
+      x-version-added: '2.0'
     indices.exists::query.allow_no_indices:
       in: query
       name: allow_no_indices

spec/namespaces/nodes.yaml

@@ -253,7 +253,7 @@ paths:
       responses:
         '200':
           $ref: '#/components/responses/nodes.usage@200'
-  /_nodes/{metric}:
+  /_nodes/{node_id_or_metric}:
     get:
       operationId: nodes.info.1
       x-operation-group: nodes.info
@@ -262,22 +262,7 @@ paths:
       externalDocs:
         url: https://opensearch.org/docs/latest/api-reference/nodes-apis/nodes-info/
       parameters:
-        - $ref: '#/components/parameters/nodes.info::path.metric'
-        - $ref: '#/components/parameters/nodes.info::query.flat_settings'
-        - $ref: '#/components/parameters/nodes.info::query.timeout'
-      responses:
-        '200':
-          $ref: '#/components/responses/nodes.info@200'
-  /_nodes/{node_id}:
-    get:
-      operationId: nodes.info.2
-      x-operation-group: nodes.info
-      x-version-added: '1.0'
-      description: Returns information about nodes in the cluster.
-      externalDocs:
-        url: https://opensearch.org/docs/latest/api-reference/nodes-apis/nodes-info/
-      parameters:
-        - $ref: '#/components/parameters/nodes.info::path.node_id'
+        - $ref: '#/components/parameters/nodes.info::path.node_id_or_metric'
         - $ref: '#/components/parameters/nodes.info::query.flat_settings'
         - $ref: '#/components/parameters/nodes.info::query.timeout'
       responses:
@@ -545,15 +530,21 @@ components:
       description: The type to sample.
       schema:
         $ref: '../schemas/nodes._common.yaml#/components/schemas/SampleType'
-    nodes.info::path.metric:
+    nodes.info::path.node_id_or_metric:
       in: path
-      name: metric
-      description: Limits the information returned to the specific metrics. Supports a comma-separated list, such as http,ingest.
+      name: node_id_or_metric
+      description: |
+        Limits the information returned to a list of node IDs or specific metrics.
+        Supports a comma-separated list, such as node1,node2 or http,ingest.
       required: true
       schema:
-        type: array
-        items:
-          $ref: '../schemas/nodes.info.yaml#/components/schemas/Metric'
+        anyOf:
+          - title: node_id
+            $ref: '../schemas/_common.yaml#/components/schemas/NodeIds'
+          - title: metric
+            type: array
+            items:
+              $ref: '../schemas/nodes.info.yaml#/components/schemas/Metric'
       style: simple
     nodes.info::path.node_id:
       in: path
@@ -563,6 +554,16 @@ components:
       schema:
         $ref: '../schemas/_common.yaml#/components/schemas/NodeIds'
       style: simple
+    nodes.info::path.metric:
+      in: path
+      name: metric
+      description: Limits the information returned to the specific metrics. Supports a comma-separated list, such as http,ingest.
+      required: true
+      schema:
+        type: array
+        items:
+          $ref: '../schemas/nodes.info.yaml#/components/schemas/Metric'
+      style: simple
     nodes.info::query.flat_settings:
       in: query
       name: flat_settings

spec/schemas/_common.yaml

@@ -6,6 +6,9 @@ info:
 paths: {}
 components:
   schemas:
+    Type:
+      type: string
+      x-version-removed: '2.0'
     Id:
       type: string
     AcknowledgedResponseBase:
@@ -1315,6 +1318,8 @@ components:
     WriteResponseBase:
       type: object
       properties:
+        _type:
+          $ref: '#/components/schemas/Type'
         _id:
           $ref: '#/components/schemas/Id'
         _index:
@@ -1732,6 +1737,11 @@ components:
         query_total:
           description: The total number of shard query operations.
           type: number
+        request:
+          type: object
+          description: Statistics about coordinator search operations for the node.
+          additionalProperties:
+            $ref: '#/components/schemas/RequestStats'
         search_idle_reactivate_count_total:
           type: number
         scroll_current:
@@ -1800,6 +1810,15 @@ components:
         - size_in_bytes
         - uncommitted_operations
         - uncommitted_size_in_bytes
+    RequestStats:
+      type: object
+      properties:
+        time_in_millis:
+          $ref: '_common.yaml#/components/schemas/DurationValueUnitMillis'    
+        current:
+          type: number
+        total:
+          type: number
     WarmerStats:
       type: object
       properties:

spec/schemas/_core.bulk.yaml

@@ -110,6 +110,10 @@ components:
     ResponseItem:
       type: object
       properties:
+        _type:
+          description: The document type.
+          type: string
+          x-version-removed: '2.0'
         _id:
           description: The document ID associated with the operation.
           anyOf:

spec/schemas/_core.get.yaml

@@ -9,6 +9,8 @@ components:
     GetResult:
       type: object
       properties:
+        _type:
+          $ref: '_common.yaml#/components/schemas/Type'
         _index:
           $ref: '_common.yaml#/components/schemas/IndexName'
         fields:

spec/schemas/_core.search.yaml

@@ -43,6 +43,8 @@ components:
     Hit:
       type: object
       properties:
+        _type:
+          $ref: '_common.yaml#/components/schemas/Type'
         _index:
           $ref: '_common.yaml#/components/schemas/IndexName'
         _id:

spec/schemas/cat.health.yaml

@@ -52,3 +52,7 @@ components:
         discovered_cluster_manager:
           description: cluster manager is discovered or not
           type: string
+        discovered_master:
+          x-version-removed: '2.0'
+          description: cluster manager is discovered or not
+          type: string