Merging to release-5.7: [TT-13761] add batch request to the latest op…

…en api specs (#6797) (#6807) ### **User description** [TT-13761] add batch request to the latest open api specs (#6797) <details open> <summary><a href="https://tyktech.atlassian.net/browse/TT-13761" title="TT-13761" target="_blank">TT-13761</a></summary> <br /> <table> <tr> <th>Summary</th> <td>add batch request to the latest open api specs</td> </tr> <tr> <th>Type</th> <td> <img alt="Story" src="https://tyktech.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10315?size=medium" /> Story </td> </tr> <tr> <th>Status</th> <td>In Code Review</td> </tr> <tr> <th>Points</th> <td>N/A</td> </tr> <tr> <th>Labels</th> <td>-</td> </tr> </table> </details>  --- When the new Gateway Open Api spec was created. The Batch request endpoint was left out .This should be added to the gateway OAS. This pr also fixes an issue where the external OAS Url we were using now return error 404. This pr changes that to use a local copy of the external oas . I.e we have changed from : https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/schemas/v3.0/schema.json to https://raw.githubusercontent.com/TykTechnologies/tyk/refs/heads/master/apidef/oas/schema/3.0.json Link: https://tyk.io/docs/5.5/tyk-gateway-api/ [TT-13761] [TT-13761]: https://tyktech.atlassian.net/browse/TT-13761?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: itachi sasuke <[email protected]> [TT-13761]: https://tyktech.atlassian.net/browse/TT-13761?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [TT-13761]: https://tyktech.atlassian.net/browse/TT-13761?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ ___ ### **PR Type** Enhancement, Bug fix ___ ### **Description** - Added support for batch requests in the OpenAPI specification. This includes a new endpoint `/{listen_path}/tyk/batch` and detailed documentation on its usage, request structure, and response format. - Introduced new schemas (`BatchRequestStructure`, `BatchReplyUnit`, and `RequestDefinition`) to support batch request functionality. - Updated references to the OpenAPI schema URL to use a local copy instead of an external URL, addressing a 404 error issue. ___ ### **Changes walkthrough** 📝 <table><thead><tr><th></th><th align="left">Relevant files</th></tr></thead><tbody><tr><td><strong>Bug fix</strong></td><td><table> <tr> <td> <details> <summary><strong>.redocly.lint-ignore.yaml</strong><dd><code>Update OpenAPI schema URL to local reference</code>                          </dd></summary> <hr> .redocly.lint-ignore.yaml <li>Updated the reference to the OpenAPI schema URL to use a local copy <br>instead of an external URL.<br> </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6807/files#diff-e4fbedb98d4b719f6be66946eec4e940438c123f2303c842b449c528ade31579">+1/-1</a>      </td> </tr> </table></td></tr><tr><td><strong>Enhancement</strong></td><td><table> <tr> <td> <details> <summary><strong>swagger.yml</strong><dd><code>Add batch request support and update OpenAPI schema references</code></dd></summary> <hr> swagger.yml <li>Added a new <code>Batch requests</code> section with detailed descriptions and <br>examples.<br> <li> Introduced a new endpoint <code>/{listen_path}/tyk/batch</code> for batch requests.<br> <li> Defined new schemas for <code>BatchRequestStructure</code>, <code>BatchReplyUnit</code>, and <br><code>RequestDefinition</code>.<br> <li> Updated references to the OpenAPI schema URL to use a local copy.<br> </details> </td> <td><a href="https://github.com/TykTechnologies/tyk/pull/6807/files#diff-8f3c4cb253eee09ae2401daa7279a8bbfbfd4168bb579c3ac0ee5c672d63bb2c">+295/-6</a>  </td> </tr> </table></td></tr></tr></tbody></table> ___ > 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull request to receive relevant information Co-authored-by: itachi sasuke <[email protected]>
TykTechnologies · Dec 22, 2024 · 04631ba · 04631ba
1 parent 23676f9
commit 04631ba
Show file tree

Hide file tree

Showing 2 changed files with 296 additions and 7 deletions.
diff --git a/.redocly.lint-ignore.yaml b/.redocly.lint-ignore.yaml
@@ -16,7 +16,7 @@ swagger.yml:
       #/paths/~1tyk~1apis~1oas~1{apiID}/patch/requestBody/content/application~1json/schema
     - >-
       #/paths/~1tyk~1apis~1oas~1import/post/requestBody/content/application~1json/schema
-https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/schemas/v3.0/schema.json:
+https://raw.githubusercontent.com/TykTechnologies/tyk/refs/heads/master/apidef/oas/schema/3.0.json:
   spec:
     - '#/id'
     - '#/$schema'

diff --git a/swagger.yml b/swagger.yml
@@ -85,6 +85,187 @@ tags:
 - description: |
     Manage OAuth clients, and manage their tokens
   name: OAuth
+- description: |
+    Tyk supports batch requests, so a client makes a single request to the API but gets a compound response object back.
+    
+    This is especially handy if clients have complex requests that have multiple synchronous dependencies and do not wish to have the entire request / response cycle running for each event.
+    
+    To enable batch request support, set the `enable_batch_request_support` value to `true`
+    
+    Batch requests that come into Tyk are *run through the whole Tyk machinery* and *use a relative path to prevent spamming*. This means that a batch request to Tyk for three resources with the same API key will have three requests applied to their session quota and request limiting could become active if they are being throttled.
+    
+    Tyk reconstructs the API request based on the data in the batch request. This is to ensure that Tyk is not being used to proxy requests to other hosts outside of the upstream API being accessed.
+    
+    Batch requests are created by POSTING to the `/{listen_path}/tyk/batch/` endpoint. These requests **do not require a valid key**, but their request list does.
+    
+    <h3>Sample Request</h3>
+    
+        ```{json}
+        {
+          "requests": [
+            {
+              "method": "GET",
+              "headers": {
+                "x-tyk-test": "1",
+                "x-tyk-version": "1.2",
+                "authorization": "1dbc83b9c431649d7698faa9797e2900f"
+              },
+              "body": "",
+              "relative_url": "get"
+          },
+          {
+            "method": "GET",
+            "headers": {
+              "x-tyk-test": "2",
+              "x-tyk-version": "1.2",
+              "authorization": "1dbc83b9c431649d7698faa9797e2900f"
+            },
+            "body": "",
+            "relative_url": "get"
+            }
+          ],
+          "suppress_parallel_execution": false
+        }
+        ```
+    
+    The response will be a structured reply that encapsulates the responses for each of the outbound requests. If `suppress_parallel_execution` is set to `true`, requests will be made synchronously. If set to `false` then they will run in parallel and the response order is not guaranteed.
+    
+    <h3>Sample Response</h3>
+    
+      ```
+      [
+        {
+          "relative_url": "get",
+          "code": 200,
+          "headers": {
+            "Access-Control-Allow-Credentials": [
+              "true"
+            ],
+            "Access-Control-Allow-Origin": [
+              "*"
+            ],
+            "Content-Length": [
+              "497"
+            ],
+            "Content-Type": [
+              "application/json"
+            ],
+            "Date": [
+              "Wed, 12 Nov 2014 15:32:43 GMT"
+            ],
+            "Server": [
+              "gunicorn/18.0"
+            ],
+            "Via": [
+              "1.1 vegur"
+            ]
+          },
+          "body": "{
+        "args": {},
+        "headers": {
+          "Accept-Encoding": "gzip",
+          "Authorization": "1dbc83b9c431649d7698faa9797e2900f",
+          "Connect-Time": "2",
+          "Connection": "close",
+          "Host": "httpbin.org",
+          "Total-Route-Time": "0",
+          "User-Agent": "Go 1.1 package http",
+          "Via": "1.1 vegur",
+          "X-Request-Id": "6a22499a-2776-4aa1-80c0-686581a8be4d",
+          "X-Tyk-Test": "2",
+          "X-Tyk-Version": "1.2"
+        },
+        "origin": "127.0.0.1, 62.232.114.250",
+        "url": "http://httpbin.org/get"
+      }"
+          },
+          {
+            "relative_url": "get",
+            "code": 200,
+            "headers": {
+              "Access-Control-Allow-Credentials": [
+                "true"
+              ],
+              "Access-Control-Allow-Origin": [
+                "*"
+              ],
+              "Content-Length": [
+                "497"
+              ],
+              "Content-Type": [
+                "application/json"
+              ],
+              "Date": [
+                "Wed, 12 Nov 2014 15:32:43 GMT"
+              ],
+              "Server": [
+                "gunicorn/18.0"
+              ],
+              "Via": [
+                "1.1 vegur"
+              ]
+            },
+            "body": "{
+        "args": {},
+        "headers": {
+          "Accept-Encoding": "gzip",
+          "Authorization": "1dbc83b9c431649d7698faa9797e2900f",
+          "Connect-Time": "7",
+          "Connection": "close",
+          "Host": "httpbin.org",
+          "Total-Route-Time": "0",
+          "User-Agent": "Go 1.1 package http",
+          "Via": "1.1 vegur",
+          "X-Request-Id": "1ab61f50-51ff-4828-a7e2-17240385a6d2",
+          "X-Tyk-Test": "1",
+          "X-Tyk-Version": "1.2"
+        },
+        "origin": "127.0.0.1, 62.232.114.250",
+        "url": "http://httpbin.org/get"
+      }"
+          }
+      ]
+      ```
+    With the body for each request string encoded in the `body` field.
+    
+    * `expire_analytics_after`: If you are running a busy API, you may want to ensure that your MongoDB database does not overflow with old data. Set the `expire_analytics_after` value to the number of seconds you would like the data to last for. Setting this flag to anything above `0` will set an `expireAt` field for each record that is written to the database.
+    
+    **Important:** Tyk will not create the expiry index for you. In order to implement data expiry for your analytics data, ensure that the index is created This is easily achieved using the [MongoDB command line interface](https://docs.mongodb.com/getting-started/shell/client/).
+    
+    * `dont_set_quota_on_create`: This setting defaults to `false`, but if set to `true`, when the API is used to edit, create or add keys, the quota cache in Redis will not be re-set. By default, all updates or creates to Keys that have Quotas set will re-set the quota (This has been the default behaviour since 1.0).
+    
+      This behaviour can be bypassed on a case-by-case basis by using the `suppress_reset` parameter when making a REST API request. This is the advised mode of operation as it allows for manual, granular control over key quotas and reset timings.
+    
+    * `cache_options`: This section enables you to configure the caching behaviour of Tyk and to enable or disable the caching middleware for your API.
+    
+    * `cache_options.enable_cache`: Set this value to `true` if the cache should be enabled for this endpoint, setting it to false will stop all caching behaviour.
+    
+    * `cache_options.cache_timeout`: The amount of time, in seconds, to keep cached objects, defaults to `60` seconds.
+    
+    * `cache_options.cache_all_safe_requests`: Set this to `true` if you want all *safe* requests (GET, HEAD, OPTIONS) to be cached. This is a blanket setting for APIs where caching is required but you don't want to set individual paths up in the definition.
+    
+    * `cache_options.enable_upstream_cache_control`: Set this to `true` if you want your application to control the cache options for Tyk (TTL and whether to cache or not). See [Caching](/docs/basic-config-and-security/reduce-latency/caching/) for more details.
+    
+    * `response_processors`: Response processors need to be specifically defined so they are loaded on API creation, otherwise the middleware will not fire. In order to have the two main response middleware components fire, the following configuration object should be supplied.
+    
+    ```{json}
+      "response_processors": [
+        {
+            "name": "header_injector",
+            "options": {
+                "add_headers": {"name": "value"},
+                "remove_headers": ["name"]
+            }
+        },
+        {
+          "name": "response_body_transform",
+          "options": {}
+        }
+      ]
+    ```
+    
+    The options for the `header_injector` are global, and will apply to all outbound requests.
+  name: "Batch requests"
 paths:
   /hello:
     get:
@@ -563,7 +744,7 @@ paths:
               schema:
                 items:
                   allOf:
-                  - $ref: https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/schemas/v3.0/schema.json
+                  - $ref: https://raw.githubusercontent.com/TykTechnologies/tyk/refs/heads/master/apidef/oas/schema/3.0.json
                   - $ref: '#/components/schemas/XTykAPIGateway'
                 type: array
           description: List of API definitions in Tyk OAS format.
@@ -664,7 +845,7 @@ paths:
                   url: https://localhost:8080
             schema:
               allOf:
-              - $ref: https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/schemas/v3.0/schema.json
+              - $ref: https://raw.githubusercontent.com/TykTechnologies/tyk/refs/heads/master/apidef/oas/schema/3.0.json
               - $ref: '#/components/schemas/XTykAPIGateway'
       responses:
         "200":
@@ -802,7 +983,7 @@ paths:
                   $ref: '#/components/examples/oasExample'
               schema:
                 allOf:
-                - $ref: https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/schemas/v3.0/schema.json
+                - $ref: https://raw.githubusercontent.com/TykTechnologies/tyk/refs/heads/master/apidef/oas/schema/3.0.json
                 - $ref: '#/components/schemas/XTykAPIGateway'
           description: OK
           headers:
@@ -912,7 +1093,7 @@ paths:
                 upstream:
                   url: https://localhost:8080
             schema:
-              $ref: https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/schemas/v3.0/schema.json
+              $ref: https://raw.githubusercontent.com/TykTechnologies/tyk/refs/heads/master/apidef/oas/schema/3.0.json
       responses:
         "200":
           content:
@@ -1025,7 +1206,7 @@ paths:
                   url: https://localhost:8080
             schema:
               allOf:
-              - $ref: https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/schemas/v3.0/schema.json
+              - $ref: https://raw.githubusercontent.com/TykTechnologies/tyk/refs/heads/master/apidef/oas/schema/3.0.json
               - $ref: '#/components/schemas/XTykAPIGateway'
       responses:
         "200":
@@ -1334,7 +1515,7 @@ paths:
               servers:
               - url: https://localhost:8080
             schema:
-              $ref: https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/schemas/v3.0/schema.json
+              $ref: https://raw.githubusercontent.com/TykTechnologies/tyk/refs/heads/master/apidef/oas/schema/3.0.json
       responses:
         "200":
           content:
@@ -4231,6 +4412,79 @@ paths:
       summary: Hot-reload a group of Tyk nodes.
       tags:
       - Hot Reload
+  /{listen_path}/tyk/batch:
+    post:
+      description: Endpoint to run batch request.
+      operationId: batch
+      parameters:
+        - description: API listen path
+          example: get
+          in: path
+          name: listen_path
+          required: true
+          schema:
+            type: string
+      requestBody:
+        content:
+          application/json:
+            example:
+              requests:
+                - body: ""
+                  headers:
+                    authorization: 1dbc83b9c431649d7698faa9797e2900f
+                    x-tyk-test: "1"
+                    x-tyk-version: "1.2"
+                  method: GET
+                  relative_url: get
+                - body: ""
+                  headers:
+                    authorization: 1dbc83b9c431649d7698faa9797e2900f
+                    x-tyk-test: "2"
+                    x-tyk-version: "1.2"
+                  method: GET
+                  relative_url: get
+              suppress_parallel_execution: false
+            schema:
+              $ref: '#/components/schemas/BatchRequestStructure'
+      responses:
+        "200":
+          content:
+            application/json:
+              example:
+                - body: '{"message": "success"}'
+                  code: 200
+                  headers:
+                    Access-Control-Allow-Credentials:
+                      - "true"
+                    Content-Type:
+                      - application/json
+                  relative_url: get
+              schema:
+                items:
+                  $ref: '#/components/schemas/BatchReplyUnit'
+                type: array
+          description: Success.
+        "400":
+          content:
+            application/json:
+              example:
+                message: Batch request malformed
+                status: error
+              schema:
+                $ref: '#/components/schemas/ApiStatusMessage'
+          description: Bad Request
+        "403":
+          content:
+            application/json:
+              example:
+                message: Attempted administrative access with invalid or missing key!
+                status: error
+              schema:
+                $ref: '#/components/schemas/ApiStatusMessage'
+          description: Forbidden
+      summary: Run batch request.
+      tags:
+        - Batch requests
   /tyk/schema:
     get:
       description: Get OAS schema definition using a version.
@@ -5284,6 +5538,41 @@ components:
         password:
           type: string
       type: object
+    BatchReplyUnit:
+      properties:
+        body:
+          type: string
+        code:
+          type: integer
+        headers:
+          $ref: '#/components/schemas/HttpHeader'
+        relative_url:
+          type: string
+      type: object
+    BatchRequestStructure:
+      properties:
+        requests:
+          items:
+            $ref: '#/components/schemas/RequestDefinition'
+          nullable: true
+          type: array
+        suppress_parallel_execution:
+          type: boolean
+      type: object
+    RequestDefinition:
+      properties:
+        body:
+          type: string
+        headers:
+          additionalProperties:
+            type: string
+          nullable: true
+          type: object
+        method:
+          type: string
+        relative_url:
+          type: string
+      type: object
     BooleanQueryParam:
       enum:
       - true