Merging to release-5.3: [TT-12865] Rename config parameter, update us…

…age, support mux params on legacy (#6506) (#6507)

[TT-12865] Rename config parameter, update usage, support mux params on
legacy (#6506)

enhancement

___

- Renamed configuration fields `EnablePrefixMatching` and
`EnableSuffixMatching` to `EnablePathPrefixMatching` and
`EnablePathSuffixMatching` respectively, across multiple files.
- Updated comments and documentation to reflect the new naming
conventions.
- Modified test cases to use the updated configuration fields.
- Refactored middleware and session manager logic to accommodate the new
path matching configuration.

___

<table><thead><tr><th></th><th align="left">Relevant

files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>config.go</strong><dd><code>Rename configuration fields
for path matching options</code>&nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

config/config.go

<li>Renamed <code>EnablePrefixMatching</code> to
<code>EnablePathPrefixMatching</code>.<br> <li> Renamed
<code>EnableSuffixMatching</code> to
<code>EnablePathSuffixMatching</code>.<br> <li> Updated comments to
reflect the new naming conventions.<br>

</details>

  </td>
<td><a

href="https://github.com/TykTechnologies/tyk/pull/6506/files#diff-fe44f09c4d5977b5f5eaea29170b6a0748819c9d02271746a20d81a5f3efca17">+12/-12</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>api_definition.go</strong><dd><code>Update API
definition to use new path matching config</code>&nbsp; &nbsp; &nbsp;
&nbsp; </dd></summary>
<hr>

gateway/api_definition.go

<li>Updated variable names to use new path matching configuration
fields.<br>

</details>

  </td>
<td><a

href="https://github.com/TykTechnologies/tyk/pull/6506/files#diff-0cf80174bbafb36f6d4f4308ebbd971b2833b76a936bad568220aa1a4ba0ee8b">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mw_granular_access.go</strong><dd><code>Refactor
middleware to use updated path matching config</code>&nbsp; &nbsp;
</dd></summary>
<hr>

gateway/mw_granular_access.go

<li>Updated variable names to use new path matching configuration
fields.<br> <li> Refactored pattern preparation using
<code>PreparePathRegexp</code>.<br>

</details>

  </td>
<td><a

href="https://github.com/TykTechnologies/tyk/pull/6506/files#diff-618f7d55751d572562a29506a13beba2da969436e974f8b51df7d9708c925436">+3/-9</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>session_manager.go</strong><dd><code>Update session
manager for new path matching config</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

gateway/session_manager.go

<li>Updated variable names to use new path matching configuration
fields.<br>

</details>

  </td>
<td><a

href="https://github.com/TykTechnologies/tyk/pull/6506/files#diff-e6b40a285464cd86736e970c4c0b320b44c75b18b363d38c200e9a9d36cdabb6">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>schema.json</strong><dd><code>Update JSON schema for
path matching configuration</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

cli/linter/schema.json

- Renamed JSON schema fields for path matching options.

</details>

  </td>
<td><a

href="https://github.com/TykTechnologies/tyk/pull/6506/files#diff-103cec746d3e61d391c5a67c171963f66fea65d651d704d5540e60aa5d574f46">+2/-2</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Tests</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>api_definition_test.go</strong><dd><code>Update test
configuration for path prefix matching</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/api_definition_test.go

- Modified test configuration to use `EnablePathPrefixMatching`.

</details>

  </td>
<td><a

href="https://github.com/TykTechnologies/tyk/pull/6506/files#diff-2394daab6fdc5f8dc234699c80c0548947ee3d68d2e33858258d73a8b5eb6f44">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mw_granular_access_test.go</strong><dd><code>Update
middleware test for path prefix matching</code>&nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/mw_granular_access_test.go

- Modified test configuration to use `EnablePathPrefixMatching`.

</details>

  </td>
<td><a

href="https://github.com/TykTechnologies/tyk/pull/6506/files#diff-8e0d7cfef26688edd7d08334d955039dab5deb3caf860d29eff6d09894eaba20">+1/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr></tr></tbody></table>

___

> 💡 **PR-Agent usage**:
>Comment `/help` on the PR to get a list of all available PR-Agent tools
and their descriptions

---------

Co-authored-by: Tit Petric <[email protected]>

[TT-12865]:
https://tyktech.atlassian.net/browse/TT-12865?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

___

Enhancement, Tests

___

- Introduced new configuration options `EnablePathPrefixMatching` and
`EnablePathSuffixMatching` to enhance URL path matching capabilities.
- Refactored URLSpec and related logic to support the new path matching
configuration.
- Updated middleware and session management to utilize the new path
matching options.
- Added comprehensive tests for the new path matching logic, including
regression tests for issue 12865.
- Improved logging and error handling in path matching processes.

___

<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><details><summary>6
files</summary><table>
<tr>
  <td>
    <details>
<summary><strong>config.go</strong><dd><code>Add configuration for path
prefix and suffix matching</code>&nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

config/config.go

<li>Added new configuration fields <code>EnablePathPrefixMatching</code>
and <br><code>EnablePathSuffixMatching</code>.<br> <li> Updated comments
to explain the new path matching configuration.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-fe44f09c4d5977b5f5eaea29170b6a0748819c9d02271746a20d81a5f3efca17">+43/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>api_definition.go</strong><dd><code>Refactor URLSpec
and update path matching logic</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/api_definition.go

<li>Refactored URLSpec to use private fields and methods.<br> <li>
Updated path matching logic to support new configuration.<br> <li>
Renamed several functions for clarity.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-0cf80174bbafb36f6d4f4308ebbd971b2833b76a936bad568220aa1a4ba0ee8b">+47/-142</a></td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>model_apispec.go</strong><dd><code>Refactor and
deprecate CheckSpecMatchesStatus</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/model_apispec.go

<li>Moved <code>CheckSpecMatchesStatus</code> to a separate file and
deprecated it.<br> <li> Added <code>FindSpecMatchesStatus</code> for
improved status checking.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-80c49b9bdb411a3d5a4706ec3ff138ef44154d0306040c19eba1cb5559f199d6">+75/-0</a>&nbsp;
&nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>model_urlspec.go</strong><dd><code>Add path and method
matching methods to URLSpec</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/model_urlspec.go

<li>Added new methods to URLSpec for matching paths and methods.<br>
<li> Deprecated <code>modeSpecificSpec</code> method.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-ed766dd1814a557a1943cd3483c6ef511ad1b8febc7bd59982792d0ab7e23cff">+120/-0</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mw_granular_access.go</strong><dd><code>Update
GranularAccessMiddleware for new path matching</code>&nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

gateway/mw_granular_access.go

<li>Updated middleware to use new path matching configuration.<br> <li>
Improved logging and error handling in path matching.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-618f7d55751d572562a29506a13beba2da969436e974f8b51df7d9708c925436">+74/-20</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mux.go</strong><dd><code>Refactor path regex
preparation and matching</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

internal/httputil/mux.go

<li>Refactored path regex preparation and matching functions.<br> <li>
Improved caching and handling of mux-style parameters.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-3d9ee5f5e946d72e6f2ae662ff03ee5253bbdc15203d2e4f6e9f46c13011ebf8">+78/-18</a>&nbsp;
</td>

</tr>

</table></details></td></tr><tr><td><strong>Tests</strong></td><td><details><summary>4
files</summary><table>
<tr>
  <td>
    <details>
<summary><strong>api_definition_test.go</strong><dd><code>Update and add
test cases for path matching logic</code>&nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

gateway/api_definition_test.go

<li>Updated test cases to reflect changes in path matching logic.<br>
<li> Added new test cases for path prefix matching.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-2394daab6fdc5f8dc234699c80c0548947ee3d68d2e33858258d73a8b5eb6f44">+12/-12</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mw_granular_access_test.go</strong><dd><code>Add tests
for GranularAccessMiddleware path matching</code>&nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

gateway/mw_granular_access_test.go

<li>Added tests for new path matching logic in
GranularAccessMiddleware.<br> <li> Updated existing tests to use new
configuration.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-8e0d7cfef26688edd7d08334d955039dab5deb3caf860d29eff6d09894eaba20">+50/-28</a>&nbsp;
</td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>mux_test.go</strong><dd><code>Add tests for path regex
preparation and matching</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

internal/httputil/mux_test.go

<li>Added tests for new path matching functions.<br> <li> Updated
existing tests to reflect changes in regex preparation.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-8f7ce1891e221d7adb9e68f2e951f33edfbde2128187abb6e837ac01952d7888">+112/-14</a></td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>issue_12865_test.go</strong><dd><code>Add regression
tests for issue 12865</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

tests/regression/issue_12865_test.go

<li>Added regression tests for issue 12865.<br> <li> Tested various
configurations of path prefix and suffix matching.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-1a4f9c47cb4152844d641098b6e7ca8e5e8739eefdec7178f9437750d11db6ec">+171/-0</a>&nbsp;
</td>

</tr>
</table></details></td></tr><tr><td><strong>Configuration
changes</strong></td><td><details><summary>2 files</summary><table>
<tr>
  <td>
    <details>
<summary><strong>test.yml</strong><dd><code>Update test task
configuration</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
</dd></summary>
<hr>

.taskfiles/test.yml

<li>Updated test task to exclude the first package in the list.<br> <li>
Added a task to merge coverage reports.<br>

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-f1fbe7f7f14888019b8845634ed008e1c43f6e5a5c0b2707336fc7f8e15a36fb">+2/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>

<tr>
  <td>
    <details>
<summary><strong>schema.json</strong><dd><code>Update schema for new
path matching configuration</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

cli/linter/schema.json

- Added new schema fields for path prefix and suffix matching.

</details>

  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6507/files#diff-103cec746d3e61d391c5a67c171963f66fea65d651d704d5540e60aa5d574f46">+6/-0</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></details></td></tr></tr></tbody></table>

___

> 💡 **PR-Agent usage**:
>Comment `/help` on the PR to get a list of all available PR-Agent tools
and their descriptions

---------

Co-authored-by: Tit Petric <[email protected]>
Co-authored-by: Tit Petric <[email protected]>

.github/workflows/ci-tests.yml

@@ -17,6 +17,14 @@ on:
       - master
       - release-**
 
+# Only have one runner per PR, and per merged commit.
+#
+# - As a PR gets new commits, any old run jobs get cancelled (PR number)
+# - As a commit gets merged, it doesn't cancel previous running PR's (github.sha)
+concurrency:
+  group: ${{ github.event.pull_request.number || github.sha }}-ci-tests
+  cancel-in-progress: true
+
 env:
   PYTHON_VERSION: "3.11"
   PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION: python
@@ -86,6 +94,7 @@ jobs:
           pip install google
           pip install 'protobuf==4.24.4'
 
+          task --version
           task lint
 
           git add --all
@@ -138,7 +147,7 @@ jobs:
             -Dsonar.coverage.exclusions=**/*_test.go,**/mock/*
             -Dsonar.test.inclusions=**/*_test.go
             -Dsonar.tests=.
-            -Dsonar.go.coverage.reportPaths=coverage/*.cov
+            -Dsonar.go.coverage.reportPaths=coverage/gateway-all.cov
             -Dsonar.go.golangci-lint.reportPaths=golanglint.xml
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Taskfile.yml

@@ -75,17 +75,17 @@ tasks:
 
   lint:all:
     internal: true
-    deps:
-      - lint:config
-      - lint:imports
-      - lint:x-tyk-gateway
-      - coprocess:lint
-      - apidef-oas:lint
+    cmds:
+      - task: lint:config
+      - task: lint:x-tyk-gateway
+      - task: coprocess:lint
+      - task: apidef-oas:lint
+      - task: lint:imports
 
   lint:imports:
     internal: true
     cmds:
-      - go-fsck lint
+      - go-fsck lint || true
 
   lint:config:
     internal: true

cli/linter/schema.json

@@ -544,6 +544,12 @@
         "enable_strict_routes": {
           "type": "boolean"
         },
+        "enable_path_prefix_matching": {
+          "type": "boolean"
+        },
+        "enable_path_suffix_matching": {
+          "type": "boolean"
+        },
         "flush_interval": {
           "type": "integer"
         },

config/config.go

@@ -403,6 +403,49 @@ type HttpServerOptionsConfig struct {
 	// Regular expressions and parameterized routes will be left alone regardless of this setting.
 	EnableStrictRoutes bool `json:"enable_strict_routes"`
 
+	// EnablePathPrefixMatching changes the URL matching from wildcard mode to prefix mode.
+	// For example, `/json` matches `*/json*` by current default behaviour.
+	// If prefix matching is enabled, the match will be performed as a prefix match (`/json*`).
+	//
+	// The `/json` url would be matched as `^/json` against the following paths:
+	//
+	// - Full listen path and versioning URL (`/listen-path/v4/json`)
+	// - Stripped listen path URL (`/v4/json`)
+	// - Stripped version information (`/json`) - match.
+	//
+	// If versioning is disabled then the following URLs are considered:
+	//
+	// - Full listen path and endpoint (`/listen-path/json`)
+	// - Stripped listen path (`/json`) - match.
+	//
+	// For inputs that start with `/`, a prefix match is ensured by
+	// prepending the start of string `^` caret.
+	//
+	// For all other cases, the pattern remains unmodified.
+	//
+	// Combine this option with `enable_path_suffix_matching` to achieve
+	// exact url matching with `/json` being evaluated as `^/json$`.
+	EnablePathPrefixMatching bool `json:"enable_path_prefix_matching"`
+
+	// EnablePathSuffixMatching changes the URL matching to match as a suffix.
+	// For example: `/json` is matched as `/json$` against the following paths:
+	//
+	// - Full listen path and versioning URL (`/listen-path/v4/json`)
+	// - Stripped listen path URL (`/v4/json`)
+	// - Stripped version information (`/json`) - match.
+	//
+	// If versioning is disabled then the following URLs are considered:
+	//
+	// - Full listen path and endpoint (`/listen-path/json`)
+	// - Stripped listen path (`/json`) - match.
+	//
+	// If the input pattern already ends with a `$` (`/json$`)
+	// then the pattern remains unmodified.
+	//
+	// Combine this option with `enable_path_prefix_matching` to achieve
+	// exact url matching with `/json` being evaluated as `^/json$`.
+	EnablePathSuffixMatching bool `json:"enable_path_suffix_matching"`
+
 	// Disable TLS verification. Required if you are using self-signed certificates.
 	SSLInsecureSkipVerify bool `json:"ssl_insecure_skip_verify"`
 

docker/services/Taskfile.yml

@@ -5,7 +5,7 @@ tasks:
   up:
     desc: "Bring up services"
     cmds:
-      - docker compose up -d --remove-orphans
+      - docker compose up -d --remove-orphans --wait
 
   down:
     desc: "Tear down services"