Skip to content

Commit

Permalink
Merging to release-5.7.0: [TT-13607] Only import components/io and co…
Browse files Browse the repository at this point in the history
…mponents/kafka (#6720) (#6722)

### **User description**
[TT-13607] Only import components/io and components/kafka (#6720)

### **User description**
Attempt to fix [TT-13607](https://tyktech.atlassian.net/browse/TT-13607)

We should not need to import all components. I also ran `go mod tidy` to
reduce the dependency graph.

Initial memory usage::

- with two components: io and kafka: 34.5 MB
- with all components: 71.2 MB

I tested the changes locally with Kafka & labs-streams sample. It works
as usual.

[TT-13607]:

https://tyktech.atlassian.net/browse/TT-13607?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ


___

### **PR Type**
enhancement, dependencies


___

### **Description**
- Optimized the import statements in `stream.go` to include only
necessary components (`io` and `kafka`), reducing memory usage on cold
start.
- Cleaned up the `go.sum` file by removing unused dependencies, which
helps in reducing the dependency graph.



___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant

files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>stream.go</strong><dd><code>Optimize component imports
to reduce memory usage</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

ee/middleware/streams/stream.go

<li>Removed import of all components.<br> <li> Added specific imports
for <code>components/io</code> and <code>components/kafka</code>.<br>


</details>


  </td>
<td><a

href="https://github.com/TykTechnologies/tyk/pull/6720/files#diff-12571ea9605d5a2dd5ab5aa36972649881f87a84a39b7074213d29d24fc396a8">+2/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Dependencies</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>go.sum</strong><dd><code>Clean up and update Go module
dependencies</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

go.sum

<li>Removed multiple unused module dependencies.<br> <li> Updated
dependency checksums.<br>


</details>


  </td>
<td><a

href="https://github.com/TykTechnologies/tyk/pull/6720/files#diff-3295df7234525439d778f1b282d146a4f1ff6b415248aaac074e8042d9f42d63">+1/-1094</a></td>

</tr>
</table></td></tr></tr></tbody></table>

___

> 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull
request to receive relevant information

[TT-13607]:
https://tyktech.atlassian.net/browse/TT-13607?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[TT-13607]:
https://tyktech.atlassian.net/browse/TT-13607?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[TT-13607]:
https://tyktech.atlassian.net/browse/TT-13607?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ


___

### **PR Type**
Enhancement, Dependencies


___

### **Description**
- Optimized the import statements in `stream.go` to include only
necessary components (`io`, `kafka`, and `nats`), reducing memory usage
on cold start.
- Cleaned up the `go.mod` file by removing unused dependencies and
updating some dependency versions, which helps in reducing the
dependency graph.



___



### **Changes walkthrough** 📝
<table><thead><tr><th></th><th align="left">Relevant
files</th></tr></thead><tbody><tr><td><strong>Enhancement</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>stream.go</strong><dd><code>Optimize component imports
to reduce memory usage</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; </dd></summary>
<hr>

ee/middleware/streams/stream.go

<li>Removed import of all components.<br> <li> Added specific imports
for <code>components/io</code>, <code>components/kafka</code>, and
<br><code>components/nats</code>.<br>


</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6722/files#diff-12571ea9605d5a2dd5ab5aa36972649881f87a84a39b7074213d29d24fc396a8">+3/-1</a>&nbsp;
&nbsp; &nbsp; </td>

</tr>
</table></td></tr><tr><td><strong>Dependencies</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>go.mod</strong><dd><code>Clean up and update Go module
dependencies</code>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; </dd></summary>
<hr>

go.mod

<li>Removed multiple unused module dependencies.<br> <li> Updated some
dependency versions.<br>


</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6722/files#diff-33ef32bf6c23acb95f5902d7097b7a1d5128ca061167ec0716715b0b9eeaa5f6">+3/-208</a>&nbsp;
</td>

</tr>
</table></td></tr><tr><td><strong>Additional files
(token-limit)</strong></td><td><table>
<tr>
  <td>
    <details>
<summary><strong>go.sum</strong><dd><code>...</code>&nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;
&nbsp; </dd></summary>
<hr>

go.sum

...



</details>


  </td>
<td><a
href="https://github.com/TykTechnologies/tyk/pull/6722/files#diff-3295df7234525439d778f1b282d146a4f1ff6b415248aaac074e8042d9f42d63">+10/-1078</a></td>

</tr>
</table></td></tr></tr></tbody></table>

___

> 💡 **PR-Agent usage**: Comment `/help "your question"` on any pull
request to receive relevant information

Co-authored-by: Burak Sezer <[email protected]>
  • Loading branch information
buger and buraksezer authored Nov 20, 2024
1 parent 41b8331 commit acad9cd
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 1,286 deletions.
4 changes: 3 additions & 1 deletion ee/middleware/streams/stream.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ import (
"github.com/sirupsen/logrus"
"gopkg.in/yaml.v2"

_ "github.com/warpstreamlabs/bento/public/components/all"
_ "github.com/warpstreamlabs/bento/public/components/io"
_ "github.com/warpstreamlabs/bento/public/components/kafka"
_ "github.com/warpstreamlabs/bento/public/components/nats"
"github.com/warpstreamlabs/bento/public/service"

_ "github.com/TykTechnologies/tyk/internal/portal"
Expand Down
Loading

0 comments on commit acad9cd

Please sign in to comment.