Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add config parameters for fine-grained routing control #126

Merged
merged 22 commits into from
Aug 8, 2024
Merged

feat: add config parameters for fine-grained routing control #126

merged 22 commits into from
Aug 8, 2024

Conversation

polsar88
Copy link
Contributor

@polsar88 polsar88 commented Aug 1, 2024 •
edited
Loading

Description

Design: https://www.notion.so/alchemotion/Error-and-latency-based-routing-in-the-Satsuma-node-gateway-3a60d896c5844f82bf90f68570b00608

Tracker: https://www.notion.so/alchemotion/Project-tracker-Error-latency-based-routing-1999d7ff3e5c430d9af3a7149923d8b9

Type of change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • 😎 New feature (non-breaking change which adds functionality)
  • ⁉️ Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • ⚒️ Refactor (no functional changes)
  • 📖 Documentation (updating or adding docs)

@polsar88 polsar88 changed the title Added instructions to README.md Routing improvements Aug 1, 2024
brianluong
brianluong previously approved these changes Aug 1, 2024
View reviewed changes
@brianluong

This comment was marked as resolved.

Sign in to view
@polsar88

This comment was marked as resolved.

Sign in to view
@polsar88 polsar88 marked this pull request as draft August 1, 2024 22:20
@polsar88 polsar88 changed the title Routing improvements feat: add config parameters for fine-grained routing control Aug 1, 2024
@polsar88 polsar88 requested a review from brianluong August 2, 2024 22:45
@polsar88 polsar88 marked this pull request as ready for review August 2, 2024 22:45
brianluong
brianluong reviewed Aug 6, 2024
View reviewed changes
Copy link
Member

@brianluong brianluong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome to see the wheels turning 🚌

internal/config/config.go Show resolved Hide resolved
internal/config/config.go Outdated Show resolved Hide resolved
internal/config/config.go Outdated Show resolved Hide resolved
@polsar88 polsar88 requested a review from brianluong August 7, 2024 01:10
brianluong
brianluong reviewed Aug 7, 2024
View reviewed changes
internal/config/config.go Outdated Show resolved Hide resolved
internal/config/config.go Outdated Show resolved Hide resolved
internal/config/config.go Outdated Show resolved Hide resolved
@polsar88 polsar88 requested a review from brianluong August 8, 2024 04:52
@polsar88 polsar88 requested a review from brianluong August 8, 2024 19:49
brianluong
brianluong previously approved these changes Aug 8, 2024
View reviewed changes
Copy link
Member

@brianluong brianluong left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Should we merge these (and future) changes into a feature branch (e.g. latency-and-error-based-routing or something)? And once it's all ready we can merge it into main?

internal/config/config.go Outdated Show resolved Hide resolved
@polsar88
Copy link
Contributor Author

polsar88 commented Aug 8, 2024

Should we merge these (and future) changes into a feature branch (e.g. latency-and-error-based-routing or something)? And once it's all ready we can merge it into main?

Sounds good!

@polsar88 polsar88 requested a review from brianluong August 8, 2024 22:03
@polsar88 polsar88 changed the base branch from main to feat/routing-enhancements August 8, 2024 22:35
@polsar88 polsar88 merged commit e949a66 into feat/routing-enhancements Aug 8, 2024
5 checks passed
@polsar88 polsar88 deleted the polsar/routing-enhancements branch August 8, 2024 22:36
@polsar88 polsar88 mentioned this pull request Aug 9, 2024
Merged
5 tasks
polsar88 added a commit that referenced this pull request Sep 25, 2024
@polsar88
feat: config parsing for error and latency-based routing control (#127)
d1b6941 
feat: add config parameters for fine-grained routing control (#126)

* Added instructions to `README.md`

* feat: update dependencies

* feat: updated dependencies

* feat: new mockery version

* feat: add routing config params

* feat: upgrade Go version

* fix: suppress legacy linter errors

* fix: field alignments in structs

* fix: field linter errors

* fix: linter errors

* feat: add routing config params

* feat: validate routing error rate

* feat: refactoring

* feat: add a TODO

* feat: `ErrorsConfig` and `LatencyConfig` are now pointers

* feat: add a field to routing config test

* feat: set defaults for `DetectionWindow` and `BanWindow`

* feat: move RoutingConfig functions closer to the struct

* feat: changed `or` to `and`

* feat: split function into validation and setting defaults components

* feat: settings the config defaults now mirrors the validation workflow

* feat: remove redundant call
polsar88 added a commit that referenced this pull request Sep 25, 2024
@polsar88
feat: upgrade Go version to 1.22 (#129)
677669e 
* feat: add config parameters for fine-grained routing control (#126)

* Added instructions to `README.md`

* feat: update dependencies

* feat: updated dependencies

* feat: new mockery version

* feat: add routing config params

* feat: upgrade Go version

* fix: suppress legacy linter errors

* fix: field alignments in structs

* fix: field linter errors

* fix: linter errors

* feat: add routing config params

* feat: validate routing error rate

* feat: refactoring

* feat: add a TODO

* feat: `ErrorsConfig` and `LatencyConfig` are now pointers

* feat: add a field to routing config test

* feat: set defaults for `DetectionWindow` and `BanWindow`

* feat: move RoutingConfig functions closer to the struct

* feat: changed `or` to `and`

* feat: split function into validation and setting defaults components

* feat: settings the config defaults now mirrors the validation workflow

* feat: remove redundant call

* feat: upgrade Go version to 1.22

* feat: config parsing enhancements (#130)

* feat: config parsing enhancements

* fix: the default threshold is now set in LatencyConfig if not specified

* fix: getLatencyThresholdForMethod

* fix: rename constant

* fix: add TODOs

* feat: expand a test

* fix: chain routing config is no longer initialized if global routing config is not specified

* feat: latency checker (#131)

* feat: latency checker

* fix: rename struct field

* fix: ShouldRunPassiveHealthChecks field is now set correctly

* feat: add method as a Prometheus label

* fix: remove extra RecordRequest call

* fix: rename method

* fix: check if we should run passive check before initializing the client

* fix: rename LatencyCheck to ErrorLatencyCheck

* fix: rename `latency` module to `errorlatency`

* fix: passive health check is no longer initialized if it is disabled

* fix: improve a metric's help text

* fix: rename metric variable names

* fix: typo

* feat: add tests

* feat: latency checker wiring (#132)

* feat: latency checker wiring

* fix: remove ShouldRun check in IsLatencyAcceptable.Apply method

* feat: support `alwaysRoute` option (#133)

* feat: support `alwaysRoute` option

* feat: add logging

* fix: add TODOs

* fix: updated comments

* fix: remove confusing "TODOs"

* fix: better phrasing

* fix: lint

* fix: add panics

* fix: receiver cannot be nil

* feat: check if error/latency routing control is enabled

* fix: improve readability

* fix: improve readability

* fix: remove a TODO

* fix: improve code readability

* fix: improve code readability

* fix: add a comment for `GlobalConfig::setDefaults` method

* fix: rename IsEnhancedRoutingControlEnabled method

* fix: rename HasEnhancedRoutingControlDefined method
polsar88 added a commit that referenced this pull request Oct 11, 2024
@polsar88
test: integration tests for RPC gateway (#137)
69160c2 
* feat: add config parameters for fine-grained routing control (#126)

* Added instructions to `README.md`

* feat: update dependencies

* feat: updated dependencies

* feat: new mockery version

* feat: add routing config params

* feat: upgrade Go version

* fix: suppress legacy linter errors

* fix: field alignments in structs

* fix: field linter errors

* fix: linter errors

* feat: add routing config params

* feat: validate routing error rate

* feat: refactoring

* feat: add a TODO

* feat: `ErrorsConfig` and `LatencyConfig` are now pointers

* feat: add a field to routing config test

* feat: set defaults for `DetectionWindow` and `BanWindow`

* feat: move RoutingConfig functions closer to the struct

* feat: changed `or` to `and`

* feat: split function into validation and setting defaults components

* feat: settings the config defaults now mirrors the validation workflow

* feat: remove redundant call

* feat: upgrade Go version to 1.22

* feat: config parsing enhancements

* feat: latency checker

* feat: latency checker wiring

* feat: support `alwaysRoute` option

* fix: the default threshold is now set in LatencyConfig if not specified

* fix: getLatencyThresholdForMethod

* fix: rename constant

* fix: rename struct field

* fix: ShouldRunPassiveHealthChecks field is now set correctly

* feat: add method as a Prometheus label

* fix: remove extra RecordRequest call

* fix: rename method

* fix: check if we should run passive check before initializing the client

* fix: remove ShouldRun check in IsLatencyAcceptable.Apply method

* feat: add logging

* fix: add TODOs

* fix: updated comments

* fix: add TODOs

* fix: rename LatencyCheck to ErrorLatencyCheck

* fix: rename `latency` module to `errorlatency`

* fix: passive health check is no longer initialized if it is disabled

* fix: improve a metric's help text

* fix: rename metric variable names

* fix: typo

* fix: remove confusing "TODOs"

* fix: better phrasing

* fix: lint

* fix: add panics

* fix: receiver cannot be nil

* feat: check if error/latency routing control is enabled

* feat: expand a test

* fix: chain routing config is no longer initialized if global routing config is not specified

* feat: add tests

* fix: improve readability

* fix: improve readability

* fix: remove a TODO

* fix: improve code readability

* fix: improve code readability

* fix: add a comment for `GlobalConfig::setDefaults` method

* fix: rename IsEnhancedRoutingControlEnabled method

* fix: rename HasEnhancedRoutingControlDefined method

* feat: disable BlockHeightCheck

* fix: bug fix

* fix: latency threshold determination for method

* fix: lint

* fix: return top-level latency threshold

* fix: move code block

* fix: adjust MinNumRequestsForRate

* feat: test servers

* fix: add a TODO

* fix: expand a TODO

* fix: skip failing tests

* fix: add an ignored directory

* feat: add a test config file

* fix: servers are now terminated correctly

* feat: start instructions file on testing

* feat: start a file containing common shell functions

* feat: start the first test script

* fix: rename a constant

* fix: remove an unused function

* fix: error rate recording

* feat: test for error rate routing

* feat: add command to run error based routing tests

* feat: add tests for error string based routing

* feat: add tests for error string and HTTP code based routing

* feat: add tests for error string and HTTP code based routing

* feat: add tests for latency based routing

* feat: add support for testing with the `alwaysRoute` option

* fix: test & fix ban window

* fix: remove a TODO

* fix: field alignment

* fix: field alignment

* fix: readability

* fix: update `README.md`

* feat: initial implementation of `AlwaysRouteRoutingStrategy`

* fix: comment formatting

* fix: comment phrasing

* fix: add a TODO

* fix: renaming

* fix: revert to the previous version of `PriorityRoundRobinStrategy`

* fix: remove `HealthStatus` field

* feat: LatencyChecker can now be configured to only check latency, errors, or both

* feat: create separate instances of LatencyChecker for error and latency checking

* feat: plumbing for `AlwaysRouteFilteringStrategy`

* fix: tests

* fix: removable filters

* feat: add debug logging

* fix: move `getFilterTypeName` function to `node_filter` module

* refactor: deduplicate filter upstreams code

* refactor: remove passive error/latency checker code

* refactor: split error latency checker into two structs

* fix: error and latency filters are now passed to regular filtering routing strategy

* fix: typo

* style: remove unused global variable

* test: add test

* refactor: test

* refactor: test

* refactor: add test

* test: add test

* test: add high latency test

* refactor: routing config struct

* test: add latency test with always route

* test: add error rate test with always route

* fix: change hardcoded constant

* test: add expected output

* test: update expected output

* test: update expected output

* docs: update instructions

* test: add expected output

* refactor: move integration test files under subdir

* test: fix readme and generate expected output

* fix: revert changes to production files
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brianluong brianluong brianluong approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@polsar88 @brianluong