Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(operator): add Model selector for scale subresource to enable HPA-based scaling #5932

Merged
merged 3 commits into from
Sep 24, 2024
Merged

feat(operator): add Model selector for scale subresource to enable HPA-based scaling #5932

merged 3 commits into from
Sep 24, 2024

Conversation

lc525
Copy link
Member

@lc525 lc525 commented Sep 23, 2024 •
edited
Loading

  • updates the Model CRD to contain a pod selector in the scale subresource
  • sets the selector to a label server=[inference-server-name] matching no actual pods
  • docs [to be moved to gitbook before merging]

Which issue(s) this PR fixes:
Fixes #1190 (internal): allow HPA-based Model autoscaling

Special notes for your reviewer:

  • docs added to the PR to show how HPA would work
  • tested in kind (small-scale)

TODO:

  • test availability during HPA scale-up/scale-down via k6 load test (can be done after merging)

@lc525
add(operator): Model selector for scale subresource to enable HPA-bas…
c930b00 
…ed scaling

- updates the Model CRD to contain a pod selector in the scale subresource
- sets the selector to a label `server=[inference-server-name]` matching no actual pods
- docs [to be moved to gitbook before merging]
@lc525 lc525 requested a review from sakoush as a code owner September 23, 2024 17:41
@lc525 lc525 added the v2 label Sep 23, 2024
@lc525 lc525 marked this pull request as draft September 23, 2024 17:42
@lc525 lc525 changed the title add(operator): Model selector for scale subresource to enable HPA-based scaling feat(operator): add Model selector for scale subresource to enable HPA-based scaling Sep 23, 2024
lc525
lc525 commented Sep 23, 2024
View reviewed changes
k8s/helm-charts/seldon-core-v2-crds/templates/seldon-v2-crds.yaml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generated

lc525
lc525 commented Sep 23, 2024
View reviewed changes
k8s/yaml/crds.yaml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generated

lc525
lc525 commented Sep 23, 2024
View reviewed changes
k8s/yaml/runtime.yaml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generated

lc525
lc525 commented Sep 23, 2024
View reviewed changes
k8s/yaml/servers.yaml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generated

lc525
lc525 commented Sep 23, 2024
View reviewed changes
operator/config/crd/bases/mlops.seldon.io_models.yaml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generated

@lc525 lc525 marked this pull request as ready for review September 23, 2024 17:49
sakoush
sakoush approved these changes Sep 24, 2024
View reviewed changes
Copy link
Member

@sakoush sakoush left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lc525 lc525 force-pushed the INFRA-1190/hpa-autoscaling-verizon branch from 0de0dc8 to 4ba04e9 Compare September 24, 2024 11:24
@lc525
improve docs
d101859 
describe the behaviour when a Model gets scaled up slightly before its Server
@lc525 lc525 force-pushed the INFRA-1190/hpa-autoscaling-verizon branch from 4ba04e9 to d101859 Compare September 24, 2024 16:08
@lc525 lc525 merged commit 1bd8d0f into SeldonIO:v2 Sep 24, 2024
3 checks passed
@lc525 lc525 deleted the INFRA-1190/hpa-autoscaling-verizon branch September 24, 2024 16:29
@lc525 lc525 mentioned this pull request Sep 25, 2024
Merged
@sakoush sakoush mentioned this pull request Oct 23, 2024
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sakoush sakoush sakoush approved these changes

Assignees
No one assigned
Labels
v2
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@lc525 @sakoush