Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement reserved free runner pool #1030

Closed
alecthomas opened this issue Mar 6, 2024 · 1 comment · Fixed by #1038 or #1044
Closed

Implement reserved free runner pool #1030

alecthomas opened this issue Mar 6, 2024 · 1 comment · Fixed by #1038 or #1044
Assignees
@matt2e

Comments

@alecthomas
Copy link
Collaborator

Currently in development we're suppressing this error:

warn:controller0: Failed to increase deployment replicas: failed to claim runners for oidcauth-86ae51d623: no idle runners found matching labels {"languages":["go"]}: not found

Because it triggers every time a new deployment is created. This message should not be suppressed however, and should probably be logged at error level.

A solution to this is to implement the reserved free runner pool that was in the original design for the autoscaling. That is, the controller will guarantee that a minimum number of free runners are always available.
@github-actions github-actions bot added the triage Issue needs triaging label Mar 6, 2024
@alecthomas alecthomas removed the triage Issue needs triaging label Mar 6, 2024
@alecthomas alecthomas mentioned this issue Mar 6, 2024
Open
@matt2e matt2e mentioned this issue Mar 7, 2024
Merged
@matt2e matt2e closed this as completed in 2b8f713 Mar 7, 2024
@matt2e matt2e mentioned this issue Mar 8, 2024
Merged
@matt2e matt2e reopened this Mar 8, 2024
@matt2e
Copy link
Collaborator

matt2e commented Mar 8, 2024

Reopened as Wes was still seeing this in the console

matt2e added a commit that referenced this issue Mar 8, 2024
@matt2e
fix: reduce error logs for failure to increase replicas (#1044)
8f36327 
fixes #1030

- Log level for failure to increase replicas depends on how many
repeated attempts (error when >= 5)
- Increase default idle runners to 3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@matt2e matt2e

Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@alecthomas @matt2e