Skip to content

Commit

Permalink
fix: ps cmd should ignore dead runners (#1050)
Browse files Browse the repository at this point in the history
fixes #1046

Runners that were found to have died (rather than cleanly being killed)
end up with deployment_id & module_name not set to null.
Changes:
- removes all dead runners from showing in the `ps` command
- correctly updates `deployment_id` and `module_name` to null in the
following cases:
    -  Runner terminates unexpectedly
    -  Runner is stale
  • Loading branch information
matt2e authored Mar 12, 2024
1 parent 4221f57 commit 11a63b1
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 6 deletions.
8 changes: 5 additions & 3 deletions backend/controller/sql/queries.sql
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,8 @@ RETURNING deployment_id;
-- name: KillStaleRunners :one
WITH matches AS (
UPDATE runners
SET state = 'dead'
SET state = 'dead',
deployment_id = NULL
WHERE state <> 'dead' AND last_seen < (NOW() AT TIME ZONE 'utc') - sqlc.arg('timeout')::INTERVAL
RETURNING 1)
SELECT COUNT(*)
Expand All @@ -119,7 +120,8 @@ FROM matches;
-- name: DeregisterRunner :one
WITH matches AS (
UPDATE runners
SET state = 'dead'
SET state = 'dead',
deployment_id = NULL
WHERE key = $1
RETURNING 1)
SELECT COUNT(*)
Expand Down Expand Up @@ -160,7 +162,7 @@ SELECT d.min_replicas,
r.endpoint,
r.labels AS runner_labels
FROM deployments d
LEFT JOIN runners r on d.id = r.deployment_id
LEFT JOIN runners r on d.id = r.deployment_id AND r.state != 'dead'
WHERE d.min_replicas > 0
ORDER BY d.name;

Expand Down
8 changes: 5 additions & 3 deletions backend/controller/sql/queries.sql.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

0 comments on commit 11a63b1

Please sign in to comment.