Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KEP-3973: Consider deployment completion in DeploymentPodReplacementPolicy #4976

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

KEP-3973: Consider deployment completion in DeploymentPodReplacementPolicy #4976

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
10753a7
Consider deployment completion in DeploymentPodReplacementPolicy
atiratree Nov 22, 2024
7e92c23
Update PodReplacementPolicy alpha to 1.33
atiratree Nov 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 29 additions & 8 deletions keps/sig-apps/3973-consider-terminating-pods-deployment/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
- [Design Details](#design-details)
- [Deployment Behavior Changes](#deployment-behavior-changes)
- [ReplicaSet Status and Deployment Status Changes](#replicaset-status-and-deployment-status-changes)
- [Deployment Completion and Progress Changes](#deployment-completion-and-progress-changes)
- [Deployment Scaling Changes and a New Annotation for ReplicaSets](#deployment-scaling-changes-and-a-new-annotation-for-replicasets)
- [kubectl Changes](#kubectl-changes)
- [API](#api)
Expand Down Expand Up @@ -279,6 +280,20 @@ To satisfy the requirement for tracking terminating pods, and for implementation
we propose a new field `.status.terminatingReplicas` to the ReplicaSet's and Deployment's
status.

### Deployment Completion and Progress Changes

Currently, when the latest ReplicaSet is fully saturated and all of its pods become available, the
Deployment is declared complete. However, there may still be old terminating pods. These pods can
still be ready and hold/accept connections, meaning that the transition to the latest revision is
not fully complete.

To avoid unexpected behavior, we should not declare the deployment complete until all of its
terminating replicas have been fully terminated. We will therefore delay setting a `NewRSAvailable`
reason to the `DeploymentProgressing` condition, when `TerminationComplete` policy is used.

We will also update the `LastUpdateTime` of the `DeploymentProgressing` condition when the number of
terminating pods decreases to reset the progress deadline.

### Deployment Scaling Changes and a New Annotation for ReplicaSets

Currently, scaling is done proportionally over all ReplicaSets to mitigate the risk of losing
Expand Down Expand Up @@ -383,14 +398,17 @@ See [kubectl Skew](#kubectl-skew) for more details.
type DeploymentPodReplacementPolicy string
const (
// TerminationStarted policy creates replacement Pods when the old Pods start
// terminating (has a .metadata.deletionTimestamp). The total number of
// Deployment Pods can be greater than specified by the Deployment's
// terminating (have a non-null .metadata.deletionTimestamp). The total number
// of Deployment Pods can be greater than specified by the Deployment's
// .spec.replicas and the DeploymentStrategy.
TerminationStarted DeploymentPodReplacementPolicy = "TerminationStarted"
// TerminationComplete policy creates replacement Pods only when the old Pods
// are fully terminated (reach Succeeded or Failed phase). The old Pods are
// subsequently removed. The total number of the Deployment Pods is
// limited by the Deployment's .spec.replicas and the DeploymentStrategy.
//
// This policy will also delay declaring the deployment as complete until all
// of its terminating replicas have been fully terminated.
TerminationComplete DeploymentPodReplacementPolicy = "TerminationComplete"
)
```
Expand All @@ -401,13 +419,15 @@ type DeploymentSpec struct {
// podReplacementPolicy specifies when to create replacement Pods.
// Possible values are:
// - TerminationStarted policy creates replacement Pods when the old Pods start
// terminating (has a .metadata.deletionTimestamp). The total number of
// Deployment Pods can be greater than specified by the Deployment's
// terminating (have a non-null .metadata.deletionTimestamp). The total number
// of Deployment Pods can be greater than specified by the Deployment's
// .spec.replicas and the DeploymentStrategy.
// - TerminationComplete policy creates replacement Pods only when the old Pods
// are fully terminated (reach Succeeded or Failed phase). The old Pods are
// subsequently removed. The total number of the Deployment Pods is
// limited by the Deployment's .spec.replicas and the DeploymentStrategy.
// This policy will also delay declaring the deployment as complete until all
// of its terminating replicas have been fully terminated.
//
// The default behavior when the policy is not specified depends on the DeploymentStrategy:
// - Recreate strategy uses TerminationComplete behavior when recreating the deployment,
Expand Down Expand Up @@ -442,11 +462,11 @@ type ReplicaSetStatus struct {
// +optional
AvailableReplicas int32 `json:"availableReplicas,omitempty" protobuf:"varint,5,opt,name=availableReplicas"`

// The number of terminating replicas (have .metadata.deletionTimestamp) for this replica set.
// The number of terminating pods (have a non-null .metadata.deletionTimestamp) for this replica set.
//
// This is an alpha field. Enable DeploymentPodReplacementPolicy to be able to use this field.
// +optional
TerminatingReplicas int32 `json:"terminatingReplicas,omitempty" protobuf:"varint,7,opt,name=terminatingReplicas"`
TerminatingReplicas *int32 `json:"terminatingReplicas,omitempty" protobuf:"varint,7,opt,name=terminatingReplicas"`
...
}
```
Expand Down Expand Up @@ -476,11 +496,11 @@ type DeploymentStatus struct {
// +optional
UnavailableReplicas int32 `json:"unavailableReplicas,omitempty" protobuf:"varint,5,opt,name=unavailableReplicas"`

// Total number of terminating pods (have .metadata.deletionTimestamp) targeted by this deployment.
// Total number of terminating pods (have a non-null .metadata.deletionTimestamp) targeted by this deployment.
//
// This is an alpha field. Enable DeploymentPodReplacementPolicy to be able to use this field.
// +optional
TerminatingReplicas int32 `json:"terminatingReplicas,omitempty" protobuf:"varint,9,opt,name=terminatingReplicas"`
TerminatingReplicas *int32 `json:"terminatingReplicas,omitempty" protobuf:"varint,9,opt,name=terminatingReplicas"`
...
}
```
Expand Down Expand Up @@ -929,6 +949,7 @@ deployment and replicaset controllers.
- 2023-05-01: First version of the KEP opened (https://github.com/kubernetes/enhancements/pull/3974).
- 2023-12-12: Second version of the KEP opened (https://github.com/kubernetes/enhancements/pull/4357).
- 2024-29-05: Added a Deployment Scaling Changes and a New Annotation for ReplicaSets section (https://github.com/kubernetes/enhancements/pull/4670).
- 2024-22-11: Added a Deployment Completion and Progress Changes section (https://github.com/kubernetes/enhancements/pull/4976).

## Drawbacks

Expand Down
4 changes: 2 additions & 2 deletions keps/sig-apps/3973-consider-terminating-pods-deployment/kep.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ stage: alpha
# The most recent milestone for which work toward delivery of this KEP has been
# done. This can be the current (upcoming) milestone, if it is being actively
# worked on.
latest-milestone: "v1.32"
latest-milestone: "v1.33"

# The milestone at which this feature was, or is targeted to be, at each stage.
milestone:
alpha: "v1.32"
alpha: "v1.33"
beta: ""
stable: ""

Expand Down