KEP-4962: Standardizing the Representation of Cluster Switch Network Topology

[dmitsh]

• One-line PR description:
  Standardizing Cluster Network Topology Representation

• Issue link: Standardizing the Representation of Cluster Switch Network Topology #4962

• Other comments:

[k8s-ci-robot]

Welcome @dmitsh!

It looks like this is your first PR to kubernetes/enhancements 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/enhancements has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @dmitsh. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[dmitsh]

/cc @aojea

[dmitsh]

/cc @brianhammons @mwielgus @tardieu @mickeyboxell @arsenetar

[k8s-ci-robot]

@dmitsh: GitHub didn't allow me to request PR reviews from the following users: tardieu, arsenetar, brianhammons.

Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @brianhammons @mwielgus @tardieu @mickeyboxell @arsenetar

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[wojtek-t]

So this object contains N items (of below structure), where N is the number of predefined topology units (accelerator, block, datacenter, zone), right?

What I want to ensure is that those needs to be changed/updated when the cluster grows/shrinks (i.e. that the don't define distance between nodes themselves). But given that for a given node its contents don't depend on other nodes (rather on placements in the physical network), this seems to be fine.

[dmitsh]

@wojtek-t , that's correct, each node will contain QoS metrics between the node and every reachable switch.

[aojea]

/assign @johnbelamaric

for sig-architecture

[aojea]

This is not really the switch, right, is the interface in the node that connects to the switch ... and we already have properties to define attributes on the network interfaces with DRA, see slice 14 https://docs.google.com/presentation/d/1Vdr7BhbYXeWjwmLjGmqnUkvJr_eOUdU0x-JxfXWxUT8/edit#slide=id.g2f750386db2_5_0 so I don't feel we need this additional annotation here

cc: @johnbelamaric @thockin

[dmitsh]

It is not a NIC on the node. These are QoS metrics from the node to every reachable switch. Also, the "switch" in this context could be a physical network device, or an aggregated entity defined by a CSP. For example, AWS returns 3 levels of switches per node, but the actual number of physical switches is unknown.

[aojea]

These are QoS metrics from the node to every reachable switch

how is the Node connected to the first switch :) ?

[aojea]

These are network interfaces on the node, I think we should better model them with DRA https://github.com/kubernetes/enhancements/pull/4965/files#r1846865095 , that also allows us to provide dynamic capabilities to the interfaces

[dmitsh]

Again, as mentioned in the earlier comment, these QoS numbers represent node-to-reachable-switch metrics. They are not per NIC.

[aojea]

does it mean that some switches may not be connected directly?
how then the latency and bandwidth are obtained to guarantee those values?

[aojea]

this is the part we need to loop in SIG architecture, I briefly touched on this with @thockin and it seems it took some time to settle on the region/zone.

So my understanding is that we need to model a hierarchy, this KEP suggest to use nested structures: zone > datacenter > block > accelerator , but we should at least describe in alternative why weights are not better than this, it seems to me that weights are more easy to standardize different topologies, as you only focus on the distance provided between layers, and can support different architectures with multiple layers

[dmitsh]

This KEP is proposing to use reserved network types for typical network architectures, while allowing to extend network topology using custom network types.
We are providing means for weighted approach by specifying distance and/or bandwidth, latency or other metrics.
These are actual measurable physical characteristics of the network, and will be more accurate than specifying static weights.
Once again, we are providing QoS between a node and a switch, so the distance is the number of hops between the node and the switch. Same goes for bandwidth/latency.

[aojea]

IIUIC the use of custom network types means that environment that use them may not be compatible with other environments, that practically removes all the benefits of standarization, another thing where I think weighted/distances model can help better

[dmitsh]

this was already addressed.

[aojea]

After seen this example I'm definitevely not in favor of custom types as it is impossible for a generic tool to infer the distance between these custom types ... it will also create fragmentation and incompatibility as multiple tools can define the same name with different meaning ... the example also talks about levels, that reinforces my idea of weights, so something like network.topology.kubernetes.io/tier: 1

In this example network.topology.kubernetes.io/tor: sw13 will be

• Node: network.topology.kubernetes.io/tier: 1
• ResourceSlice:

apiVersion: resource.k8s.io/v1alpha3
kind: ResourceSlice
…
spec:
    devices:
    - basic:
        attributes:
          tier:
            int: 1
          type:
            string: nvlink
          ip:
            string: 10.1.1.1/24
          latency:
            string: "50us"
          bandwith:
            string: "100gbps"
          

[dmitsh]

When using custom network types, it is mandatory to define distance in the QoS annotation.
We explicitly expressed that in the KEP.

[keithmattix]

Are there existing topology formats or consumers that we want Kubernetes to integrate with? If so, are these integrations goals or non-goals?

[dmitsh]

To the best of my knowledge, only EKS exposes their custom node labels for network layers.
The goal of this KEP is to create a standard way of describing switch network topology.
Ultimately, we want to use this standard in development of Kubernetes-native network-aware scheduler plugin for multi-node workloads. We put this task as a "no goal", as it would be an independent effort.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dmitsh
Once this PR has been reviewed and has the lgtm label, please ask for approval from johnbelamaric. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• keps/sig-network/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sftim]

I'm afraid that if I encountered this in my professional work (cloud consultancy), I'd recommend blocking the change.

Overall, I propose adding these annotations and other details to a new external project, using Kubernetes extensibility mechanisms.

If it's a really useful project, build it and donate it to the CNCF. We like working with other CNCF projects; it can be really positive to do that.

I don't see the case for the annotations and labels and conventions be part of Kubernetes and maintained as part of Kubernetes.

[sftim]

Not keen on "us" here. Either use quantities, or actually write μs, or use integer nanoseconds, or something.

"us" is a term that is hard for people to recognize and easy to confuse (it can be mistaken for the English word "us").

[dmitsh]

Agree. This is mostly for an example of using arbitrary QoS metrics.

[sftim]

Should these be FQDNs maybe?

[dmitsh]

The switch ID could be a network abstraction, as in the case of AWS.

[sftim]

OK, but: why does this standardised way need to be part of Kubernetes itself? You need to explain that as well.

A viable alternative could be making something like NVIDIA topograph but vendor neutral, that isn't part of Kubernetes itself.

[dmitsh]

I just updated the "Motivation" part. Hope it will bring more clarity.

[dmitsh]

The end-goal is to use these labels in Kubernetes-native scheduler plugins.
Hence the proposal to make them consistent across CSPs and on-prem.

[sftim]

OK, so that needs to go into the KEP text. Without that detail, the motivation isn't obvious.

(I reread it, and that detail is there, but it wasn't obvious to me that the key motivation was to enable the scheduling work). Maybe it's worth capturing that a bit more. I agree that Kubernetes' own scheduler should be cautious about depending on the meaning of external or vendor-specific labels.

[sftim]

OK, but: why does this standardised way need to be part of Kubernetes itself? You need to explain that as well.

A viable alternative could be making something like NVIDIA topograph but vendor neutral, that isn't part of Kubernetes itself.

[sftim]

The motivation here could also serve for a vendor-neutral alternative, not part of Kubernetes. For example, you could be proposing that https://opennetworking.org/ be the umbrella organization, not Kubernetes.

[dmitsh]

The purpose of label standardization is to serve as a foundational step toward developing a network-aware gang scheduler for Kubernetes. By establishing a consistent way to represent cluster network topology, this initiative lays the groundwork for advanced scheduling capabilities that take network performance into account. Since this work directly enhances Kubernetes functionality, we submitted it specifically as a KEP (Kubernetes Enhancement Proposal).

[aojea]

I don't see the case for the annotations and labels and conventions be part of Kubernetes and maintained as part of Kubernetes.

Let me write my thoughts on this:

I see value in a predefined set of labels that describe topology, similar to what we have with zone and region, we can have a number of levels that represent proximity for network workloads, the KEP introduces:

1, accelerator: Network interconnect for direct accelerator communication (e.g., Multi-node NVLink interconnect between NVIDIA GPUs)
2. block: Rack-level switches connecting hosts in one or more racks as a block.
3. datacenter: Spine-level switches connecting multiple blocks inside a datacenter.
4.zone: Zonal switches connecting multiple datacenters inside an availability zone.

It can be other names, but having a number of predefined levels standard across environments will benefit the projects that implement scheduling based on network topology, I think 4 levels should be ok for most environments.

And now the friction points:

1. Custom types or arbitrary number of levels, we all know that is hard to represent all the topologies, but if we leave freedom to add custom types then this solution no longer works because it means any deployment will be different, or there can be collision between environments with the same types but different meanings. It also will allow people to use official kubernetes labels for a completely different purpose, it is not likely the proposal of custom types will be accepted, I personally will not be in favor.

2. The annotation suggestions misses who should add the annotation. It also relies on subjective and inconsistently measured attributes, we need things that can be measured the same way everywhere. The NIC or Interface of the node have some physical attributes that influence its potential performance, but the actual latency, bandwidth, and speed depends on a combination of factors, including the network environment and other devices involved. Determining the latency, bandwidth, and speed in a multi-tier network architecture adds another layer of complexity, since it also depends on the hardware, protocols, interconnections and it most probably has dependencies or shared resources that will vary over time.

[sftim]

OK, but responding to #4965 (comment): if the labels (and annotations?) were inside networking.foundation.example and not qos.kubernetes.io, topology.kubernetes.io etc: what do we lose?

If we can't articulate that, we should have pause. Maybe some other project should define the standard (not Kubernetes).

[sftim]

How will we describe multihomed nodes?

For example, a node directly connects to two neighbours, and to two top of rack switches. Pod to pod communication can use either best-effort routing or computed paths with source route metadata added to packets on egress.

Kubernetes doesn't say you can't do that, but this standard implies a hierarchical network layout. Are we tacitly backing a particular network management paradigm?

[sftim]

Suggested change

	<!--
	Why should this KEP _not_ be implemented?
	-->
	This proposal assumes that the network topology is hierarchical.

[dmitsh]

This is not accurate.
You can easy describe a node connected to multiple switches:

A simple example, where we assume that performance of switches is comparable:
Labels:

network.topology.kubernetes.io/level_a: sw01
network.topology.kubernetes.io/level_b: sw02

Annotations:

network.qos.kubernetes.io/switches: {
   "sw01": {
      "distance": 1
   },
   "sw02": {
      "distance": 1
   }
}

An example, where we provide performance metrics:
Labels:

network.topology.kubernetes.io/level_a: sw01
network.topology.kubernetes.io/level_b: sw02

Annotations:

network.qos.kubernetes.io/switches: {
   "sw01": {
      "distance": 1,
      "bandwidth": "40Gbps"
   },
   "sw02": {
      "distance": 1,
      "bandwidth": "60Gbps"
   }
}

[aojea]

OK, but responding to #4965 (comment): if the labels (and annotations?) were inside networking.foundation.example and not qos.kubernetes.io, topology.kubernetes.io etc: what do we lose?

I do see value on topology.kubernetes.io/zone and topology.kubernetes.io/region . It made a difference to deploy High Available applications and consolidate environments, tooling and projects across the ecosystem.

So I think that aiming for something similar for the new type of workloads that require a more granular definition of network topology makes sense ... I really think we should work on what are the new region and zone for these type of workloads first , and adding four new well known labels makes sense to me, I think I can try to get consensues from Google side in 4 labels (@wojtek-t WDYT) , but we'll need people from other cloud providers to agree (@dims , @ritazh , ...) ... this only works if we all agree

[sftim]

Following up #4965 (comment)

I still have concerns; I'll explain more.

We can encourage cloud providers to define their own labels as well.
For example, on AWS a node can be labelled with topology.kubernetes.io/zone and with topology.k8s.aws/zone-id; labelling like that is of course additive.

I agree about the value topology.kubernetes.io/zone and topology.kubernetes.io/region; if we didn't have those, they would be roughly 100% worth defining. With those existing labels, there is also much less risk of wanting to assert that a node is in two of something. A node might connect to two switches, but we very very rarely put nodes in two regions. (If it's in two regions, why did you put your datacenter in Baarle-Hertog or wherever 😉?)

So this KEP wouldn't persuade providers and / or hardware vendors to define their own parallel, provider-specific labels - but encouraging that kind of guidance would bring something on top of a common denominator of labels that Kubernetes does define.

I remain worried about tacitly promoting one network paradigm (a switching hierarchy), especially for a project that might be around another 10 years. This KEP should articulate its alternatives and make clear why we picked the approach we select.

[aojea]

I remain worried about tacitly promoting one network paradigm (a switching hierarchy),

agree, but I see this is about network hierarchy, switching hierarchy is just an implementation of a network hierarchy

Level 1: there are things that are closest , this can be a switch a rack or the PCI bus, the implementer decide this
Level 2: there are things that have to go to hops to get to level 1, this connect multiple level 1, the number of hops is abstracted, you just know your are not at the same level of the things in level 1, the implementer decide this
Level 3: same as level 2 for level 1, implementer decide, it connects level 2s
Level 4: ...

[sftim]

I guess as an actual alternative, we could define an API for representing network topology.

For example, custom resources:

• NetworkTopologyDomain (eg: a row of racks in a datacenter; eg an abstract node placement group)
• InterconnectionClass (eg Ethernet, 1000Mbs, best effort)
• PacketForwarder (switches and routers)

Maybe also:

• OverlayLink (eg internal VPNs)
• LinkBindingClaim ("this Pod should / does use that interface")
• IPAddressRangeAssignment (obvious I hope)
• PathReservation (maybe "please allow this Pod to send 2Gbps to this destination")
• NetworkIsolationGroup (for abstractions like hardware-accelerated packet filtering, or for exemptions from an otherwise zero-trust architecture)
• either or both:
  • InterfacePair (eg WAN links between zones)
  • InterfaceLinkSet (the above, plus shared-media examples like wireless Ethernet)

Remember, alternatives in KEP feedback don't have to be the choice we select. They can instead be viable options we rule out but still list as alternatives we considered.

[bowei]

/ok-to-test

[k8s-ci-robot]

@dmitsh: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-enhancements-test	bb95a3c	link	true	/test pull-enhancements-test
pull-enhancements-verify	bb95a3c	link	true	/test pull-enhancements-verify

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[dmitsh]

I remain worried about tacitly promoting one network paradigm (a switching hierarchy), especially for a project that might be around another 10 years. This KEP should articulate its alternatives and make clear why we picked the approach we select.

I want to stress that we are not advocating for or favoring any specific network hierarchy. On the contrary, our goal is to make the system flexible and adaptable to any type of network.

To achieve this, we propose using reserved types for common configurations, enabling simplicity and brevity. Additionally, we recommend supporting custom types as a more generic solution, where key details like the number of hops (as a minimum input) and additional performance metrics can be specified.

This information will be crucial for optimally scheduling multi-node training jobs or sets of interdependent, data-intensive services. It ensures that network-aware decisions can be made to enhance performance and efficiency.