From 16ef356c1b0fd0c8bd3fa71743536269ca3f6ae6 Mon Sep 17 00:00:00 2001 From: Logan Blyth Date: Tue, 29 Oct 2024 14:59:28 -0400 Subject: [PATCH] first draft of diagrams --- docs/install-upgrade/build-wiring.md | 166 +++++++++++++++++++++++++++ 1 file changed, 166 insertions(+) diff --git a/docs/install-upgrade/build-wiring.md b/docs/install-upgrade/build-wiring.md index 8b6b863..f163e34 100644 --- a/docs/install-upgrade/build-wiring.md +++ b/docs/install-upgrade/build-wiring.md @@ -85,6 +85,27 @@ This section is meant to help the reader understand how to assemble the primitiv A VPC allows for isolation at layer 3. This is the main building block for users when creating their architecture. Hosts inside of a VPC belong to the same broadcast domain and can communicate with each other, if desired a single VPC can be configured with multiple broadcast domains. The hosts inside of a VPC will likely need to connect to other VPCs or the outside world. To communicate between two VPC a *peering* will need to be created. A VPC can be a logical separation of workloads. By separating these workloads additional controls are available. The logical separation doesn't have to be the traditional database, web, and compute layers it could be development teams who need isolation, it could be tenants inside of an office building, or any separation that allows for better control of the network. Once your VPCs are decided, the rest of the fabric will come together. With the VPCs decided traffic can be prioritized, security can be put into place, and the wiring can begin. The fabric allows for the VPC to span more than a than one switch, which provides great flexibility, for instance workload mobility. +``` mermaid +graph TD + L1([Leaf 1]) + L2([Leaf 2]) + S1["Server 1 + 10.1.0.1"] + S2["Server 2 + 10.1.0.2"] + S3["Server 3 + 10.1.0.3"] + + L1 <--> S1 + L1 <--> S2 + L2 <--> S3 + + subgraph VPC 1 + S1 + S2 + S3 + end +``` ### Connection A connection represents the physical wires in your data center. They connect switches to other switches or switches to servers. @@ -98,6 +119,56 @@ A server connection is a connection used to connect servers to the fabric. The f - *MCLAG* - Two cables going to two different switches, also called dual homing. The switches will need a fabric link between them. - *ESLAG* - Two to four cables going to different switches, also called multi-homing. If four links are used there will need to be four switches connected to a single server with four NIC ports. +``` mermaid +graph TD + S1([Spine 1]) + S2([Spine 2]) + L1([Leaf 1]) + L2([Leaf 2]) + L3([Leaf 3]) + L4([Leaf 4]) + L5([Leaf 5]) + L6([Leaf 6]) + L7([Leaf 7]) + + TS1[Server1] + TS2[Server2] + TS3[Server3] + TS4[Server4] + + S1 ---- L1 & L2 & L3 & L4 & L5 & L6 & L7 + S2 ---- L1 & L2 & L3 & L4 & L5 & L6 & L7 + L1 <-- Bundled --> TS1 + L1 <-- Bundled --> TS1 + L1 <-- Unbundled --> TS2 + L2 <-- MCLAG --> TS3 + L3 <-- MCLAG --> TS3 + L4 <-- ESLAG --> TS4 + L5 <-- ESLAG --> TS4 + L6 <-- ESLAG --> TS4 + L7 <-- ESLAG --> TS4 + + subgraph VPC 1 + TS1 + TS2 + TS3 + TS4 + end + + subgraph MCLAG + L2 + L3 + end + + subgraph ESLAG + L3 + L4 + L5 + L6 + L7 + end + +``` #### Fabric Connections Fabric connections serve as connections between switches, they form the fabric of the network. @@ -107,14 +178,109 @@ Fabric connections serve as connections between switches, they form the fabric o VPCs need VPC Peerings to talk to each other. VPC Peerings come in two varieties: local and remote. +``` mermaid +graph TD + S1([Spine 1]) + S2([Spine 2]) + L1([Leaf 1]) + L2([Leaf 2]) + TS1[Server1] + TS2[Server2] + TS3[Server3] + TS4[Server4] + + S1 <--> L1 + S1 <--> L2 + S2 <--> L1 + S2 <--> L2 + L1 <--> TS1 + L1 <--> TS2 + L2 <--> TS3 + L2 <--> TS4 + + + subgraph VPC 1 + TS1 + TS2 + end + + subgraph VPC 2 + TS3 + TS4 + end +``` + #### Local VPC Peering When there is no dedicated border/peering switch available in the fabric we can use local VPC peering. This kind of peering tries sends traffic between the two VPC's on the switch where either of the VPC's has workloads attached. Due to limitation in the Sonic network operating system this kind of peering bandwidth is limited to the number of VPC loopbacks you have selected while initializing the fabric. Traffic between the VPCs will use the loopback interface, the bandwidth of this connection will be equal to the bandwidth of port used in the loopback. +``` mermaid +graph TD + L1([Leaf 1]) + S1[Server1] + S2[Server2] + S3[Server3] + S4[Server4] + + L1 <-. loopback .-> L1; + L1 <-.-> S1; + L1 <--> S2; + L1 <-.-> S3; + L1 <--> S4; + + linkStyle 1,2,4 color:blue; + + subgraph VPC 1 + S1 + S2 + end + + subgraph VPC 2 + S3 + S4 + end +``` + + #### Remote VPC Peering Remote Peering is used when you need a high bandwidth connection between the VPCs, you will dedicate a switch to the peering traffic. This is either done on the border leaf or on a switch where either of the VPC's are not present. This kind of peering allows peer traffic between different VPC's at line rate and is only limited by fabric bandwidth. Remote peering introduces a few additional hops in the traffic and may cause a small increase in latency. +``` mermaid +graph TD + S1([Spine 1]) + S2([Spine 2]) + L1([Leaf 1]) + L2([Leaf 2]) + L3([Leaf 3]) + TS1[Server1] + TS2[Server2] + TS3[Server3] + TS4[Server4] + + S1 <-.5.-> L1; + S1 <-.2.-> L2; + S1 <-.3,4.-> L3; + S2 <--> L1; + S2 <--> L2; + S2 <--> L3; + L1 <-.6.-> TS1; + L1 <--> TS2; + L2 <--> TS3; + L2 <-.1.-> TS4; + + + subgraph VPC 1 + TS1 + TS2 + end + + subgraph VPC 2 + TS3 + TS4 + end +``` +The dotted line in the diagram shows the traffic flow for remote peering. The traffic could take a different path because of ECMP. It is important to note that Leaf 3 cannot have any servers from VPC 1 or VPC 2 on it, but it can have a different VPC attached to it. #### VPC Loopback