diff --git a/alpha-6/search/search_index.json b/alpha-6/search/search_index.json index f243db8..9819498 100755 --- a/alpha-6/search/search_index.json +++ b/alpha-6/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Introduction","text":"

Hedgehog Open Network Fabric is an open networking platform that brings the user experience enjoyed by so many in the public cloud to private environments. It comes without vendor lock-in.

Fabric is built around the concept of VPCs (Virtual Private Clouds), similar to the public clouds. It provides a multi-tenant API to define the user intent on network isolation and connectivity, which gets automatically transformed into configuration for switches and software appliances.

You can read more about its concepts and architecture in the documentation.

You can find how to download and try Fabric on the self-hosted fully virtualized lab or on hardware.

"},{"location":"architecture/fabric/","title":"Hedgehog Network Fabric","text":"

The Hedgehog Open Network Fabric is an open-source network architecture that provides connectivity between virtual and physical workloads and provides a way to achieve network isolation between different groups of workloads using standard BGP EVPN and VXLAN technology. The fabric provides a standard Kubernetes interface to manage the elements in the physical network and provides a mechanism to configure virtual networks and define attachments to these virtual networks. The Hedgehog Fabric provides isolation between different groups of workloads by placing them in different virtual networks called VPC's. To achieve this, it defines different abstractions starting from the physical network where a set of Connection objects defines how a physical server on the network connects to a physical switch on the fabric.

"},{"location":"architecture/fabric/#underlay-network","title":"Underlay Network","text":"

The Hedgehog Fabric currently supports two underlay network topologies.

"},{"location":"architecture/fabric/#collapsed-core","title":"Collapsed Core","text":"

A collapsed core topology is just a pair of switches connected in a MCLAG configuration with no other network elements. All workloads attach to these two switches.

The leaves in this setup are configured to be in a MCLAG pair and servers can either be connected to both switches as a MCLAG port channel or as orphan ports connected to only one switch. Both the leaves peer to external networks using BGP and act as gateway for workloads attached to them. The configuration of the underlay in the collapsed core is very simple and is ideal for very small deployments.

"},{"location":"architecture/fabric/#spine-leaf","title":"Spine-Leaf","text":"

A spine-leaf topology is a standard Clos network with workloads attaching to leaf switches and the spines providing connectivity between different leaves.

This kind of topology is useful for bigger deployments and provides all the advantages of a typical Clos network. The underlay network is established using eBGP where each leaf has a separate ASN and peers will all spines in the network. RFC7938 was used as the reference for establishing the underlay network.

"},{"location":"architecture/fabric/#overlay-network","title":"Overlay Network","text":"

The overlay network runs on top the underlay network to create a virtual network. The overlay network isolates control and data plane traffic between different virtual networks and the underlay network. Visualization is achieved in the Hedgehog Fabric by encapsulating workload traffic over VXLAN tunnels that are source and terminated on the leaf switches in the network. The fabric uses BGP-EVPN/VXLAN to enable the creation and management of virtual networks on top of the physical one. The fabric supports multiple virtual networks over the same underlay network to support multi-tenancy. Each virtual network in the Hedgehog Fabric is identified by a VPC. The following subsections contain a high-level overview of how VPCs are implemented in the Hedgehog Fabric and its associated objects.

"},{"location":"architecture/fabric/#vpc","title":"VPC","text":"

The previous subsections have described what a VPC is, and how to attach workloads to a specific VPC. Here comes a description of how VPCs are actually implemented on the network to provide the view of a private network.

"},{"location":"architecture/fabric/#vpc-peering","title":"VPC Peering","text":"

To enable communication between 2 different VPCs, one needs to configure a VPC peering policy. The Hedgehog Fabric supports two different peering modes.

"},{"location":"architecture/overview/","title":"Overview","text":"

Under construction.

"},{"location":"concepts/overview/","title":"Concepts","text":""},{"location":"concepts/overview/#introduction","title":"Introduction","text":"

Hedgehog Open Network Fabric is built on top of Kubernetes and uses Kubernetes API to manage its resources. It means that all user-facing APIs are Kubernetes Custom Resources (CRDs), so you can use standard Kubernetes tools to manage Fabric resources.

Hedgehog Fabric consists of the following components:

"},{"location":"concepts/overview/#fabric-api","title":"Fabric API","text":"

All infrastructure is represented as a set of Fabric resource (Kubernetes CRDs) and named Wiring Diagram. With this representation, Fabric defines switches, servers, control nodes, external systems and connections between them in a single place and then uses these definitions to deploy and manage the whole infrastructure. On top of the Wiring Diagram, Fabric provides a set of APIs to manage the VPCs and the connections between them and between VPCs and External systems.

"},{"location":"concepts/overview/#wiring-diagram-api","title":"Wiring Diagram API","text":"

Wiring Diagram consists of the following resources:

"},{"location":"concepts/overview/#user-facing-api","title":"User-facing API","text":""},{"location":"concepts/overview/#fabricator","title":"Fabricator","text":"

Installer builder and VLAB.

"},{"location":"concepts/overview/#das-boot","title":"Das Boot","text":"

Switch boot and installation.

"},{"location":"concepts/overview/#fabric","title":"Fabric","text":"

Control plane and switch agent.

"},{"location":"contribute/docs/","title":"Documentation","text":""},{"location":"contribute/docs/#getting-started","title":"Getting started","text":"

This documentation is done using MkDocs with multiple plugins enabled. It's based on the Markdown, you can find basic syntax overview here.

In order to contribute to the documentation, you'll need to have Git and Docker installed on your machine as well as any editor of your choice, preferably supporting Markdown preview. You can run the preview server using following command:

make serve\n

Now you can open continuously updated preview of your edits in browser at http://127.0.0.1:8000. Pages will be automatically updated while you're editing.

Additionally you can run

make build\n

to make sure that your changes will be built correctly and doesn't break documentation.

"},{"location":"contribute/docs/#workflow","title":"Workflow","text":"

If you want to quick edit any page in the documentation, you can press the Edit this page icon at the top right of the page. It'll open the page in the GitHub editor. You can edit it and create a pull request with your changes.

Please, never push to the master or release/* branches directly. Always create a pull request and wait for the review.

Each pull request will be automatically built and preview will be deployed. You can find the link to the preview in the comments in pull request.

"},{"location":"contribute/docs/#repository","title":"Repository","text":"

Documentation is organized in per-release branches:

Latest release branch is referenced as latest version in the documentation and will be used by default when you open the documentation.

"},{"location":"contribute/docs/#file-layout","title":"File layout","text":"

All documentation files are located in docs directory. Each file is a Markdown file with .md extension. You can create subdirectories to organize your files. Each directory can have a .pages file that overrides the default navigation order and titles.

For example, top-level .pages in this repository looks like this:

nav:\n  - index.md\n  - getting-started\n  - concepts\n  - Wiring Diagram: wiring\n  - Install & Upgrade: install-upgrade\n  - User Guide: user-guide\n  - Reference: reference\n  - Troubleshooting: troubleshooting\n  - ...\n  - release-notes\n  - contribute\n

Where you can add pages by file name like index.md and page title will be taked from the file (first line with #). Additionally, you can reference the whole directory to created nested section in navigation. You can also add custom titles by using : separator like Wiring Diagram: wiring where Wiring Diagram is a title and wiring is a file/directory name.

More details in the MkDocs Pages plugin.

"},{"location":"contribute/docs/#abbreaviations","title":"Abbreaviations","text":"

You can find abbreviations in includes/abbreviations.md file. You can add various abbreviations there and all usages of the defined words in the documentation will get a highlight.

For example, we have following in includes/abbreviations.md:

*[HHFab]: Hedgehog Fabricator - a tool for building Hedgehog Fabric\n

It'll highlight all usages of HHFab in the documentation and show a tooltip with the definition like this: HHFab.

"},{"location":"contribute/docs/#markdown-extensions","title":"Markdown extensions","text":"

We're using MkDocs Material theme with multiple extensions enabled. You can find detailed reference here, but here you can find some of the most useful ones.

To view code for examples, please, check the source code of this page.

"},{"location":"contribute/docs/#text-formatting","title":"Text formatting","text":"

Text can be deleted and replacement text added. This can also be combined into onea single operation. Highlighting is also possible and comments can be added inline.

Formatting can also be applied to blocks by putting the opening and closing tags on separate lines and adding new lines between the tags and the content.

Keyboard keys can be written like so:

Ctrl+Alt+Del

Amd inline icons/emojis can be added like this:

:fontawesome-regular-face-laugh-wink:\n:fontawesome-brands-twitter:{ .twitter }\n

"},{"location":"contribute/docs/#admonitions","title":"Admonitions","text":"

Admonitions, also known as call-outs, are an excellent choice for including side content without significantly interrupting the document flow. Different types of admonitions are available, each with a unique icon and color. Details can be found here.

Lorem ipsum

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor massa, nec semper lorem quam in massa.

"},{"location":"contribute/docs/#code-blocks","title":"Code blocks","text":"

Details can be found here.

Simple code block with line nums and highlighted lines:

bubble_sort.py
def bubble_sort(items):\n    for i in range(len(items)):\n        for j in range(len(items) - 1 - i):\n            if items[j] > items[j + 1]:\n                items[j], items[j + 1] = items[j + 1], items[j]\n

Code annotations:

theme:\n  features:\n    - content.code.annotate # (1)\n
  1. I'm a code annotation! I can contain code, formatted text, images, ... basically anything that can be written in Markdown.
"},{"location":"contribute/docs/#tabs","title":"Tabs","text":"

You can use Tabs to better organize content.

CC++ 
#include <stdio.h>\n\nint main(void) {\n  printf(\"Hello world!\\n\");\n  return 0;\n}\n
#include <iostream>\n\nint main(void) {\n  std::cout << \"Hello world!\" << std::endl;\n  return 0;\n}\n
"},{"location":"contribute/docs/#tables","title":"Tables","text":"Method Description GET Fetch resource PUT Update resource DELETE Delete resource"},{"location":"contribute/docs/#diagrams","title":"Diagrams","text":"

You can directly include Mermaid diagrams in your Markdown files. Details can be found here.

graph LR\n  A[Start] --> B{Error?};\n  B -->|Yes| C[Hmm...];\n  C --> D[Debug];\n  D --> B;\n  B ---->|No| E[Yay!];
sequenceDiagram\n  autonumber\n  Alice->>John: Hello John, how are you?\n  loop Healthcheck\n      John->>John: Fight against hypochondria\n  end\n  Note right of John: Rational thoughts!\n  John-->>Alice: Great!\n  John->>Bob: How about you?\n  Bob-->>John: Jolly good!
"},{"location":"contribute/overview/","title":"Overview","text":"

Under construction.

"},{"location":"getting-started/download/","title":"Download","text":""},{"location":"getting-started/download/#getting-access","title":"Getting access","text":"

Prior to General Availability, access to the full software is limited and requires Design Partner Agreement. Please submit a ticket with the request using Hedgehog Support Portal.

After that you will be provided with the credentials to access the software on GitHub Package. In order to use the software, log in to the registry using the following command:

docker login ghcr.io\n
"},{"location":"getting-started/download/#downloading-the-software","title":"Downloading the software","text":"

The main entry point for the software is the Hedgehog Fabricator CLI named hhfab. All software is published into the OCI registry GitHub Package including binaries, container images, or Helm charts. Download the latest stable hhfab binary from the GitHub Package using the following command:

curl -fsSL https://i.hhdev.io/hhfab | bash\n

Or download a specific version using the following command:

curl -fsSL https://i.hhdev.io/hhfab | VERSION=alpha-X bash\n

Use the VERSION environment variable to specify the version of the software to download. By default, the latest release is downloaded. You can pick a specific release series (e.g. alpha-2) or a specific release.

The download script requires ORAS to be installed. ORAS is used to download the binary from the OCI registry and can be installed using following command:

curl -fsSL https://i.hhdev.io/oras | bash\n

Currently only Linux x86 is supported for running hhfab.

"},{"location":"getting-started/download/#next-steps","title":"Next steps","text":""},{"location":"install-upgrade/build-wiring/","title":"Build Wiring Diagram","text":"

Under construction.

In the meantime, to have a look at working wiring diagram for Hedgehog Fabric, run the sample generator that produces VLAB-compatible wiring diagrams:

ubuntu@sl-dev:~$ hhfab wiring sample -h\nNAME:\n   hhfab wiring sample - sample wiring diagram (would work for vlab)\n\nUSAGE:\n   hhfab wiring sample [command options] [arguments...]\n\nOPTIONS:\n   --brief, -b                    brief output (only warn and error) (default: false)\n   --fabric-mode value, -m value  fabric mode (one of: collapsed-core, spine-leaf) (default: \"spine-leaf\")\n   --help, -h                     show help\n   --verbose, -v                  verbose output (includes debug) (default: false)\n\n   wiring generator options:\n\n   --chain-control-link         chain control links instead of all switches directly connected to control node if fabric mode is spine-leaf (default: false)\n   --control-links-count value  number of control links if chain-control-link is enabled (default: 0)\n   --fabric-links-count value   number of fabric links if fabric mode is spine-leaf (default: 0)\n   --mclag-leafs-count value    number of mclag leafs (should be even) (default: 0)\n   --mclag-peer-links value     number of mclag peer links for each mclag leaf (default: 0)\n   --mclag-session-links value  number of mclag session links for each mclag leaf (default: 0)\n   --orphan-leafs-count value   number of orphan leafs (default: 0)\n   --spines-count value         number of spines if fabric mode is spine-leaf (default: 0)\n   --vpc-loopbacks value        number of vpc loopbacks for each switch (default: 0)\n
"},{"location":"install-upgrade/config/","title":"Fabric Configuration","text":"

For more information about how to use hhfab init, run hhfab init --help.

"},{"location":"install-upgrade/config/#configure-switch-users","title":"Configure switch users","text":"

It's currently only possible by using a config yaml file for the hhfab init -c <config-file.yaml> command. You can specify users to be configured on the switches in the following format:

config:\n    ...\n    fabric:\n        ...\n        switchUsers:\n          - name: test\n            password: $5$oj/NxDtFw3eTyini$VHwdjWXSNYRxlFMu.1S5ZlGJbUF/CGmCAZIBroJlax4\n            role: operator\n

Where name is the username, password is the password hash created with openssl passwd -5 command, and role is the role of the user, one of admin or operator (read-only access to sonic-cli command on the switches).

"},{"location":"install-upgrade/config/#forward-switch-metrics-and-logs","title":"Forward switch metrics and logs","text":"

There is an option to enable Grafana Alloy on all switches to forward metrics and logs to the configured targets using Prometheus Remote-Write API and Loki API. If those APIs are available from Control Node(s), but not from the switches, it's possible to enable HTTP Proxy on Control Node(s) that will be used by Grafana Alloy running on the switches to access the configured targets. It could be done by passing --control-proxy=true to hhfab init.

Metrics includes port speeds, counters, errors, operational status, transceivers, fans, power supplies, temperature sensors, BGP neighbors, LLDP neighbors, and more. Logs include agent logs.

Configuring the exporters and targets is currently only possible by using a config yaml file for the hhfab init -c <config-file.yaml> command using the following format:

config:\n    ...\n    fabric:\n        ...\n        alloy:\n            agentScrapeIntervalSeconds: 120\n            controlProxyURL: http://172.30.1.1:31028\n            lokiTargets:\n                grafana_cloud: # target name, multiple targets can be configured\n                    basicAuth: # optional\n                        password: \"<password>\"\n                        username: \"<username>\"\n                    labels: # labels to be added to all logs\n                        env: env-1\n                    url: https://logs-prod-021.grafana.net/loki/api/v1/push\n                    useControlProxy: true # if the Loki API is not available from the switches directly, use the Control Node as a proxy\n            prometheusTargets:\n                grafana_cloud: # target name, multiple targets can be configured\n                    basicAuth: # optional\n                        password: \"<password>\"\n                        username: \"<username>\"\n                    labels: # labels to be added to all metrics\n                        env: env-1\n                    sendIntervalSeconds: 120\n                    url: https://prometheus-prod-36-prod-us-west-0.grafana.net/api/prom/push\n                    useControlProxy: true # if the Loki API is not available from the switches directly, use the Control Node as a proxy\n            unixExporterCollectors: # list of node-exporter collectors to enable, https://grafana.com/docs/alloy/latest/reference/components/prometheus.exporter.unix/#collectors-list\n                - cpu\n                - filesystem\n                - loadavg\n                - meminfo\n            unixExporterEnabled: true\n            unixScrapeIntervalSeconds: 120\n

For additional options, see the AlloyConfig struct in Fabric repo.

"},{"location":"install-upgrade/onie-update/","title":"ONIE Update/Upgrade","text":""},{"location":"install-upgrade/onie-update/#hedgehog-onie-honie-supported-systems","title":"Hedgehog ONIE (HONIE) Supported Systems","text":""},{"location":"install-upgrade/onie-update/#updating-onie","title":"Updating ONIE","text":""},{"location":"install-upgrade/onie-update/#via-usb","title":"Via USB","text":"

This example shows how to update a DELL S5248 to Hedgehog ONIE (HONIE).

Note: the USB port is on the back of the switch with the Management and Console.

  1. Prepare the USB stick by burning the honie-usb.img to a 4G or larger USB drive.

  2. Insert the USB drive into the switch. For example, burn the file to disk X of a macOS machine with sudo dd if=honie-usb.img of=/dev/rdiskX bs=1m.

  3. Boot into ONIE Installer

  4. ONIE will install the ONIE update and reboot. Here are some sample logs:

    ONIE: OS Install Mode ...\nPlatform\u00a0 : x86_64-dellemc_s5200_c3538-r0\nVersion \u00a0 : 3.40.1.1-7 <- Non HONIE version\nBuild Date: 2020-03-24T20:44-07:00\nInfo: Mounting EFI System on /boot/efi ...\nInfo: BIOS mode: UEFI\nInfo: Making NOS install boot mode persistent.\nInfo: Using eth0 MAC address: 3c:2c:30:66:f0:00\nInfo: eth0:\u00a0 Checking link... up.\nInfo: Trying DHCPv4 on interface: eth0\nWarning: Unable to configure interface using DHCPv4: eth0\nONIE: Using link-local IPv4 addr: eth0: 169.254.95.249/16\nStarting: klogd... done.\nStarting: dropbear ssh daemon... done.\nStarting: telnetd... done.\ndiscover: installer mode detected.\u00a0 Running installer.\nStarting: discover... done.\nPlease press Enter to activate this console. Info: eth0:\u00a0 Checking link... up.\nInfo: Trying DHCPv4 on interface: eth0\nWarning: Unable to configure interface using DHCPv4: eth0\nONIE: Using link-local IPv4 addr: eth0: 169.254.6.139/16\nONIE: Starting ONIE Service Discovery\nInfo: Attempting file://dev/sdb1/onie-installer-x86_64-dellemc_s5248f_c3538-r0 ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64-dellemc_s5248f_c3538-r0 ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64-dellemc_s5248f_c3538-r0.bin ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64-dellemc_s5248f_c3538.bin ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-dellemc_s5248f_c3538 ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-dellemc_s5248f_c3538.bin ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64-bcm ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64-bcm.bin ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64 ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64.bin ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer.bin ...\nONIE: Executing installer: file://dev/sdb1/onie-installer-x86_64-dellemc_s5248f_c3538-r0\nVerifying image checksum ... OK.\nPreparing image archive ... OK.\nONIE: Version \u00a0 \u00a0 \u00a0 : 3.40.1.1-8 <- HONIE Version\nONIE: Architecture\u00a0 : x86_64\nONIE: Machine \u00a0 \u00a0 \u00a0 : dellemc_s5200_c3538\nONIE: Machine Rev \u00a0 : 0\nONIE: Config Version: 1\nONIE: Build Date\u00a0 \u00a0 : 2023-12-15T23:43+00:00\nInstalling ONIE on: /dev/sda\nONIE: NOS install successful: file://dev/sdb1/onie-installer-x86_64-dellemc_s5248f_c3538-r0\nONIE: Rebooting...\ndiscover: installer mode detected.\nStopping: discover...start-stop-daemon: warning: killing process 665: No such process\nInfo: Unmounting kernel filesystems\numount: can't unmount /: Invalid argument\nThe system is going down NOW!\nSent SIGTERM to all processes\nSent SIGKILL to all processes\nRequesting system reboot\n

  5. The system is now ready for use.

"},{"location":"install-upgrade/overview/","title":"Install Fabric","text":"

Under construction.

"},{"location":"install-upgrade/overview/#prerequisites","title":"Prerequisites","text":""},{"location":"install-upgrade/overview/#main-steps","title":"Main steps","text":"

This chapter is dedicated to the Hedgehog Fabric installation on bare-metal control node(s) and switches, their preparation and configuration.

Get hhfab installed following instructions from the Download section.

The main steps to install Fabric are:

  1. Install hhfab on the machines with access to the Internet
    1. Prepare Wiring Diagram
    2. Select Fabric Configuration
    3. Build Control Node configuration and installer
  2. Install Control Node
    1. Install Flatcar Linux on the Control Node
    2. Upload and run Control Node installer on the Control Node
  3. Prepare supported switches
    1. Install Hedgehog ONIE (HONIE) on them
    2. Reboot them into ONIE Install Mode to have them automatically provisioned
"},{"location":"install-upgrade/overview/#build-control-node-configuration-and-installer","title":"Build Control Node configuration and installer","text":"

It's the only step that requires Internet access, to download artifacts and build the installer.

Once you've prepared the Wiring Diagram, initialize Fabricator by running hhfab init command and passing optional configuration into it as well as wiring diagram file(s) as flags. Additionally, there are a lot of customizations available as flags, e.g. to setup default credentials, keys and etc. For more details on the command invocation, refer to hhfab init --help.

The --dev option activates the development mode which enables default credentials and keys for the Control Node and switches:

Alternatively, you can pass your own credentials and keys using --authorized-key and --control-password-hash flags. Generate a password hash with command openssl passwd -5. Further customization items are available in the config file and can be passed using the --config flag.

hhfab init --preset lab --dev --wiring file1.yaml --wiring file2.yaml\nhhfab build\n

As a result, you will get the following files in the .hhfab directory or the one you've passed using --basedir flag:

More details on configuring the Fabric are available in the Configuration section.

"},{"location":"install-upgrade/overview/#install-control-node","title":"Install Control Node","text":"

Control Node installation is fully air-gapped and doesn't require Internet access.

Download the latest stable Flatcar Container Linux ISO and boot into it (using IPMI attaching media, USB stick or any other way).

Once you've booted into the Flatcar installer, upload the file ignition.json built during the previous step to the system and run the Flatcar installation:

sudo flatcar-install -d /dev/sda -i ignition.json\n

Where /dev/sda is a disk you want to install Control Node to and ignition.json is the control-os/ignition.json file from previous step uploaded to the Flatcar installer.

Once the installation is finished, reboot the machine and wait for it to boot into the installed Flatcar Linux.

At that point, you should get into the installed Flatcar Linux using the dev or provided credentials with user core and you can now install Hedgehog Open Network Fabric on it. Download control-install.tgz to the just installed Control Node (for example, by using scp) and run it.

tar xzf control-install.tgz && cd control-install && sudo ./hhfab-recipe run\n

The command prints the logs generated while installing Fabric (including logs from the Kubernetes cluster, miscellaneous OCI registry misc components, and more). At the end, you should observe lines similar to the following:

...\n01:34:45 INF Running name=reloader-image op=\"push fabricator/reloader:v1.0.40\"\n01:34:47 INF Running name=reloader-chart op=\"push fabricator/charts/reloader:1.0.40\"\n01:34:47 INF Running name=reloader-install op=\"file /var/lib/rancher/k3s/server/manifests/hh-reloader-install.yaml\"\n01:34:47 INF Running name=reloader-wait op=\"wait deployment/reloader-reloader\"\ndeployment.apps/reloader-reloader condition met\n01:35:15 INF Done took=3m39.586394608s\n

At that point, you can start interacting with the Fabric using kubectl, kubectl fabric and k9s, all preinstalled as part of the Control Node installer.

You can now get HONIE installed on your switches and reboot them into ONIE Install Mode to have them automatically provisioned from the Control Node.

"},{"location":"install-upgrade/requirements/","title":"System Requirements","text":""},{"location":"install-upgrade/requirements/#non-ha-minimal-setup-1-control-node","title":"Non-HA (minimal) setup - 1 Control Node","text":" Minimal Recommended CPU 4 8 RAM 12 GB 16 GB Disk 100 GB 250 GB"},{"location":"install-upgrade/requirements/#future-ha-setup-3-control-nodes-per-node","title":"(Future) HA setup - 3+ Control Nodes (per node)","text":" Minimal Recommended CPU 4 8 RAM 12 GB 16 GB Disk 100 GB 250 GB"},{"location":"install-upgrade/requirements/#device-participating-in-the-hedgehog-fabric-eg-switch","title":"Device participating in the Hedgehog Fabric (e.g. switch)","text":" Minimal Recommended CPU 1 2 RAM 1 GB 1.5 GB Disk 5 GB 10 GB"},{"location":"install-upgrade/supported-devices/","title":"Supported Devices","text":""},{"location":"install-upgrade/supported-devices/#spine","title":"Spine","text":""},{"location":"install-upgrade/supported-devices/#leaf","title":"Leaf","text":""},{"location":"reference/api/","title":"API Reference","text":""},{"location":"reference/api/#packages","title":"Packages","text":""},{"location":"reference/api/#agentgithedgehogcomv1alpha2","title":"agent.githedgehog.com/v1alpha2","text":"

Package v1alpha2 contains API Schema definitions for the agent v1alpha2 API group. This is the internal API group for the switch and control node agents. Not intended to be modified by the user.

"},{"location":"reference/api/#resource-types","title":"Resource Types","text":""},{"location":"reference/api/#adminstatus","title":"AdminStatus","text":"

Underlying type: string

Appears in: - SwitchStateInterface

"},{"location":"reference/api/#agent","title":"Agent","text":"

Agent is an internal API object used by the controller to pass all relevant information to the agent running on a specific switch in order to fully configure it and manage its lifecycle. It is not intended to be used directly by users. Spec of the object isn't user-editable, it is managed by the controller. Status of the object is updated by the agent and is used by the controller to track the state of the agent and the switch it is running on. Name of the Agent object is the same as the name of the switch it is running on and it's created in the same namespace as the Switch object.

Field Description Default Validation apiVersion string agent.githedgehog.com/v1alpha2 kind string Agent metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. status AgentStatus Status is the observed state of the Agent"},{"location":"reference/api/#agentstatus","title":"AgentStatus","text":"

AgentStatus defines the observed state of the agent running on a specific switch and includes information about the switch itself as well as the state of the agent and applied configuration.

Appears in: - Agent

Field Description Default Validation version string Current running agent version installID string ID of the agent installation, used to track NOS re-installs runID string ID of the agent run, used to track NOS reboots lastHeartbeat Time Time of the last heartbeat from the agent lastAttemptTime Time Time of the last attempt to apply configuration lastAttemptGen integer Generation of the last attempt to apply configuration lastAppliedTime Time Time of the last successful configuration application lastAppliedGen integer Generation of the last successful configuration application state SwitchState Detailed switch state updated with each heartbeat conditions Condition array Conditions of the agent, includes readiness marker for use with kubectl wait"},{"location":"reference/api/#bgpmessages","title":"BGPMessages","text":"

Appears in: - SwitchStateBGPNeighbor

Field Description Default Validation received BGPMessagesCounters sent BGPMessagesCounters"},{"location":"reference/api/#bgpmessagescounters","title":"BGPMessagesCounters","text":"

Appears in: - BGPMessages

Field Description Default Validation capability integer keepalive integer notification integer open integer routeRefresh integer update integer"},{"location":"reference/api/#bgpneighborsessionstate","title":"BGPNeighborSessionState","text":"

Underlying type: string

Appears in: - SwitchStateBGPNeighbor

"},{"location":"reference/api/#bgppeertype","title":"BGPPeerType","text":"

Underlying type: string

Appears in: - SwitchStateBGPNeighbor

"},{"location":"reference/api/#operstatus","title":"OperStatus","text":"

Underlying type: string

Appears in: - SwitchStateInterface

"},{"location":"reference/api/#switchstate","title":"SwitchState","text":"

Appears in: - AgentStatus

Field Description Default Validation nos SwitchStateNOS Information about the switch and NOS interfaces object (keys:string, values:SwitchStateInterface) Switch interfaces state (incl. physical, management and port channels) breakouts object (keys:string, values:SwitchStateBreakout) Breakout ports state (port -> breakout state) bgpNeighbors object (keys:string, values:map[string]SwitchStateBGPNeighbor) State of all BGP neighbors (VRF -> neighbor address -> state) platform SwitchStatePlatform State of the switch platform (fans, PSUs, sensors) criticalResources SwitchStateCRM State of the critical resources (ACLs, routes, etc.)"},{"location":"reference/api/#switchstatebgpneighbor","title":"SwitchStateBGPNeighbor","text":"

Appears in: - SwitchState

Field Description Default Validation connectionsDropped integer enabled boolean establishedTransitions integer lastEstablished Time lastRead Time lastResetReason string lastResetTime Time lastWrite Time localAS integer messages BGPMessages peerAS integer peerGroup string peerPort integer peerType BGPPeerType remoteRouterID string sessionState BGPNeighborSessionState shutdownMessage string prefixes object (keys:string, values:SwitchStateBGPNeighborPrefixes)"},{"location":"reference/api/#switchstatebgpneighborprefixes","title":"SwitchStateBGPNeighborPrefixes","text":"

Appears in: - SwitchStateBGPNeighbor

Field Description Default Validation received integer receivedPrePolicy integer sent integer"},{"location":"reference/api/#switchstatebreakout","title":"SwitchStateBreakout","text":"

Appears in: - SwitchState

Field Description Default Validation mode string members string array status string"},{"location":"reference/api/#switchstatecrm","title":"SwitchStateCRM","text":"

Appears in: - SwitchState

Field Description Default Validation aclStats SwitchStateCRMACLStats stats SwitchStateCRMStats"},{"location":"reference/api/#switchstatecrmacldetails","title":"SwitchStateCRMACLDetails","text":"

Appears in: - SwitchStateCRMACLInfo

Field Description Default Validation groupsAvailable integer groupsUsed integer tablesAvailable integer tablesUsed integer"},{"location":"reference/api/#switchstatecrmaclinfo","title":"SwitchStateCRMACLInfo","text":"

Appears in: - SwitchStateCRMACLStats

Field Description Default Validation lag SwitchStateCRMACLDetails port SwitchStateCRMACLDetails rif SwitchStateCRMACLDetails switch SwitchStateCRMACLDetails vlan SwitchStateCRMACLDetails"},{"location":"reference/api/#switchstatecrmaclstats","title":"SwitchStateCRMACLStats","text":"

Appears in: - SwitchStateCRM

Field Description Default Validation egress SwitchStateCRMACLInfo ingress SwitchStateCRMACLInfo"},{"location":"reference/api/#switchstatecrmstats","title":"SwitchStateCRMStats","text":"

Appears in: - SwitchStateCRM

Field Description Default Validation dnatEntriesAvailable integer dnatEntriesUsed integer fdbEntriesAvailable integer fdbEntriesUsed integer ipmcEntriesAvailable integer ipmcEntriesUsed integer ipv4NeighborsAvailable integer ipv4NeighborsUsed integer ipv4NexthopsAvailable integer ipv4NexthopsUsed integer ipv4RoutesAvailable integer ipv4RoutesUsed integer ipv6NeighborsAvailable integer ipv6NeighborsUsed integer ipv6NexthopsAvailable integer ipv6NexthopsUsed integer ipv6RoutesAvailable integer ipv6RoutesUsed integer nexthopGroupMembersAvailable integer nexthopGroupMembersUsed integer nexthopGroupsAvailable integer nexthopGroupsUsed integer snatEntriesAvailable integer snatEntriesUsed integer"},{"location":"reference/api/#switchstateinterface","title":"SwitchStateInterface","text":"

Appears in: - SwitchState

Field Description Default Validation enabled boolean adminStatus AdminStatus operStatus OperStatus mac string lastChanged Time speed string counters SwitchStateInterfaceCounters transceiver SwitchStateTransceiver lldpNeighbors SwitchStateLLDPNeighbor array"},{"location":"reference/api/#switchstateinterfacecounters","title":"SwitchStateInterfaceCounters","text":"

Appears in: - SwitchStateInterface

Field Description Default Validation inBitsPerSecond float inDiscards integer inErrors integer inPktsPerSecond float inUtilization integer lastClear Time outBitsPerSecond float outDiscards integer outErrors integer outPktsPerSecond float outUtilization integer"},{"location":"reference/api/#switchstatelldpneighbor","title":"SwitchStateLLDPNeighbor","text":"

Appears in: - SwitchStateInterface

Field Description Default Validation chassisID string systemName string systemDescription string portID string portDescription string manufacturer string model string serialNumber string"},{"location":"reference/api/#switchstatenos","title":"SwitchStateNOS","text":"

SwitchStateNOS contains information about the switch and NOS received from the switch itself by the agent

Appears in: - SwitchState

Field Description Default Validation asicVersion string ASIC name, such as \"broadcom\" or \"vs\" buildCommit string NOS build commit buildDate string NOS build date builtBy string NOS build user configDbVersion string NOS config DB version, such as \"version_4_2_1\" distributionVersion string Distribution version, such as \"Debian 10.13\" hardwareVersion string Hardware version, such as \"X01\" hwskuVersion string Hwsku version, such as \"DellEMC-S5248f-P-25G-DPB\" kernelVersion string Kernel version, such as \"5.10.0-21-amd64\" mfgName string Manufacturer name, such as \"Dell EMC\" platformName string Platform name, such as \"x86_64-dellemc_s5248f_c3538-r0\" productDescription string NOS product description, such as \"Enterprise SONiC Distribution by Broadcom - Enterprise Base package\" productVersion string NOS product version, empty for Broadcom SONiC serialNumber string Switch serial number softwareVersion string NOS software version, such as \"4.2.0-Enterprise_Base\" uptime string Switch uptime, such as \"21:21:27 up 1 day, 23:26, 0 users, load average: 1.92, 1.99, 2.00 \""},{"location":"reference/api/#switchstateplatform","title":"SwitchStatePlatform","text":"

Appears in: - SwitchState

Field Description Default Validation fans object (keys:string, values:SwitchStatePlatformFan) psus object (keys:string, values:SwitchStatePlatformPSU) temperature object (keys:string, values:SwitchStatePlatformTemperature)"},{"location":"reference/api/#switchstateplatformfan","title":"SwitchStatePlatformFan","text":"

Appears in: - SwitchStatePlatform

Field Description Default Validation direction string speed float presense boolean status boolean"},{"location":"reference/api/#switchstateplatformpsu","title":"SwitchStatePlatformPSU","text":"

Appears in: - SwitchStatePlatform

Field Description Default Validation inputCurrent float inputPower float inputVoltage float outputCurrent float outputPower float outputVoltage float presense boolean status boolean"},{"location":"reference/api/#switchstateplatformtemperature","title":"SwitchStatePlatformTemperature","text":"

Appears in: - SwitchStatePlatform

Field Description Default Validation temperature float alarms string highThreshold float criticalHighThreshold float lowThreshold float criticalLowThreshold float"},{"location":"reference/api/#switchstatetransceiver","title":"SwitchStateTransceiver","text":"

Appears in: - SwitchStateInterface

Field Description Default Validation description string cableClass string formFactor string connectorType string present string cableLength float operStatus string temperature float voltage float serialNumber string vendor string vendorPart string vendorOUI string vendorRev string"},{"location":"reference/api/#dhcpgithedgehogcomv1alpha2","title":"dhcp.githedgehog.com/v1alpha2","text":"

Package v1alpha2 contains API Schema definitions for the dhcp v1alpha2 API group. It is the primary internal API group for the intended Hedgehog DHCP server configuration and storing leases as well as making them available to the end user through API. Not intended to be modified by the user.

"},{"location":"reference/api/#resource-types_1","title":"Resource Types","text":""},{"location":"reference/api/#dhcpallocated","title":"DHCPAllocated","text":"

DHCPAllocated is a single allocated IP with expiry time and hostname from DHCP requests, it's effectively a DHCP lease

Appears in: - DHCPSubnetStatus

Field Description Default Validation ip string Allocated IP address expiry Time Expiry time of the lease hostname string Hostname from DHCP request"},{"location":"reference/api/#dhcpsubnet","title":"DHCPSubnet","text":"

DHCPSubnet is the configuration (spec) for the Hedgehog DHCP server and storage for the leases (status). It's primary internal API group, but it makes allocated IPs / leases information available to the end user through API. Not intended to be modified by the user.

Field Description Default Validation apiVersion string dhcp.githedgehog.com/v1alpha2 kind string DHCPSubnet metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec DHCPSubnetSpec Spec is the desired state of the DHCPSubnet status DHCPSubnetStatus Status is the observed state of the DHCPSubnet"},{"location":"reference/api/#dhcpsubnetspec","title":"DHCPSubnetSpec","text":"

DHCPSubnetSpec defines the desired state of DHCPSubnet

Appears in: - DHCPSubnet

Field Description Default Validation subnet string Full VPC subnet name (including VPC name), such as \"vpc-0/default\" cidrBlock string CIDR block to use for VPC subnet, such as \"10.10.10.0/24\" gateway string Gateway, such as 10.10.10.1 startIP string Start IP from the CIDRBlock to allocate IPs, such as 10.10.10.10 endIP string End IP from the CIDRBlock to allocate IPs, such as 10.10.10.99 vrf string VRF name to identify specific VPC (will be added to DHCP packets by DHCP relay in suboption 151), such as \"VrfVvpc-1\" as it's named on switch circuitID string VLAN ID to identify specific subnet withing the VPC, such as \"Vlan1000\" as it's named on switch pxeURL string PXEURL (optional) to identify the pxe server to use to boot hosts connected to this segment such as http://10.10.10.99/bootfilename or tftp://10.10.10.99/bootfilename, http query strings are not supported"},{"location":"reference/api/#dhcpsubnetstatus","title":"DHCPSubnetStatus","text":"

DHCPSubnetStatus defines the observed state of DHCPSubnet

Appears in: - DHCPSubnet

Field Description Default Validation allocated object (keys:string, values:DHCPAllocated) Allocated is a map of allocated IPs with expiry time and hostname from DHCP requests"},{"location":"reference/api/#vpcgithedgehogcomv1alpha2","title":"vpc.githedgehog.com/v1alpha2","text":"

Package v1alpha2 contains API Schema definitions for the vpc v1alpha2 API group. It is public API group for the VPCs and Externals APIs. Intended to be used by the user.

"},{"location":"reference/api/#resource-types_2","title":"Resource Types","text":""},{"location":"reference/api/#external","title":"External","text":"

External object represents an external system connected to the Fabric and available to the specific IPv4Namespace. Users can do external peering with the external system by specifying the name of the External Object without need to worry about the details of how external system is attached to the Fabric.

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string External metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec ExternalSpec Spec is the desired state of the External status ExternalStatus Status is the observed state of the External"},{"location":"reference/api/#externalattachment","title":"ExternalAttachment","text":"

ExternalAttachment is a definition of how specific switch is connected with external system (External object). Effectively it represents BGP peering between the switch and external system including all needed configuration.

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string ExternalAttachment metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec ExternalAttachmentSpec Spec is the desired state of the ExternalAttachment status ExternalAttachmentStatus Status is the observed state of the ExternalAttachment"},{"location":"reference/api/#externalattachmentneighbor","title":"ExternalAttachmentNeighbor","text":"

ExternalAttachmentNeighbor defines the BGP neighbor configuration for the external attachment

Appears in: - ExternalAttachmentSpec

Field Description Default Validation asn integer ASN is the ASN of the BGP neighbor ip string IP is the IP address of the BGP neighbor to peer with"},{"location":"reference/api/#externalattachmentspec","title":"ExternalAttachmentSpec","text":"

ExternalAttachmentSpec defines the desired state of ExternalAttachment

Appears in: - ExternalAttachment

Field Description Default Validation external string External is the name of the External object this attachment belongs to connection string Connection is the name of the Connection object this attachment belongs to (essentialy the name of the switch/port) switch ExternalAttachmentSwitch Switch is the switch port configuration for the external attachment neighbor ExternalAttachmentNeighbor Neighbor is the BGP neighbor configuration for the external attachment"},{"location":"reference/api/#externalattachmentstatus","title":"ExternalAttachmentStatus","text":"

ExternalAttachmentStatus defines the observed state of ExternalAttachment

Appears in: - ExternalAttachment

"},{"location":"reference/api/#externalattachmentswitch","title":"ExternalAttachmentSwitch","text":"

ExternalAttachmentSwitch defines the switch port configuration for the external attachment

Appears in: - ExternalAttachmentSpec

Field Description Default Validation vlan integer VLAN (optional) is the VLAN ID used for the subinterface on a switch port specified in the connection, set to 0 if no VLAN is used ip string IP is the IP address of the subinterface on a switch port specified in the connection"},{"location":"reference/api/#externalpeering","title":"ExternalPeering","text":"

ExternalPeering is the Schema for the externalpeerings API

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string ExternalPeering metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec ExternalPeeringSpec Spec is the desired state of the ExternalPeering status ExternalPeeringStatus Status is the observed state of the ExternalPeering"},{"location":"reference/api/#externalpeeringspec","title":"ExternalPeeringSpec","text":"

ExternalPeeringSpec defines the desired state of ExternalPeering

Appears in: - ExternalPeering

Field Description Default Validation permit ExternalPeeringSpecPermit Permit defines the peering policy - which VPC and External to peer with and which subnets/prefixes to permit"},{"location":"reference/api/#externalpeeringspecexternal","title":"ExternalPeeringSpecExternal","text":"

ExternalPeeringSpecExternal defines the External-side of the configuration to peer with

Appears in: - ExternalPeeringSpecPermit

Field Description Default Validation name string Name is the name of the External to peer with prefixes ExternalPeeringSpecPrefix array Prefixes is the list of prefixes to permit from the External to the VPC"},{"location":"reference/api/#externalpeeringspecpermit","title":"ExternalPeeringSpecPermit","text":"

ExternalPeeringSpecPermit defines the peering policy - which VPC and External to peer with and which subnets/prefixes to permit

Appears in: - ExternalPeeringSpec

Field Description Default Validation vpc ExternalPeeringSpecVPC VPC is the VPC-side of the configuration to peer with external ExternalPeeringSpecExternal External is the External-side of the configuration to peer with"},{"location":"reference/api/#externalpeeringspecprefix","title":"ExternalPeeringSpecPrefix","text":"

ExternalPeeringSpecPrefix defines the prefix to permit from the External to the VPC

Appears in: - ExternalPeeringSpecExternal

Field Description Default Validation prefix string Prefix is the subnet to permit from the External to the VPC, e.g. 0.0.0.0/0 for any route including default route.It matches any prefix length less than or equal to 32 effectively permitting all prefixes within the specified one."},{"location":"reference/api/#externalpeeringspecvpc","title":"ExternalPeeringSpecVPC","text":"

ExternalPeeringSpecVPC defines the VPC-side of the configuration to peer with

Appears in: - ExternalPeeringSpecPermit

Field Description Default Validation name string Name is the name of the VPC to peer with subnets string array Subnets is the list of subnets to advertise from VPC to the External"},{"location":"reference/api/#externalpeeringstatus","title":"ExternalPeeringStatus","text":"

ExternalPeeringStatus defines the observed state of ExternalPeering

Appears in: - ExternalPeering

"},{"location":"reference/api/#externalspec","title":"ExternalSpec","text":"

ExternalSpec describes IPv4 namespace External belongs to and inbound/outbound communities which are used to filter routes from/to the external system.

Appears in: - External

Field Description Default Validation ipv4Namespace string IPv4Namespace is the name of the IPv4Namespace this External belongs to inboundCommunity string InboundCommunity is the inbound community to filter routes from the external system (e.g. 65102:5000) outboundCommunity string OutboundCommunity is theoutbound community that all outbound routes will be stamped with (e.g. 50000:50001)"},{"location":"reference/api/#externalstatus","title":"ExternalStatus","text":"

ExternalStatus defines the observed state of External

Appears in: - External

"},{"location":"reference/api/#ipv4namespace","title":"IPv4Namespace","text":"

IPv4Namespace represents a namespace for VPC subnets allocation. All VPC subnets withing a single IPv4Namespace are non-overlapping. Users can create multiple IPv4Namespaces to allocate same VPC subnets.

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string IPv4Namespace metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec IPv4NamespaceSpec Spec is the desired state of the IPv4Namespace status IPv4NamespaceStatus Status is the observed state of the IPv4Namespace"},{"location":"reference/api/#ipv4namespacespec","title":"IPv4NamespaceSpec","text":"

IPv4NamespaceSpec defines the desired state of IPv4Namespace

Appears in: - IPv4Namespace

Field Description Default Validation subnets string array Subnets is the list of subnets to allocate VPC subnets from, couldn't overlap between each other and with Fabric reserved subnets MaxItems: 20 MinItems: 1"},{"location":"reference/api/#ipv4namespacestatus","title":"IPv4NamespaceStatus","text":"

IPv4NamespaceStatus defines the observed state of IPv4Namespace

Appears in: - IPv4Namespace

"},{"location":"reference/api/#vpc","title":"VPC","text":"

VPC is Virtual Private Cloud, similar to the public cloud VPC it provides an isolated private network for the resources with support for multiple subnets each with user-provided VLANs and on-demand DHCP.

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string VPC metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec VPCSpec Spec is the desired state of the VPC status VPCStatus Status is the observed state of the VPC"},{"location":"reference/api/#vpcattachment","title":"VPCAttachment","text":"

VPCAttachment is the Schema for the vpcattachments API

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string VPCAttachment metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec VPCAttachmentSpec Spec is the desired state of the VPCAttachment status VPCAttachmentStatus Status is the observed state of the VPCAttachment"},{"location":"reference/api/#vpcattachmentspec","title":"VPCAttachmentSpec","text":"

VPCAttachmentSpec defines the desired state of VPCAttachment

Appears in: - VPCAttachment

Field Description Default Validation subnet string Subnet is the full name of the VPC subnet to attach to, such as \"vpc-1/default\" connection string Connection is the name of the connection to attach to the VPC nativeVLAN boolean NativeVLAN is the flag to indicate if the native VLAN should be used for attaching the VPC subnet"},{"location":"reference/api/#vpcattachmentstatus","title":"VPCAttachmentStatus","text":"

VPCAttachmentStatus defines the observed state of VPCAttachment

Appears in: - VPCAttachment

"},{"location":"reference/api/#vpcdhcp","title":"VPCDHCP","text":"

VPCDHCP defines the on-demand DHCP configuration for the subnet

Appears in: - VPCSubnet

Field Description Default Validation relay string Relay is the DHCP relay IP address, if specified, DHCP server will be disabled enable boolean Enable enables DHCP server for the subnet range VPCDHCPRange Range (optional) is the DHCP range for the subnet if DHCP server is enabled pxeURL string PXEURL (optional) to identify the pxe server to use to boot hosts connected to this segment such as http://10.10.10.99/bootfilename or tftp://10.10.10.99/bootfilename, http query strings are not supported"},{"location":"reference/api/#vpcdhcprange","title":"VPCDHCPRange","text":"

VPCDHCPRange defines the DHCP range for the subnet if DHCP server is enabled

Appears in: - VPCDHCP

Field Description Default Validation start string Start is the start IP address of the DHCP range end string End is the end IP address of the DHCP range"},{"location":"reference/api/#vpcpeer","title":"VPCPeer","text":"

Appears in: - VPCPeeringSpec

Field Description Default Validation subnets string array Subnets is the list of subnets to advertise from current VPC to the peer VPC MaxItems: 10 MinItems: 1"},{"location":"reference/api/#vpcpeering","title":"VPCPeering","text":"

VPCPeering represents a peering between two VPCs with corresponding filtering rules. Minimal example of the VPC peering showing vpc-1 to vpc-2 peering with all subnets allowed:

spec:\n  permit:\n  - vpc-1: {}\n    vpc-2: {}\n
Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string VPCPeering metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec VPCPeeringSpec Spec is the desired state of the VPCPeering status VPCPeeringStatus Status is the observed state of the VPCPeering"},{"location":"reference/api/#vpcpeeringspec","title":"VPCPeeringSpec","text":"

VPCPeeringSpec defines the desired state of VPCPeering

Appears in: - VPCPeering

Field Description Default Validation remote string permit map[string]VPCPeer array Permit defines a list of the peering policies - which VPC subnets will have access to the peer VPC subnets. MaxItems: 10 MinItems: 1"},{"location":"reference/api/#vpcpeeringstatus","title":"VPCPeeringStatus","text":"

VPCPeeringStatus defines the observed state of VPCPeering

Appears in: - VPCPeering

"},{"location":"reference/api/#vpcspec","title":"VPCSpec","text":"

VPCSpec defines the desired state of VPC. At least one subnet is required.

Appears in: - VPC

Field Description Default Validation subnets object (keys:string, values:VPCSubnet) Subnets is the list of VPC subnets to configure ipv4Namespace string IPv4Namespace is the name of the IPv4Namespace this VPC belongs to (if not specified, \"default\" is used) vlanNamespace string VLANNamespace is the name of the VLANNamespace this VPC belongs to (if not specified, \"default\" is used) defaultIsolated boolean DefaultIsolated sets default bahivour for isolated mode for the subnets (disabled by default) defaultRestricted boolean DefaultRestricted sets default bahivour for restricted mode for the subnets (disabled by default) permit string array array Permit defines a list of the access policies between the subnets within the VPC - each policy is a list of subnets that have access to each other.It's applied on top of the subnet isolation flag and if subnet isn't isolated it's not required to have it in a permit list while if vpc is markedas isolated it's required to have it in a permit list to have access to other subnets. staticRoutes VPCStaticRoute array StaticRoutes is the list of additional static routes for the VPC"},{"location":"reference/api/#vpcstaticroute","title":"VPCStaticRoute","text":"

VPCStaticRoute defines the static route for the VPC

Appears in: - VPCSpec

Field Description Default Validation prefix string Prefix for the static route (mandatory), e.g. 10.42.0.0/24 nextHops string array NextHops for the static route (at least one is required), e.g. 10.99.0.0"},{"location":"reference/api/#vpcstatus","title":"VPCStatus","text":"

VPCStatus defines the observed state of VPC

Appears in: - VPC

"},{"location":"reference/api/#vpcsubnet","title":"VPCSubnet","text":"

VPCSubnet defines the VPC subnet configuration

Appears in: - VPCSpec

Field Description Default Validation subnet string Subnet is the subnet CIDR block, such as \"10.0.0.0/24\", should belong to the IPv4Namespace and be unique within the namespace dhcp VPCDHCP DHCP is the on-demand DHCP configuration for the subnet vlan string VLAN is the VLAN ID for the subnet, should belong to the VLANNamespace and be unique within the namespace isolated boolean Isolated is the flag to enable isolated mode for the subnet which means no access to and from the other subnets within the VPC restricted boolean Restricted is the flag to enable restricted mode for the subnet which means no access between hosts within the subnet itself"},{"location":"reference/api/#wiringgithedgehogcomv1alpha2","title":"wiring.githedgehog.com/v1alpha2","text":"

Package v1alpha2 contains API Schema definitions for the wiring v1alpha2 API group. It is public API group mainly for the underlay definition including Switches, Server, wiring between them and etc. Intended to be used by the user.

"},{"location":"reference/api/#resource-types_3","title":"Resource Types","text":""},{"location":"reference/api/#baseportname","title":"BasePortName","text":"

BasePortName defines the full name of the switch port

Appears in: - ConnExternalLink - ConnFabricLinkSwitch - ConnMgmtLinkServer - ConnMgmtLinkSwitch - ConnStaticExternalLinkSwitch - ServerToSwitchLink - SwitchToSwitchLink

Field Description Default Validation port string Port defines the full name of the switch port in the format of \"device/port\", such as \"spine-1/Ethernet1\".SONiC port name is used as a port name and switch name should be same as the name of the Switch object."},{"location":"reference/api/#connbundled","title":"ConnBundled","text":"

ConnBundled defines the bundled connection (port channel, single server to a single switch with multiple links)

Appears in: - ConnectionSpec

Field Description Default Validation links ServerToSwitchLink array Links is the list of server-to-switch links mtu integer MTU is the MTU to be configured on the switch port or port channel"},{"location":"reference/api/#conneslag","title":"ConnESLAG","text":"

ConnESLAG defines the ESLAG connection (port channel, single server to 2-4 switches with multiple links)

Appears in: - ConnectionSpec

Field Description Default Validation links ServerToSwitchLink array Links is the list of server-to-switch links MinItems: 2 mtu integer MTU is the MTU to be configured on the switch port or port channel fallback boolean Fallback is the optional flag that used to indicate one of the links in LACP port channel to be used as a fallback link"},{"location":"reference/api/#connexternal","title":"ConnExternal","text":"

ConnExternal defines the external connection (single switch to a single external device with a single link)

Appears in: - ConnectionSpec

Field Description Default Validation link ConnExternalLink Link is the external connection link"},{"location":"reference/api/#connexternallink","title":"ConnExternalLink","text":"

ConnExternalLink defines the external connection link

Appears in: - ConnExternal

Field Description Default Validation switch BasePortName"},{"location":"reference/api/#connfabric","title":"ConnFabric","text":"

ConnFabric defines the fabric connection (single spine to a single leaf with at least one link)

Appears in: - ConnectionSpec

Field Description Default Validation links FabricLink array Links is the list of spine-to-leaf links MinItems: 1"},{"location":"reference/api/#connfabriclinkswitch","title":"ConnFabricLinkSwitch","text":"

ConnFabricLinkSwitch defines the switch side of the fabric link

Appears in: - FabricLink

Field Description Default Validation port string Port defines the full name of the switch port in the format of \"device/port\", such as \"spine-1/Ethernet1\".SONiC port name is used as a port name and switch name should be same as the name of the Switch object. ip string IP is the IP address of the switch side of the fabric link (switch port configuration) Pattern: ^((25[0-5]|(2[0-4]|1\\d|[1-9]|)\\d)\\.?\\b){4}/([1-2]?[0-9]|3[0-2])$"},{"location":"reference/api/#connmclag","title":"ConnMCLAG","text":"

ConnMCLAG defines the MCLAG connection (port channel, single server to pair of switches with multiple links)

Appears in: - ConnectionSpec

Field Description Default Validation links ServerToSwitchLink array Links is the list of server-to-switch links MinItems: 2 mtu integer MTU is the MTU to be configured on the switch port or port channel fallback boolean Fallback is the optional flag that used to indicate one of the links in LACP port channel to be used as a fallback link"},{"location":"reference/api/#connmclagdomain","title":"ConnMCLAGDomain","text":"

ConnMCLAGDomain defines the MCLAG domain connection which makes two switches into a single logical switch or redundancy group and allows to use MCLAG connections to connect servers in a multi-homed way.

Appears in: - ConnectionSpec

Field Description Default Validation peerLinks SwitchToSwitchLink array PeerLinks is the list of peer links between the switches, used to pass server traffic between switch MinItems: 1 sessionLinks SwitchToSwitchLink array SessionLinks is the list of session links between the switches, used only to pass MCLAG control plane and BGPtraffic between switches MinItems: 1"},{"location":"reference/api/#connmgmt","title":"ConnMgmt","text":"

ConnMgmt defines the management connection (single control node/server to a single switch with a single link)

Appears in: - ConnectionSpec

Field Description Default Validation link ConnMgmtLink"},{"location":"reference/api/#connmgmtlink","title":"ConnMgmtLink","text":"

ConnMgmtLink defines the management connection link

Appears in: - ConnMgmt

Field Description Default Validation server ConnMgmtLinkServer Server is the server side of the management link switch ConnMgmtLinkSwitch Switch is the switch side of the management link"},{"location":"reference/api/#connmgmtlinkserver","title":"ConnMgmtLinkServer","text":"

ConnMgmtLinkServer defines the server side of the management link

Appears in: - ConnMgmtLink

Field Description Default Validation port string Port defines the full name of the switch port in the format of \"device/port\", such as \"spine-1/Ethernet1\".SONiC port name is used as a port name and switch name should be same as the name of the Switch object. ip string IP is the IP address of the server side of the management link (control node port configuration) Pattern: ^((25[0-5]|(2[0-4]|1\\d|[1-9]|)\\d)\\.?\\b){4}/([1-2]?[0-9]|3[0-2])$ mac string MAC is an optional MAC address of the control node port for the management link, if specified will be used tocreate a \"virtual\" link with the connection names on the control node"},{"location":"reference/api/#connmgmtlinkswitch","title":"ConnMgmtLinkSwitch","text":"

ConnMgmtLinkSwitch defines the switch side of the management link

Appears in: - ConnMgmtLink

Field Description Default Validation port string Port defines the full name of the switch port in the format of \"device/port\", such as \"spine-1/Ethernet1\".SONiC port name is used as a port name and switch name should be same as the name of the Switch object. ip string IP is the IP address of the switch side of the management link (switch port configuration) Pattern: ^((25[0-5]|(2[0-4]|1\\d|[1-9]|)\\d)\\.?\\b){4}/([1-2]?[0-9]|3[0-2])$ oniePortName string ONIEPortName is an optional ONIE port name of the switch side of the management link that's only used by the IPv6 Link Local discovery"},{"location":"reference/api/#connstaticexternal","title":"ConnStaticExternal","text":"

ConnStaticExternal defines the static external connection (single switch to a single external device with a single link)

Appears in: - ConnectionSpec

Field Description Default Validation link ConnStaticExternalLink Link is the static external connection link withinVPC string WithinVPC is the optional VPC name to provision the static external connection within the VPC VRF instead of default one to make resource available to the specific VPC"},{"location":"reference/api/#connstaticexternallink","title":"ConnStaticExternalLink","text":"

ConnStaticExternalLink defines the static external connection link

Appears in: - ConnStaticExternal

Field Description Default Validation switch ConnStaticExternalLinkSwitch Switch is the switch side of the static external connection link"},{"location":"reference/api/#connstaticexternallinkswitch","title":"ConnStaticExternalLinkSwitch","text":"

ConnStaticExternalLinkSwitch defines the switch side of the static external connection link

Appears in: - ConnStaticExternalLink

Field Description Default Validation port string Port defines the full name of the switch port in the format of \"device/port\", such as \"spine-1/Ethernet1\".SONiC port name is used as a port name and switch name should be same as the name of the Switch object. ip string IP is the IP address of the switch side of the static external connection link (switch port configuration) Pattern: ^((25[0-5]|(2[0-4]|1\\d|[1-9]|)\\d)\\.?\\b){4}/([1-2]?[0-9]|3[0-2])$ nextHop string NextHop is the next hop IP address for static routes that will be created for the subnets Pattern: ^((25[0-5]|(2[0-4]|1\\d|[1-9]|)\\d)\\.?\\b){4}$ subnets string array Subnets is the list of subnets that will get static routes using the specified next hop vlan integer VLAN is the optional VLAN ID to be configured on the switch port"},{"location":"reference/api/#connunbundled","title":"ConnUnbundled","text":"

ConnUnbundled defines the unbundled connection (no port channel, single server to a single switch with a single link)

Appears in: - ConnectionSpec

Field Description Default Validation link ServerToSwitchLink Link is the server-to-switch link mtu integer MTU is the MTU to be configured on the switch port or port channel"},{"location":"reference/api/#connvpcloopback","title":"ConnVPCLoopback","text":"

ConnVPCLoopback defines the VPC loopback connection (multiple port pairs on a single switch) that enables automated workaround named \"VPC Loopback\" that allow to avoid switch hardware limitations and traffic going through CPU in some cases

Appears in: - ConnectionSpec

Field Description Default Validation links SwitchToSwitchLink array Links is the list of VPC loopback links MinItems: 1"},{"location":"reference/api/#connection","title":"Connection","text":"

Connection object represents a logical and physical connections between any devices in the Fabric (Switch, Server and External objects). It's needed to define all physical and logical connections between the devices in the Wiring Diagram. Connection type is defined by the top-level field in the ConnectionSpec. Exactly one of them could be used in a single Connection object.

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string Connection metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec ConnectionSpec Spec is the desired state of the Connection status ConnectionStatus Status is the observed state of the Connection"},{"location":"reference/api/#connectionspec","title":"ConnectionSpec","text":"

ConnectionSpec defines the desired state of Connection

Appears in: - Connection

Field Description Default Validation unbundled ConnUnbundled Unbundled defines the unbundled connection (no port channel, single server to a single switch with a single link) bundled ConnBundled Bundled defines the bundled connection (port channel, single server to a single switch with multiple links) management ConnMgmt Management defines the management connection (single control node/server to a single switch with a single link) mclag ConnMCLAG MCLAG defines the MCLAG connection (port channel, single server to pair of switches with multiple links) eslag ConnESLAG ESLAG defines the ESLAG connection (port channel, single server to 2-4 switches with multiple links) mclagDomain ConnMCLAGDomain MCLAGDomain defines the MCLAG domain connection which makes two switches into a single logical switch for server multi-homing fabric ConnFabric Fabric defines the fabric connection (single spine to a single leaf with at least one link) vpcLoopback ConnVPCLoopback VPCLoopback defines the VPC loopback connection (multiple port pairs on a single switch) for automated workaround external ConnExternal External defines the external connection (single switch to a single external device with a single link) staticExternal ConnStaticExternal StaticExternal defines the static external connection (single switch to a single external device with a single link)"},{"location":"reference/api/#connectionstatus","title":"ConnectionStatus","text":"

ConnectionStatus defines the observed state of Connection

Appears in: - Connection

"},{"location":"reference/api/#fabriclink","title":"FabricLink","text":"

FabricLink defines the fabric connection link

Appears in: - ConnFabric

Field Description Default Validation spine ConnFabricLinkSwitch Spine is the spine side of the fabric link leaf ConnFabricLinkSwitch Leaf is the leaf side of the fabric link"},{"location":"reference/api/#location","title":"Location","text":"

Location defines the geographical position of the device in a datacenter

Appears in: - SwitchSpec

Field Description Default Validation location string aisle string row string rack string slot string"},{"location":"reference/api/#locationsig","title":"LocationSig","text":"

LocationSig contains signatures for the location UUID as well as the device location itself

Appears in: - SwitchSpec

Field Description Default Validation sig string uuidSig string"},{"location":"reference/api/#rack","title":"Rack","text":"

Rack is the Schema for the racks API

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string Rack metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec RackSpec status RackStatus"},{"location":"reference/api/#rackposition","title":"RackPosition","text":"

RackPosition defines the geographical position of the rack in a datacenter

Appears in: - RackSpec

Field Description Default Validation location string aisle string row string"},{"location":"reference/api/#rackspec","title":"RackSpec","text":"

RackSpec defines the properties of a rack which we are modelling

Appears in: - Rack

Field Description Default Validation position RackPosition"},{"location":"reference/api/#rackstatus","title":"RackStatus","text":"

RackStatus defines the observed state of Rack

Appears in: - Rack

"},{"location":"reference/api/#server","title":"Server","text":"

Server is the Schema for the servers API

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string Server metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec ServerSpec Spec is desired state of the server status ServerStatus Status is the observed state of the server"},{"location":"reference/api/#serverfacingconnectionconfig","title":"ServerFacingConnectionConfig","text":"

ServerFacingConnectionConfig defines any server-facing connection (unbundled, bundled, mclag, etc.) configuration

Appears in: - ConnBundled - ConnESLAG - ConnMCLAG - ConnUnbundled

Field Description Default Validation mtu integer MTU is the MTU to be configured on the switch port or port channel"},{"location":"reference/api/#serverspec","title":"ServerSpec","text":"

ServerSpec defines the desired state of Server

Appears in: - Server

Field Description Default Validation type ServerType Type is the type of server, could be control for control nodes or default (empty string) for everything else Enum: [control] description string Description is a description of the server profile string Profile is the profile of the server, name of the ServerProfile object to be used for this server, currently not used by the Fabric"},{"location":"reference/api/#serverstatus","title":"ServerStatus","text":"

ServerStatus defines the observed state of Server

Appears in: - Server

"},{"location":"reference/api/#servertoswitchlink","title":"ServerToSwitchLink","text":"

ServerToSwitchLink defines the server-to-switch link

Appears in: - ConnBundled - ConnESLAG - ConnMCLAG - ConnUnbundled

Field Description Default Validation server BasePortName Server is the server side of the connection switch BasePortName Switch is the switch side of the connection"},{"location":"reference/api/#servertype","title":"ServerType","text":"

Underlying type: string

ServerType is the type of server, could be control for control nodes or default (empty string) for everything else

Validation: - Enum: [control]

Appears in: - ServerSpec

"},{"location":"reference/api/#switch","title":"Switch","text":"

Switch is the Schema for the switches API

All switches should always have 1 labels defined: wiring.githedgehog.com/rack. It represents name of the rack it belongs to.

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string Switch metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec SwitchSpec Spec is desired state of the switch status SwitchStatus Status is the observed state of the switch"},{"location":"reference/api/#switchgroup","title":"SwitchGroup","text":"

SwitchGroup is the marker API object to group switches together, switch can belong to multiple groups

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string SwitchGroup metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec SwitchGroupSpec Spec is the desired state of the SwitchGroup status SwitchGroupStatus Status is the observed state of the SwitchGroup"},{"location":"reference/api/#switchgroupspec","title":"SwitchGroupSpec","text":"

SwitchGroupSpec defines the desired state of SwitchGroup

Appears in: - SwitchGroup

"},{"location":"reference/api/#switchgroupstatus","title":"SwitchGroupStatus","text":"

SwitchGroupStatus defines the observed state of SwitchGroup

Appears in: - SwitchGroup

"},{"location":"reference/api/#switchredundancy","title":"SwitchRedundancy","text":"

SwitchRedundancy is the switch redundancy configuration which includes name of the redundancy group switch belongs to and its type, used both for MCLAG and ESLAG connections. It defines how redundancy will be configured and handled on the switch as well as which connection types will be available. If not specified, switch will not be part of any redundancy group. If name isn't empty, type must be specified as well and name should be the same as one of the SwitchGroup objects.

Appears in: - SwitchSpec

Field Description Default Validation group string Group is the name of the redundancy group switch belongs to type RedundancyType Type is the type of the redundancy group, could be mclag or eslag"},{"location":"reference/api/#switchrole","title":"SwitchRole","text":"

Underlying type: string

SwitchRole is the role of the switch, could be spine, server-leaf or border-leaf or mixed-leaf

Validation: - Enum: [spine server-leaf border-leaf mixed-leaf virtual-edge]

Appears in: - SwitchSpec

"},{"location":"reference/api/#switchspec","title":"SwitchSpec","text":"

SwitchSpec defines the desired state of Switch

Appears in: - Switch

Field Description Default Validation role SwitchRole Role is the role of the switch, could be spine, server-leaf or border-leaf or mixed-leaf Enum: [spine server-leaf border-leaf mixed-leaf virtual-edge] Required: {} description string Description is a description of the switch profile string Profile is the profile of the switch, name of the SwitchProfile object to be used for this switch, currently not used by the Fabric location Location Location is the location of the switch, it is used to generate the location UUID and location signature locationSig LocationSig LocationSig is the location signature for the switch groups string array Groups is a list of switch groups the switch belongs to redundancy SwitchRedundancy Redundancy is the switch redundancy configuration including name of the redundancy group switch belongs to and its type, used both for MCLAG and ESLAG connections vlanNamespaces string array VLANNamespaces is a list of VLAN namespaces the switch is part of, their VLAN ranges could not overlap asn integer ASN is the ASN of the switch ip string IP is the IP of the switch that could be used to access it from other switches and control nodes in the Fabric vtepIP string VTEPIP is the VTEP IP of the switch protocolIP string ProtocolIP is used as BGP Router ID for switch configuration portGroupSpeeds object (keys:string, values:string) PortGroupSpeeds is a map of port group speeds, key is the port group name, value is the speed, such as '\"2\": 10G' portSpeeds object (keys:string, values:string) PortSpeeds is a map of port speeds, key is the port name, value is the speed portBreakouts object (keys:string, values:string) PortBreakouts is a map of port breakouts, key is the port name, value is the breakout configuration, such as \"1/55: 4x25G\""},{"location":"reference/api/#switchstatus","title":"SwitchStatus","text":"

SwitchStatus defines the observed state of Switch

Appears in: - Switch

"},{"location":"reference/api/#switchtoswitchlink","title":"SwitchToSwitchLink","text":"

SwitchToSwitchLink defines the switch-to-switch link

Appears in: - ConnMCLAGDomain - ConnVPCLoopback

Field Description Default Validation switch1 BasePortName Switch1 is the first switch side of the connection switch2 BasePortName Switch2 is the second switch side of the connection"},{"location":"reference/api/#vlannamespace","title":"VLANNamespace","text":"

VLANNamespace is the Schema for the vlannamespaces API

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string VLANNamespace metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec VLANNamespaceSpec Spec is the desired state of the VLANNamespace status VLANNamespaceStatus Status is the observed state of the VLANNamespace"},{"location":"reference/api/#vlannamespacespec","title":"VLANNamespaceSpec","text":"

VLANNamespaceSpec defines the desired state of VLANNamespace

Appears in: - VLANNamespace

Field Description Default Validation ranges VLANRange array Ranges is a list of VLAN ranges to be used in this namespace, couldn't overlap between each other and with Fabric reserved VLAN ranges MaxItems: 20 MinItems: 1"},{"location":"reference/api/#vlannamespacestatus","title":"VLANNamespaceStatus","text":"

VLANNamespaceStatus defines the observed state of VLANNamespace

Appears in: - VLANNamespace

"},{"location":"reference/cli/","title":"Fabric CLI","text":"

Under construction.

Currently Fabric CLI is represented by a kubectl plugin kubectl-fabric automatically installed on the Control Node. It is a wrapper around kubectl and Kubernetes client which allows to manage Fabric resources in a more convenient way. Fabric CLI only provides a subset of the functionality available via Fabric API and is focused on simplifying objects creation and some manipulation with the already existing objects while main get/list/update operations are expected to be done using kubectl.

core@control-1 ~ $ kubectl fabric\nNAME:\n   hhfctl - Hedgehog Fabric user client\n\nUSAGE:\n   hhfctl [global options] command [command options] [arguments...]\n\nVERSION:\n   v0.23.0\n\nCOMMANDS:\n   vpc                VPC commands\n   switch, sw, agent  Switch/Agent commands\n   connection, conn   Connection commands\n   switchgroup, sg    SwitchGroup commands\n   external           External commands\n   help, h            Shows a list of commands or help for one command\n\nGLOBAL OPTIONS:\n   --verbose, -v  verbose output (includes debug) (default: true)\n   --help, -h     show help\n   --version, -V  print the version\n
"},{"location":"reference/cli/#vpc","title":"VPC","text":"

Create VPC named vpc-1 with subnet 10.0.1.0/24 and VLAN 1001 with DHCP enabled and DHCP range starting from 10.0.1.10 (optional):

core@control-1 ~ $ kubectl fabric vpc create --name vpc-1 --subnet 10.0.1.0/24 --vlan 1001 --dhcp --dhcp-start 10.0.1.10\n

Attach previously created VPC to the server server-01 (which is connected to the Fabric using the server-01--mclag--leaf-01--leaf-02 Connection):

core@control-1 ~ $ kubectl fabric vpc attach --vpc-subnet vpc-1/default --connection server-01--mclag--leaf-01--leaf-02\n

To peer VPC with another VPC (e.g. vpc-2) use the following command:

core@control-1 ~ $ kubectl fabric vpc peer --vpc vpc-1 --vpc vpc-2\n
"},{"location":"release-notes/","title":"Release notes","text":""},{"location":"release-notes/#alpha-6","title":"Alpha-6","text":""},{"location":"release-notes/#observability","title":"Observability","text":""},{"location":"release-notes/#telemetry-prometheus-exporter","title":"Telemetry - Prometheus Exporter","text":""},{"location":"release-notes/#logging","title":"Logging","text":""},{"location":"release-notes/#agent-status-api-enhancements","title":"Agent Status API Enhancements","text":""},{"location":"release-notes/#networking-enhancements","title":"Networking enhancements","text":""},{"location":"release-notes/#other-improvements","title":"Other improvements","text":""},{"location":"release-notes/#bugs-fixed","title":"Bugs fixed","text":""},{"location":"release-notes/#alpha-5","title":"Alpha-5","text":""},{"location":"release-notes/#open-source","title":"Open Source","text":""},{"location":"release-notes/#dhcppxe-boot-support-for-multi-homed-connections","title":"DHCP/PXE boot support for multi-homed connections","text":""},{"location":"release-notes/#improvements","title":"Improvements","text":""},{"location":"release-notes/#alpha-4","title":"Alpha-4","text":""},{"location":"release-notes/#documentation","title":"Documentation","text":""},{"location":"release-notes/#host-connectivity-dual-homing-improvements","title":"Host connectivity dual homing improvements","text":""},{"location":"release-notes/#improved-vpc-security-policy-better-zero-trust","title":"Improved VPC security policy - better Zero Trust","text":""},{"location":"release-notes/#static-external-connection","title":"Static External Connection","text":""},{"location":"release-notes/#internal-improvements","title":"Internal Improvements","text":""},{"location":"release-notes/#known-issues","title":"Known Issues","text":""},{"location":"release-notes/#alpha-3","title":"Alpha-3","text":""},{"location":"release-notes/#sonic-support","title":"SONiC support","text":""},{"location":"release-notes/#multiple-ipv4-namespaces","title":"Multiple IPv4 namespaces","text":""},{"location":"release-notes/#hedgehog-fabric-dhcp-and-ipam-service","title":"Hedgehog Fabric DHCP and IPAM Service","text":""},{"location":"release-notes/#hedgehog-fabric-ntp-service","title":"Hedgehog Fabric NTP Service","text":""},{"location":"release-notes/#staticexternal-connections","title":"StaticExternal connections","text":""},{"location":"release-notes/#dhcp-relay-to-3rd-party-dhcp-service","title":"DHCP Relay to 3rd party DHCP service","text":"

Support for 3rd party DHCP server (DHCP Relay config) through the API

"},{"location":"release-notes/#alpha-2","title":"Alpha-2","text":""},{"location":"release-notes/#controller","title":"Controller","text":"

A single controller. No controller redundancy.

"},{"location":"release-notes/#controller-connectivity","title":"Controller connectivity","text":"

For CLOS/LEAF-SPINE fabrics, it is recommended that the controller connects to one or more leaf switches in the fabric on front-facing data ports. Connection to two or more leaf switches is recommended for redundancy and performance. No port break-out functionality is supported for controller connectivity.

Spine controller connectivity is not supported.

For Collapsed Core topology, the controller can connect on front-facing data ports, as described above, or on management ports. Note that every switch in the collapsed core topology must be connected to the controller.

Management port connectivity can also be supported for CLOS/LEAF-SPINE topology but requires all switches connected to the controllers via management ports. No chain booting is possible for this configuration.

"},{"location":"release-notes/#controller-requirements","title":"Controller requirements","text":""},{"location":"release-notes/#chain-booting","title":"Chain booting","text":"

Switches not directly connecting to the controllers can chain boot via the data network.

"},{"location":"release-notes/#topology-support","title":"Topology support","text":"

CLOS/LEAF-SPINE and Collapsed Core topologies are supported.

"},{"location":"release-notes/#leaf-roles-for-clos-topology","title":"LEAF Roles for CLOS topology","text":"

server leaf, border leaf, and mixed leaf modes are supported.

"},{"location":"release-notes/#collapsed-core-topology","title":"Collapsed Core Topology","text":"

Two ToR/LEAF switches with MCLAG server connection.

"},{"location":"release-notes/#server-multihoming","title":"Server multihoming","text":"

MCLAG-only.

"},{"location":"release-notes/#device-support","title":"Device support","text":""},{"location":"release-notes/#leafs","title":"LEAFs","text":""},{"location":"release-notes/#spines","title":"SPINEs","text":""},{"location":"release-notes/#underlay-configuration","title":"Underlay configuration:","text":"

Port speed, port group speed, port breakouts are configurable through the API

"},{"location":"release-notes/#vpc-overlay-implementation","title":"VPC (overlay) Implementation","text":"

VXLAN-based BGP eVPN.

"},{"location":"release-notes/#multi-subnet-vpcs","title":"Multi-subnet VPCs","text":"

A VPC consists of subnets, each with a user-specified VLAN for external host/server connectivity.

"},{"location":"release-notes/#multiple-ip-address-namespaces","title":"Multiple IP address namespaces","text":"

Multiple IP address namespaces are supported per fabric. Each VPC belongs to the corresponding IPv4 namespace. There are no subnet overlaps within a single IPv4 namespace. IP address namespaces can mutually overlap.

"},{"location":"release-notes/#vlan-namespace","title":"VLAN Namespace","text":"

VLAN Namespaces guarantee the uniqueness of VLANs for a set of participating devices. Each switch belongs to a list of VLAN namespaces with non-overlapping VLAN ranges. Each VPC belongs to the VLAN namespace. There are no VLAN overlaps within a single VLAN namespace.

This feature is useful when multiple VM-management domains (like separate VMware clusters connect to the fabric).

"},{"location":"release-notes/#switch-groups","title":"Switch Groups","text":"

Each switch belongs to a list of switch groups used for identifying redundancy groups for things like external connectivity.

"},{"location":"release-notes/#mutual-vpc-peering","title":"Mutual VPC Peering","text":"

VPC peering is supported and possible between a pair of VPCs that belong to the same IPv4 and VLAN namespaces.

"},{"location":"release-notes/#external-vpc-peering","title":"External VPC Peering","text":"

VPC peering provides the means of peering with external networking devices (edge routers, firewalls, or data center interconnects). VPC egress/ingress is pinned to a specific group of the border or mixed leaf switches. Multiple \u201cexternal systems\u201d with multiple devices/links in each of them are supported.

The user controls what subnets/prefixes to import and export from/to the external system.

No NAT function is supported for external peering.

"},{"location":"release-notes/#host-connectivity","title":"Host connectivity","text":"

Servers can be attached as Unbundled, Bundled (LAG) and MCLAG

"},{"location":"release-notes/#dhcp-service","title":"DHCP Service","text":"

VPC is provided with an optional DHCP service with simple IPAM

"},{"location":"release-notes/#local-vpc-peering-loopbacks","title":"Local VPC peering loopbacks","text":"

To enable local inter-vpc peering that allows routing of traffic between VPCs, local loopbacks are required to overcome silicon limitations.

"},{"location":"release-notes/#scale","title":"Scale","text":""},{"location":"release-notes/#software-versions","title":"Software versions","text":""},{"location":"release-notes/#known-limitations","title":"Known Limitations","text":""},{"location":"release-notes/#alpha-1","title":"Alpha-1","text":""},{"location":"troubleshooting/overview/","title":"Troubleshooting","text":"

Under construction.

"},{"location":"user-guide/connections/","title":"Connections","text":"

The Connection object represents logical and physical connections between any devices in the Fabric (Switch, Server and External objects). It's needed to define all connections between the devices in the Wiring Diagram.

There are multiple types of connections.

"},{"location":"user-guide/connections/#server-connections-user-facing","title":"Server connections (user-facing)","text":"

Server connections are used to connect workload servers to the switches.

"},{"location":"user-guide/connections/#unbundled","title":"Unbundled","text":"

Unbundled server connections are used to connect servers to a single switch using a single port.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: server-4--unbundled--s5248-02\n  namespace: default\nspec:\n  unbundled:\n    link: # Defines a single link between a server and a switch\n      server:\n        port: server-4/enp2s1\n      switch:\n        port: s5248-02/Ethernet3\n
"},{"location":"user-guide/connections/#bundled","title":"Bundled","text":"

Bundled server connections are used to connect servers to a single switch using multiple ports (port channel, LAG).

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: server-3--bundled--s5248-01\n  namespace: default\nspec:\n  bundled:\n    links: # Defines multiple links between a single server and a single switch\n    - server:\n        port: server-3/enp2s1\n      switch:\n        port: s5248-01/Ethernet3\n    - server:\n        port: server-3/enp2s2\n      switch:\n        port: s5248-01/Ethernet4\n
"},{"location":"user-guide/connections/#mclag","title":"MCLAG","text":"

MCLAG server connections are used to connect servers to a pair of switches using multiple ports (Dual-homing). Switches should be configured as an MCLAG pair which requires them to be in a single redundancy group of type mclag and a Connection with type mclag-domain between them. MCLAG switches should also have the same spec.ASN and spec.VTEPIP.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: server-1--mclag--s5248-01--s5248-02\n  namespace: default\nspec:\n  mclag:\n    links: # Defines multiple links between a single server and a pair of switches\n    - server:\n        port: server-1/enp2s1\n      switch:\n        port: s5248-01/Ethernet1\n    - server:\n        port: server-1/enp2s2\n      switch:\n        port: s5248-02/Ethernet1\n
"},{"location":"user-guide/connections/#eslag","title":"ESLAG","text":"

ESLAG server connections are used to connect servers to the 2-4 switches using multiple ports (Multi-homing). Switches should belong to the same redundancy group with type eslag, but contrary to the MCLAG case, no other configuration is required.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: server-1--eslag--s5248-01--s5248-02\n  namespace: default\nspec:\n  eslag:\n    links: # Defines multiple links between a single server and a 2-4 switches\n    - server:\n        port: server-1/enp2s1\n      switch:\n        port: s5248-01/Ethernet1\n    - server:\n        port: server-1/enp2s2\n      switch:\n        port: s5248-02/Ethernet1\n
"},{"location":"user-guide/connections/#switch-connections-fabric-facing","title":"Switch connections (fabric-facing)","text":"

Switch connections are used to connect switches to each other and provide any needed \"service\" connectivity to implement the Fabric features.

"},{"location":"user-guide/connections/#fabric","title":"Fabric","text":"

A Fabric Connections is used between specific spine and leaf, it covers all actual wires between a single pair.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: s5232-01--fabric--s5248-01\n  namespace: default\nspec:\n  fabric:\n    links: # Defines multiple links between a spine-leaf pair of switches with IP addresses\n    - leaf:\n        ip: 172.30.30.1/31\n        port: s5248-01/Ethernet48\n      spine:\n        ip: 172.30.30.0/31\n        port: s5232-01/Ethernet0\n    - leaf:\n        ip: 172.30.30.3/31\n        port: s5248-01/Ethernet56\n      spine:\n        ip: 172.30.30.2/31\n        port: s5232-01/Ethernet4\n
"},{"location":"user-guide/connections/#mclag-domain","title":"MCLAG-Domain","text":"

MCLAG-Domain connections define a pair of MCLAG switches with Session and Peer link between them. Switches should be configured as an MCLAG, pair which requires them to be in a single redundancy group of type mclag and Connection with type mclag-domain between them. MCLAG switches should also have the same spec.ASN and spec.VTEPIP.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: s5248-01--mclag-domain--s5248-02\n  namespace: default\nspec:\n  mclagDomain:\n    peerLinks: # Defines multiple links between a pair of MCLAG switches for Peer link\n    - switch1:\n        port: s5248-01/Ethernet72\n      switch2:\n        port: s5248-02/Ethernet72\n    - switch1:\n        port: s5248-01/Ethernet73\n      switch2:\n        port: s5248-02/Ethernet73\n    sessionLinks: # Defines multiple links between a pair of MCLAG switches for Session link\n    - switch1:\n        port: s5248-01/Ethernet74\n      switch2:\n        port: s5248-02/Ethernet74\n    - switch1:\n        port: s5248-01/Ethernet75\n      switch2:\n        port: s5248-02/Ethernet75\n
"},{"location":"user-guide/connections/#vpc-loopback","title":"VPC-Loopback","text":"

VPC-Loopback connections are required in order to implement a workaround for the local VPC peering (when both VPC are attached to the same switch), which is caused by a hardware limitation of the currently supported switches.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: s5248-01--vpc-loopback\n  namespace: default\nspec:\n  vpcLoopback:\n    links: # Defines multiple loopbacks on a single switch\n    - switch1:\n        port: s5248-01/Ethernet16\n      switch2:\n        port: s5248-01/Ethernet17\n    - switch1:\n        port: s5248-01/Ethernet18\n      switch2:\n        port: s5248-01/Ethernet19\n
"},{"location":"user-guide/connections/#management","title":"Management","text":"

Management connections define connections to the Control Node.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: control-1--mgmt--s5248-01-front\n  namespace: default\nspec:\n  management:\n    link: # Defines a single link between a control node and a switch\n      server:\n        ip: 172.30.20.0/31\n        port: control-1/enp2s1\n      switch:\n        ip: 172.30.20.1/31\n        port: s5248-01/Ethernet0\n
"},{"location":"user-guide/connections/#connecting-fabric-to-the-outside-world","title":"Connecting Fabric to the outside world","text":"

Connections in this section provide connectivity to the outside world. For example, they can be connections to the Internet, to other networks, or to some other systems such as DHCP, NTP, LMA, or AAA services.

"},{"location":"user-guide/connections/#staticexternal","title":"StaticExternal","text":"

StaticExternal connections provide a simple way to connect things like DHCP servers directly to the Fabric by connecting them to specific switch ports.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: third-party-dhcp-server--static-external--s5248-04\n  namespace: default\nspec:\n  staticExternal:\n    link:\n      switch:\n        port: s5248-04/Ethernet1 # Switch port to use\n        ip: 172.30.50.5/24 # IP address that will be assigned to the switch port\n        vlan: 1005 # Optional VLAN ID to use for the switch port; if 0, no VLAN is configured\n        subnets: # List of subnets to route to the switch port using static routes and next hop\n          - 10.99.0.1/24\n          - 10.199.0.100/32\n        nextHop: 172.30.50.1 # Next hop IP address to use when configuring static routes for the \"subnets\" list\n

Additionally, it's possible to configure StaticExternal within the VPC to provide access to the third-party resources within a specific VPC, with the rest of the YAML configuration remaining unchanged.

...\nspec:\n  staticExternal:\n    withinVPC: vpc-1 # VPC name to attach the static external to\n    link:\n      ...\n
"},{"location":"user-guide/connections/#external","title":"External","text":"

Connection to external systems, such as edge/provider routers using BGP peering and configuring Inbound/Outbound communities as well as granularly controlling what gets advertised and which routes are accepted.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: s5248-03--external--5835\n  namespace: default\nspec:\n  external:\n    link: # Defines a single link between a switch and an external system\n      switch:\n        port: s5248-03/Ethernet3\n
"},{"location":"user-guide/devices/","title":"Switches and Servers","text":"

All devices in the Hedgehog Fabric are divided into two groups: switches and servers, represented by the corresponding Switch and Server objects in the API. These objects are needed to define all participants of the Fabric and their roles in the Wiring Diagram as well as Connections between them.

"},{"location":"user-guide/devices/#switches","title":"Switches","text":"

Switches are the main building blocks of the Fabric. They are represented by Switch objects in the API. These objects consist of basic metadata like name, description, location, role, as well as port group speeds, port breakouts, ASN, IP addresses, and more.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Switch\nmetadata:\n  name: s5248-01\n  namespace: default\nspec:\n  asn: 65101 # ASN of the switch\n  description: leaf-1\n  ip: 172.30.10.100/32 # Switch IP that will be accessible from the Control Node\n  location:\n    location: gen--default--s5248-01\n  locationSig:\n    sig: <undefined>\n    uuidSig: <undefined>\n  portBreakouts: # Configures port breakouts for the switch\n    1/55: 4x25G\n  portGroupSpeeds: # Configures port group speeds for the switch\n    \"1\": 10G\n    \"2\": 10G\n  protocolIP: 172.30.11.100/32 # Used as BGP router ID\n  role: server-leaf # Role of the switch, one of server-leaf, border-leaf and mixed-leaf\n  vlanNamespaces: # Defines which VLANs could be used to attach servers\n  - default\n  vtepIP: 172.30.12.100/32\n  groups: # Defines which groups the switch belongs to\n  - some-group\n  redundancy: # Optional field to define that switch belongs to the redundancy group\n    group: eslag-1 # Name of the redundancy group\n    type: eslag # Type of the redundancy group, one of mclag or eslag\n

The SwitchGroup is just a marker at that point and doesn't have any configuration options.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: SwitchGroup\nmetadata:\n  name: border\n  namespace: default\nspec: {}\n
"},{"location":"user-guide/devices/#redundancy-groups","title":"Redundancy Groups","text":"

Redundancy groups are used to define the redundancy between switches. It's a regular SwitchGroup used by multiple switches and currently it could be MCLAG or ESLAG (EVPN MH / ESI). A switch can only belong to a single redundancy group.

MCLAG is only supported for pair of switches and ESLAG is supported for up to 4 switches. Multiple types of redundancy groups can be used in the fabric simultaneously.

Connections with types mclag and eslag are used to define the servers connections to switches. They are only supported if the switch belongs to a redundancy group with the corresponding type.

In order to define a MCLAG or ESLAG redundancy group, you need to create a SwitchGroup object and assign it to the switches using the redundancy field.

Example of switch configured for ESLAG:

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: SwitchGroup\nmetadata:\n  name: eslag-1\n  namespace: default\nspec: {}\n---\napiVersion: wiring.githedgehog.com/v1alpha2\nkind: Switch\nmetadata:\n  name: s5248-03\n  namespace: default\nspec:\n  ...\n  redundancy:\n    group: eslag-1\n    type: eslag\n  ...\n

And example of switch configured for MCLAG:

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: SwitchGroup\nmetadata:\n  name: mclag-1\n  namespace: default\nspec: {}\n---\napiVersion: wiring.githedgehog.com/v1alpha2\nkind: Switch\nmetadata:\n  name: s5248-01\n  namespace: default\nspec:\n  ...\n  redundancy:\n    group: mclag-1\n    type: mclag\n  ...\n

In case of MCLAG it's required to have a special connection with type mclag-domain that defines the peer and session links between switches. For more details, see Connections.

"},{"location":"user-guide/devices/#servers","title":"Servers","text":"

Servers include both control nodes and user's workload servers.

Control Node:

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Server\nmetadata:\n  name: control-1\n  namespace: default\nspec:\n  type: control # Type of the server, one of control or \"\" (empty) for regular workload server\n

Regular workload server:

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Server\nmetadata:\n  name: server-1\n  namespace: default\nspec:\n  description: MH s5248-01/E1 s5248-02/E1\n
"},{"location":"user-guide/external/","title":"External Peering","text":"

Hedgehog Fabric uses the Border Leaf concept to exchange VPC routes outside the Fabric and provide L3 connectivity. The External Peering feature allows you to set up an external peering endpoint and to enforce several policies between internal and external endpoints.

Note

Hedgehog Fabric does not operate Edge side devices.

"},{"location":"user-guide/external/#overview","title":"Overview","text":"

Traffic exits from the Fabric on Border Leaves that are connected with Edge devices. Border Leaves are suitable to terminate L2VPN connections, to distinguish VPC L3 routable traffic towards Edge devices, and to land VPC servers. Border Leaves (or Borders) can connect to several Edge devices.

Note

External Peering is only available on the switch devices that are capable for sub-interfaces.

"},{"location":"user-guide/external/#connect-border-leaf-to-edge-device","title":"Connect Border Leaf to Edge device","text":"

In order to distinguish VPC traffic, an Edge device should be able to: - Set up BGP IPv4 to advertise and receive routes from the Fabric - Connect to a Fabric Border Leaf over VLAN - Be able to mark egress routes towards the Fabric with BGP Communities - Be able to filter ingress routes from the Fabric by BGP Communities

All other filtering and processing of L3 Routed Fabric traffic should be done on the Edge devices.

"},{"location":"user-guide/external/#control-plane","title":"Control Plane","text":"

Fabric is sharing VPC routes with Edge devices via BGP. Peering is done over VLAN in IPv4 Unicast AFI/SAFI.

"},{"location":"user-guide/external/#data-plane","title":"Data Plane","text":"

VPC L3 routable traffic will be tagged with VLAN and sent to Edge device. Later processing of VPC traffic (NAT, PBR, etc) should happen on Edge devices.

"},{"location":"user-guide/external/#vpc-access-to-edge-device","title":"VPC access to Edge device","text":"

Each VPC within the Fabric can be allowed to access Edge devices. Additional filtering can be applied to the routes that the VPC can export to Edge devices and import from the Edge devices.

"},{"location":"user-guide/external/#api-and-implementation","title":"API and implementation","text":""},{"location":"user-guide/external/#external","title":"External","text":"

General configuration starts with the specification of External objects. Each object of External type can represent a set of Edge devices, or a single BGP instance on Edge device, or any other united Edge entities that can be described with the following configuration:

Each External should be bound to some VPC IP Namespace, otherwise prefixes overlap may happen.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: External\nmetadata:\n  name: default--5835\nspec:\n  ipv4Namespace: # VPC IP Namespace\n  inboundCommunity: # BGP Standard Community of routes from Edge devices\n  outboundCommunity: # BGP Standard Community required to be assigned on prefixes advertised from Fabric\n
"},{"location":"user-guide/external/#connection","title":"Connection","text":"

A Connection of type external is used to identify the switch port on Border leaf that is cabled with an Edge device.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: # specified or generated\nspec:\n  external:\n    link:\n      switch:\n        port: # SwitchName/EthernetXXX\n
"},{"location":"user-guide/external/#external-attachment","title":"External Attachment","text":"

External Attachment defines BGP Peering and traffic connectivity between a Border leaf and External. Attachments are bound to a Connection with type external and they specify an optional vlan that will be used to segregate particular Edge peering.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: ExternalAttachment\nmetadata:\n  name: #\nspec:\n  connection: # Name of the Connection with type external\n  external: # Name of the External to pick config\n  neighbor:\n    asn: # Edge device ASN\n    ip: # IP address of Edge device to peer with\n  switch:\n    ip: # IP address on the Border Leaf to set up BGP peering\n    vlan: # VLAN (optional) ID to tag control and data traffic, use 0 for untagged\n

Several External Attachment can be configured for the same Connection but for different vlan.

"},{"location":"user-guide/external/#external-vpc-peering","title":"External VPC Peering","text":"

To allow a specific VPC to have access to Edge devices, bind the VPC to a specific External object. To do so, define an External Peering object.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: ExternalPeering\nmetadata:\n  name: # Name of ExternalPeering\nspec:\n  permit:\n    external:\n      name: # External Name\n      prefixes: # List of prefixes (routes) to be allowed to pick up from External\n      - # IPv4 prefix\n    vpc:\n      name: # VPC Name\n      subnets: # List of VPC subnets name to be allowed to have access to External (Edge)\n      - # Name of the subnet within VPC\n

Prefixes is the list of subnets to permit from the External to the VPC. It matches any prefix length less than or equal to 32, effectively permitting all prefixes within the specified one. Use 0.0.0.0/0 for any route, including the default route.

This example allows any IPv4 prefix that came from External:

spec:\n  permit:\n    external:\n      name: ###\n      prefixes:\n      - prefix: 0.0.0.0/0 # Any route will be allowed including default route\n

This example allows all prefixes that match the default route, with any prefix length:

spec:\n  permit:\n    external:\n      name: ###\n      prefixes:\n      - prefix: 77.0.0.0/8 # Any route that belongs to the specified prefix is allowed (such as 77.0.0.0/8 or 77.1.2.0/24)\n
"},{"location":"user-guide/external/#examples","title":"Examples","text":"

This example shows how to peer with the External object with name HedgeEdge, given a Fabric VPC with name vpc-1 on the Border Leaf switchBorder that has a cable connecting it to an Edge device on the port Ethernet42. Specifying vpc-1 is required to receive any prefixes advertised from the External.

"},{"location":"user-guide/external/#fabric-api-configuration","title":"Fabric API configuration","text":""},{"location":"user-guide/external/#external_1","title":"External","text":"
# hhfctl external create --name HedgeEdge --ipns default --in 65102:5000 --out 5000:65102\n
apiVersion: vpc.githedgehog.com/v1alpha2\nkind: External\nmetadata:\n  name: HedgeEdge\n  namespace: default\nspec:\n  inboundCommunity: 65102:5000\n  ipv4Namespace: default\n  outboundCommunity: 5000:65102\n
"},{"location":"user-guide/external/#connection_1","title":"Connection","text":"

Connection should be specified in the wiring diagram.

###\n### switchBorder--external--HedgeEdge\n###\napiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: switchBorder--external--HedgeEdge\nspec:\n  external:\n    link:\n      switch:\n        port: switchBorder/Ethernet42\n
"},{"location":"user-guide/external/#externalattachment","title":"ExternalAttachment","text":"

Specified in wiring diagram

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: ExternalAttachment\nmetadata:\n  name: switchBorder--HedgeEdge\nspec:\n  connection: switchBorder--external--HedgeEdge\n  external: HedgeEdge\n  neighbor:\n    asn: 65102\n    ip: 100.100.0.6\n  switch:\n    ip: 100.100.0.1/24\n    vlan: 100\n
"},{"location":"user-guide/external/#externalpeering","title":"ExternalPeering","text":"
apiVersion: vpc.githedgehog.com/v1alpha2\nkind: ExternalPeering\nmetadata:\n  name: vpc-1--HedgeEdge\nspec:\n  permit:\n    external:\n      name: HedgeEdge\n      prefixes:\n      - prefix: 0.0.0.0/0\n    vpc:\n      name: vpc-1\n      subnets:\n      - default\n
"},{"location":"user-guide/external/#example-edge-side-bgp-configuration-based-on-sonic-os","title":"Example Edge side BGP configuration based on SONiC OS","text":"

Warning

Hedgehog does not recommend using the following configuration for production. It is only provided as an example of Edge Peer configuration.

Interface configuration:

interface Ethernet2.100\n encapsulation dot1q vlan-id 100\n description switchBorder--Ethernet42\n no shutdown\n ip vrf forwarding VrfHedge\n ip address 100.100.0.6/24\n

BGP configuration:

!\nrouter bgp 65102 vrf VrfHedge\n log-neighbor-changes\n timers 60 180\n !\n address-family ipv4 unicast\n  maximum-paths 64\n  maximum-paths ibgp 1\n  import vrf VrfPublic\n !\n neighbor 100.100.0.1\n  remote-as 65103\n  !\n  address-family ipv4 unicast\n   activate\n   route-map HedgeIn in\n   route-map HedgeOut out\n   send-community both\n !\n

Route Map configuration:

route-map HedgeIn permit 10\n match community Hedgehog\n!\nroute-map HedgeOut permit 10\n set community 65102:5000\n!\n\nbgp community-list standard HedgeIn permit 5000:65102\n
"},{"location":"user-guide/harvester/","title":"Using VPCs with Harvester","text":"

This section contains an example of how Hedgehog Fabric can be used with Harvester or any hypervisor on the servers connected to Fabric. It assumes that you have already installed Fabric and have some servers running Harvester attached to it.

You need to define a Server object for each server running Harvester and a Connection object for each server connection to the switches.

You can have multiple VPCs created and attached to the Connections to the servers to make them available to the VMs in Harvester or any other hypervisor.

"},{"location":"user-guide/harvester/#configure-harvester","title":"Configure Harvester","text":""},{"location":"user-guide/harvester/#add-a-cluster-network","title":"Add a Cluster Network","text":"

From the \"Cluster Networks/Configs\" side menu, create a new Cluster Network.

Here is a cleaned-up version of what the CRD looks like:

apiVersion: network.harvesterhci.io/v1beta1\nkind: ClusterNetwork\nmetadata:\n  name: testnet\n
"},{"location":"user-guide/harvester/#add-a-network-config","title":"Add a Network Config","text":"

Click \"Create Network Config\". Add your connections and select the bonding type.

The resulting CRD (cleaned up) looks like the following:

apiVersion: network.harvesterhci.io/v1beta1\nkind: VlanConfig\nmetadata:\n  name: testconfig\n  labels:\n    network.harvesterhci.io/clusternetwork: testnet\nspec:\n  clusterNetwork: testnet\n  uplink:\n    bondOptions:\n      miimon: 100\n      mode: 802.3ad\n    linkAttributes:\n      txQLen: -1\n    nics:\n      - enp5s0f0\n      - enp3s0f1\n
"},{"location":"user-guide/harvester/#add-vlan-based-vm-networks","title":"Add VLAN based VM Networks","text":"

Browse over to \"VM Networks\" and add one network for each VLAN you want to support. Assign them to the cluster network.

Here is what the CRDs will look like for both VLANs:

apiVersion: k8s.cni.cncf.io/v1\nkind: NetworkAttachmentDefinition\nmetadata:\n  labels:\n    network.harvesterhci.io/clusternetwork: testnet\n    network.harvesterhci.io/ready: 'true'\n    network.harvesterhci.io/type: L2VlanNetwork\n    network.harvesterhci.io/vlan-id: '1001'\n  name: testnet1001\n  namespace: default\nspec:\n  config: >-\n    {\"cniVersion\":\"0.3.1\",\"name\":\"testnet1001\",\"type\":\"bridge\",\"bridge\":\"testnet-br\",\"promiscMode\":true,\"vlan\":1001,\"ipam\":{}}\n
apiVersion: k8s.cni.cncf.io/v1\nkind: NetworkAttachmentDefinition\nmetadata:\n  name: testnet1000\n  labels:\n    network.harvesterhci.io/clusternetwork: testnet\n    network.harvesterhci.io/ready: 'true'\n    network.harvesterhci.io/type: L2VlanNetwork\n    network.harvesterhci.io/vlan-id: '1000'\n    #  key: string\n  namespace: default\nspec:\n  config: >-\n    {\"cniVersion\":\"0.3.1\",\"name\":\"testnet1000\",\"type\":\"bridge\",\"bridge\":\"testnet-br\",\"promiscMode\":true,\"vlan\":1000,\"ipam\":{}}\n
"},{"location":"user-guide/harvester/#using-the-vpcs","title":"Using the VPCs","text":"

Now you can choose the new VM Networks when creating a VM in Harvester, and have them created as part of the VPC.

"},{"location":"user-guide/overview/","title":"Overview","text":"

This chapter gives an overview of the main features of Hedgehog Fabric and their usage.

"},{"location":"user-guide/shrink-expand/","title":"Fabric Shrink/Expand","text":"

This section provides a brief overview of how to add or remove switches within the fabric using Hedgehog Fabric API, and how to manage connections between them.

Manipulating API objects is done with the assumption that target devices are correctly cabeled and connected.

This article operates terms that can be found in the Hedgehog Concepts, the User Guide documentation, and the Fabric API reference.

"},{"location":"user-guide/shrink-expand/#add-a-switch-to-the-existing-fabric","title":"Add a switch to the existing fabric","text":"

To be added to the Hedgehog Fabric, a switch should have a corresponding Switch object. An example on how to define this object is available in the User Guilde.

Note

If theSwitch will be used in ESLAG or MCLAG groups, appropriate groups should exist. Redundancy groups should be specified in the Switch object before creation.

After the Switch object has been created, you can define and create dedicated device Connections. The types of the connections may differ based on the Switch role given to the device. For more details, refer to Connections section.

Note

If the switch is facing a Control Node Connection on the front-panel port, the switch port should be described in a Management connection.

Note

Switch devices should be booted in ONIE or HONIE installation mode to install SONiC OS and configure the Fabric Agent.

"},{"location":"user-guide/shrink-expand/#remove-a-switch-from-the-existing-fabric","title":"Remove a switch from the existing fabric","text":"

Before you decommission a switch from the Hedgehog Fabric, several preparation steps are necessary.

Warning

Currently the Wiring diagram used for initial deployment is saved in /var/lib/rancher/k3s/server/manifests/hh-wiring.yaml on the Control node. Fabric will sustain objects within the original wiring diagram. In order to remove any object, first remove the dedicated API objects from this file. It is recommended to reapply hh-wiring.yaml after changing its internals.

"},{"location":"user-guide/vpcs/","title":"VPCs and Namespaces","text":""},{"location":"user-guide/vpcs/#vpc","title":"VPC","text":"

A Virtual Private Cloud (VPC) is similar to a public cloud VPC. It provides an isolated private network for the resources with support for multiple subnets, each with user-provided VLANs and on-demand DHCP.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPC\nmetadata:\n  name: vpc-1\n  namespace: default\nspec:\n  ipv4Namespace: default # Limits to which subnets could be used by VPC to guarantee non-overlapping IPv4 ranges\n  vlanNamespace: default # Limits to which switches VPC could be attached to guarantee non-overlapping VLANs\n\n  defaultIsolated: true # Sets default behavior for the current VPC subnets to be isolated\n  defaultRestricted: true # Sets default behavior for the current VPC subnets to be restricted\n\n  subnets:\n    default: # Each subnet is named, \"default\" subnet isn't required, but actively used by CLI\n      dhcp:\n        enable: true # On-demand DHCP server\n        range: # Optionally, start/end range could be specified\n          start: 10.10.1.10\n          end: 10.10.1.99\n          pxeURL: tftp://10.10.10.99/bootfilename # PXEURL (optional) to identify the PXE server to use to boot hosts; HTTP query strings are not supported\n      subnet: 10.10.1.0/24 # User-defined subnet from ipv4 namespace\n      gateway: 10.10.1.1 # User-defined gateway (optional, default is .1)\n      vlan: \"1001\" # User-defined VLAN from VLAN namespace\n      isolated: true # Makes subnet isolated from other subnets within the VPC (doesn't affect VPC peering)\n      restricted: true # Causes all hosts in the subnet to be isolated from each other\n\n    thrird-party-dhcp: # Another subnet\n      dhcp:\n        relay: 10.99.0.100/24 # Use third-party DHCP server (DHCP relay configuration), access to it could be enabled using StaticExternal connection\n      subnet: \"10.10.2.0/24\"\n      vlan: \"1002\"\n\n    another-subnet: # Minimal configuration is just a name, subnet and VLAN\n      subnet: 10.10.100.0/24\n      vlan: \"1100\"\n\n  permit: # Defines which VPCs could communicate to each other, applied on top of subnets \"isolated\" flag (doesn't affect VPC peering)\n    - [subnet-1, subnet-2, subnet-3] # 1, 2 and 3 subnets could communicate to each other\n    - [subnet-4, subnet-5] # Possible to define multiple lists\n\n  staticRoutes: # Optional, static routes to be added to the VPC\n    - prefix: 10.100.0.0/24 # Destination prefix\n      nextHops: # Next hop IP addresses\n        - 10.200.0.0\n
"},{"location":"user-guide/vpcs/#isolated-and-restricted-subnets-permit-lists","title":"Isolated and restricted subnets, permit lists","text":"

Subnets can be isolated and restricted, with the ability to define permit lists to allow communication between specific isolated subnets. The permit list is applied on top of the isolated flag and doesn't affect VPC peering.

Isolated subnet means that the subnet has no connectivity with other subnets within the VPC, but it could still be allowed by permit lists.

Restricted subnet means that all hosts in the subnet are isolated from each other within the subnet.

A Permit list is defined as a list of subnets that could communicate with each other.

"},{"location":"user-guide/vpcs/#third-party-dhcp-server","title":"Third-party DHCP server","text":"

In case you use a third-party DHCP server by configuring spec.subnets.<subnet>.dhcp.relay, additional information is added to the DHCP packet forwarded to the DHCP server to make it possible to identify the VPC and subnet. This information is added under the RelayAgentInfo (option 82) in the DHCP packet. The relay sets two suboptions in the packet:

"},{"location":"user-guide/vpcs/#vpcattachment","title":"VPCAttachment","text":"

A VPCAttachment represents a specific VPC subnet assignment to the Connection object which means a binding between an exact server port and a VPC. It basically leads to the VPC being available on the specific server port(s) on a subnet VLAN.

VPC could be attached to a switch that is part of the VLAN namespace used by the VPC.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPCAttachment\nmetadata:\n  name: vpc-1-server-1--mclag--s5248-01--s5248-02\n  namespace: default\nspec:\n  connection: server-1--mclag--s5248-01--s5248-02 # Connection name representing the server port(s)\n  subnet: vpc-1/default # VPC subnet name\n  nativeVLAN: true # Optional, if true the port will be configured as a native VLAN port (untagged)\n
"},{"location":"user-guide/vpcs/#vpcpeering","title":"VPCPeering","text":"

A VPCPeering enables VPC-to-VPC connectivity. There are two types of VPC peering:

VPC peering is only possible between VPCs attached to the same IPv4 namespace.

"},{"location":"user-guide/vpcs/#local-vpc-peering","title":"Local VPC peering","text":"
apiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPCPeering\nmetadata:\n  name: vpc-1--vpc-2\n  namespace: default\nspec:\n  permit: # Defines a pair of VPCs to peer\n  - vpc-1: {} # Meaning all subnets of two VPCs will be able to communicate with each other\n    vpc-2: {} # See \"Subnet filtering\" for more advanced configuration\n
"},{"location":"user-guide/vpcs/#remote-vpc-peering","title":"Remote VPC peering","text":"
apiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPCPeering\nmetadata:\n  name: vpc-1--vpc-2\n  namespace: default\nspec:\n  permit:\n  - vpc-1: {}\n    vpc-2: {}\n  remote: border # Indicates a switch group to implement the peering on\n
"},{"location":"user-guide/vpcs/#subnet-filtering","title":"Subnet filtering","text":"

It's possible to specify which specific subnets of the peering VPCs could communicate to each other using the permit field.

```yaml\napiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPCPeering\nmetadata:\n  name: vpc-1--vpc-2\n  namespace: default\nspec:\n  permit: # subnet-1 and subnet-2 of vpc-1 could communicate to subnet-3 of vpc-2 as well as subnet-4 of vpc-2 could communicate to subnet-5 and subnet-6 of vpc-2\n  - vpc-1:\n      subnets: [subnet-1, subnet-2]\n    vpc-2:\n      subnets: [subnet-3]\n  - vpc-1:\n      subnets: [subnet-4]\n    vpc-2:\n      subnets: [subnet-5, subnet-6]\n
"},{"location":"user-guide/vpcs/#ipv4namespace","title":"IPv4Namespace","text":"

An IPv4Namespace defines non-overlapping VLAN ranges for attaching servers. Each switch belongs to a list of VLAN namespaces with non-overlapping VLAN ranges.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: IPv4Namespace\nmetadata:\n  name: default\n  namespace: default\nspec:\n  subnets: # List of the subnets that VPCs can pick their subnets from\n  - 10.10.0.0/16\n
"},{"location":"user-guide/vpcs/#vlannamespace","title":"VLANNamespace","text":"

A VLANNamespace defines non-overlapping IPv4 ranges for VPC subnets. Each VPC belongs to a specific IPv4 namespace.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: VLANNamespace\nmetadata:\n  name: default\n  namespace: default\nspec:\n  ranges: # List of VLAN ranges that VPCs can pick their subnet VLANs from\n  - from: 1000\n    to: 2999\n
"},{"location":"vlab/demo/","title":"Demo on VLAB","text":"

The goal of this demo is to show how to use VPCs, attach and peer them and run test connectivity between the servers. Examples are based on the default VLAB topology.

You can find instructions on how to setup VLAB in the Overview and Running VLAB sections.

"},{"location":"vlab/demo/#default-topology","title":"Default topology","text":"

The default topology is Spine-Leaf with 2 spines, 2 MCLAG leaves and 1 non-MCLAG leaf. Optionally, you can choose to run the default Collapsed Core topology using flag --fabric-mode collapsed-core (or -m collapsed-core) which only consists of 2 switches.

For more details on customizing topologies see the Running VLAB section.

In the default topology, the following Control Node and Switch VMs are created:

graph TD\n    CN[Control Node]\n\n    S1[Spine 1]\n    S2[Spine 2]\n\n    L1[MCLAG Leaf 1]\n    L2[MCLAG Leaf 2]\n    L3[Leaf 3]\n\n    CN --> L1\n    CN --> L2\n\n    S1 --> L1\n    S1 --> L2\n    S2 --> L2\n    S2 --> L3

As well as the following test servers:

graph TD\n    L1[MCLAG Leaf 1]\n    L2[MCLAG Leaf 2]\n    L3[Leaf 3]\n    L4[Leaf 4]\n\n    TS1[Test Server 1]\n    TS2[Test Server 2]\n    TS3[Test Server 3]\n    TS4[Test Server 4]\n    TS5[Test Server 5]\n    TS6[Test Server 6]\n\n    TS1 --> L1\n    TS1 --> L2\n\n    TS2 --> L1\n    TS2 --> L2\n\n    TS3 --> L1\n    TS4 --> L2\n\n    TS5 --> L3\n    TS6 --> L4
"},{"location":"vlab/demo/#creating-and-attaching-vpcs","title":"Creating and attaching VPCs","text":"

You can create and attach VPCs to the VMs using the kubectl fabric vpc command on the Control Node or outside of the cluster using the kubeconfig. For example, run the following commands to create 2 VPCs with a single subnet each, a DHCP server enabled with its optional IP address range start defined, and to attach them to some of the test servers:

core@control-1 ~ $ kubectl get conn | grep server\nserver-01--mclag--leaf-01--leaf-02   mclag          5h13m\nserver-02--mclag--leaf-01--leaf-02   mclag          5h13m\nserver-03--unbundled--leaf-01        unbundled      5h13m\nserver-04--bundled--leaf-02          bundled        5h13m\nserver-05--unbundled--leaf-03        unbundled      5h13m\nserver-06--bundled--leaf-03          bundled        5h13m\n\ncore@control-1 ~ $ kubectl fabric vpc create --name vpc-1 --subnet 10.0.1.0/24 --vlan 1001 --dhcp --dhcp-start 10.0.1.10\n06:48:46 INF VPC created name=vpc-1\n\ncore@control-1 ~ $ kubectl fabric vpc create --name vpc-2 --subnet 10.0.2.0/24 --vlan 1002 --dhcp --dhcp-start 10.0.2.10\n06:49:04 INF VPC created name=vpc-2\n\ncore@control-1 ~ $ kubectl fabric vpc attach --vpc-subnet vpc-1/default --connection server-01--mclag--leaf-01--leaf-02\n06:49:24 INF VPCAttachment created name=vpc-1--default--server-01--mclag--leaf-01--leaf-02\n\ncore@control-1 ~ $ kubectl fabric vpc attach --vpc-subnet vpc-2/default --connection server-02--mclag--leaf-01--leaf-02\n06:49:34 INF VPCAttachment created name=vpc-2--default--server-02--mclag--leaf-01--leaf-02\n

The VPC subnet should belong to an IPv4Namespace, the default one in the VLAB is 10.0.0.0/16:

core@control-1 ~ $ kubectl get ipns\nNAME      SUBNETS           AGE\ndefault   [\"10.0.0.0/16\"]   5h14m\n

After you created the VPCs and VPCAttachments, you can check the status of the agents to make sure that the requested configuration was applied to the switches:

core@control-1 ~ $ kubectl get agents\nNAME       ROLE          DESCR           APPLIED   APPLIEDG   CURRENTG   VERSION\nleaf-01    server-leaf   VS-01 MCLAG 1   2m2s      5          5          v0.23.0\nleaf-02    server-leaf   VS-02 MCLAG 1   2m2s      4          4          v0.23.0\nleaf-03    server-leaf   VS-03           112s      5          5          v0.23.0\nspine-01   spine         VS-04           16m       3          3          v0.23.0\nspine-02   spine         VS-05           18m       4          4          v0.23.0\n

In this example, the values in columns APPLIED and APPLIEDG are equal which means that the requested configuration has been applied.

"},{"location":"vlab/demo/#setting-up-networking-on-test-servers","title":"Setting up networking on test servers","text":"

You can use hhfab vlab ssh on the host to SSH into the test servers and configure networking there. For example, for both server-01 (MCLAG attached to both leaf-01 and leaf-02) we need to configure a bond with a VLAN on top of it and for the server-05 (single-homed unbundled attached to leaf-03) we need to configure just a VLAn and they both will get an IP address from the DHCP server. You can use the ip command to configure networking on the servers or use the little helper preinstalled by Fabricator on test servers.

For server-01:

core@server-01 ~ $ hhnet cleanup\ncore@server-01 ~ $ hhnet bond 1001 enp2s1 enp2s2\n10.0.1.10/24\ncore@server-01 ~ $ ip a\n...\n3: enp2s1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000\n    link/ether 06:5a:e8:38:3b:ea brd ff:ff:ff:ff:ff:ff permaddr 0c:20:12:fe:01:01\n4: enp2s2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000\n    link/ether 06:5a:e8:38:3b:ea brd ff:ff:ff:ff:ff:ff permaddr 0c:20:12:fe:01:02\n6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link/ether 06:5a:e8:38:3b:ea brd ff:ff:ff:ff:ff:ff\n    inet6 fe80::45a:e8ff:fe38:3bea/64 scope link\n       valid_lft forever preferred_lft forever\n7: bond0.1001@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link/ether 06:5a:e8:38:3b:ea brd ff:ff:ff:ff:ff:ff\n    inet 10.0.1.10/24 metric 1024 brd 10.0.1.255 scope global dynamic bond0.1001\n       valid_lft 86396sec preferred_lft 86396sec\n    inet6 fe80::45a:e8ff:fe38:3bea/64 scope link\n       valid_lft forever preferred_lft forever\n

And for server-02:

core@server-02 ~ $ hhnet cleanup\ncore@server-02 ~ $ hhnet bond 1002 enp2s1 enp2s2\n10.0.2.10/24\ncore@server-02 ~ $ ip a\n...\n3: enp2s1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000\n    link/ether 5e:10:b1:f7:d0:4c brd ff:ff:ff:ff:ff:ff permaddr 0c:20:12:fe:02:01\n4: enp2s2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000\n    link/ether 5e:10:b1:f7:d0:4c brd ff:ff:ff:ff:ff:ff permaddr 0c:20:12:fe:02:02\n8: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link/ether 5e:10:b1:f7:d0:4c brd ff:ff:ff:ff:ff:ff\n    inet6 fe80::5c10:b1ff:fef7:d04c/64 scope link\n       valid_lft forever preferred_lft forever\n9: bond0.1002@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link/ether 5e:10:b1:f7:d0:4c brd ff:ff:ff:ff:ff:ff\n    inet 10.0.2.10/24 metric 1024 brd 10.0.2.255 scope global dynamic bond0.1002\n       valid_lft 86185sec preferred_lft 86185sec\n    inet6 fe80::5c10:b1ff:fef7:d04c/64 scope link\n       valid_lft forever preferred_lft forever\n
"},{"location":"vlab/demo/#testing-connectivity-before-peering","title":"Testing connectivity before peering","text":"

You can test connectivity between the servers before peering the switches using the ping command:

core@server-01 ~ $ ping 10.0.2.10\nPING 10.0.2.10 (10.0.2.10) 56(84) bytes of data.\nFrom 10.0.1.1 icmp_seq=1 Destination Net Unreachable\nFrom 10.0.1.1 icmp_seq=2 Destination Net Unreachable\nFrom 10.0.1.1 icmp_seq=3 Destination Net Unreachable\n^C\n--- 10.0.2.10 ping statistics ---\n3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2003ms\n
core@server-02 ~ $ ping 10.0.1.10\nPING 10.0.1.10 (10.0.1.10) 56(84) bytes of data.\nFrom 10.0.2.1 icmp_seq=1 Destination Net Unreachable\nFrom 10.0.2.1 icmp_seq=2 Destination Net Unreachable\nFrom 10.0.2.1 icmp_seq=3 Destination Net Unreachable\n^C\n--- 10.0.1.10 ping statistics ---\n3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2004ms\n
"},{"location":"vlab/demo/#peering-vpcs-and-testing-connectivity","title":"Peering VPCs and testing connectivity","text":"

To enable connectivity between the VPCs, peer them using kubectl fabric vpc peer:

core@control-1 ~ $ kubectl fabric vpc peer --vpc vpc-1 --vpc vpc-2\n07:04:58 INF VPCPeering created name=vpc-1--vpc-2\n

Make sure to wait until the peering is applied to the switches using kubectl get agents command. After that, you can test connectivity between the servers again:

core@server-01 ~ $ ping 10.0.2.10\nPING 10.0.2.10 (10.0.2.10) 56(84) bytes of data.\n64 bytes from 10.0.2.10: icmp_seq=1 ttl=62 time=6.25 ms\n64 bytes from 10.0.2.10: icmp_seq=2 ttl=62 time=7.60 ms\n64 bytes from 10.0.2.10: icmp_seq=3 ttl=62 time=8.60 ms\n^C\n--- 10.0.2.10 ping statistics ---\n3 packets transmitted, 3 received, 0% packet loss, time 2004ms\nrtt min/avg/max/mdev = 6.245/7.481/8.601/0.965 ms\n
core@server-02 ~ $ ping 10.0.1.10\nPING 10.0.1.10 (10.0.1.10) 56(84) bytes of data.\n64 bytes from 10.0.1.10: icmp_seq=1 ttl=62 time=5.44 ms\n64 bytes from 10.0.1.10: icmp_seq=2 ttl=62 time=6.66 ms\n64 bytes from 10.0.1.10: icmp_seq=3 ttl=62 time=4.49 ms\n^C\n--- 10.0.1.10 ping statistics ---\n3 packets transmitted, 3 received, 0% packet loss, time 2004ms\nrtt min/avg/max/mdev = 4.489/5.529/6.656/0.886 ms\n

If you delete the VPC peering with kubectl delete applied to the relevant object and wait for the agent to apply the configuration on the switches, you can observe that connectivity is lost again:

core@control-1 ~ $ kubectl delete vpcpeering/vpc-1--vpc-2\nvpcpeering.vpc.githedgehog.com \"vpc-1--vpc-2\" deleted\n
core@server-01 ~ $ ping 10.0.2.10\nPING 10.0.2.10 (10.0.2.10) 56(84) bytes of data.\nFrom 10.0.1.1 icmp_seq=1 Destination Net Unreachable\nFrom 10.0.1.1 icmp_seq=2 Destination Net Unreachable\nFrom 10.0.1.1 icmp_seq=3 Destination Net Unreachable\n^C\n--- 10.0.2.10 ping statistics ---\n3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2004ms\n

You can see duplicate packets in the output of the ping command between some of the servers. This is expected behavior and is caused by the limitations in the VLAB environment.

core@server-01 ~ $ ping 10.0.5.10\nPING 10.0.5.10 (10.0.5.10) 56(84) bytes of data.\n64 bytes from 10.0.5.10: icmp_seq=1 ttl=62 time=9.58 ms\n64 bytes from 10.0.5.10: icmp_seq=1 ttl=62 time=9.58 ms (DUP!)\n64 bytes from 10.0.5.10: icmp_seq=2 ttl=62 time=6.99 ms\n64 bytes from 10.0.5.10: icmp_seq=2 ttl=62 time=6.99 ms (DUP!)\n64 bytes from 10.0.5.10: icmp_seq=3 ttl=62 time=9.59 ms\n64 bytes from 10.0.5.10: icmp_seq=3 ttl=62 time=9.60 ms (DUP!)\n^C\n--- 10.0.5.10 ping statistics ---\n3 packets transmitted, 3 received, +3 duplicates, 0% packet loss, time 2003ms\nrtt min/avg/max/mdev = 6.987/8.720/9.595/1.226 ms\n
"},{"location":"vlab/demo/#using-vpcs-with-overlapping-subnets","title":"Using VPCs with overlapping subnets","text":"

First, create a second IPv4Namespace with the same subnet as the default one:

core@control-1 ~ $ kubectl get ipns\nNAME      SUBNETS           AGE\ndefault   [\"10.0.0.0/16\"]   24m\n\ncore@control-1 ~ $ cat <<EOF > ipns-2.yaml\napiVersion: vpc.githedgehog.com/v1alpha2\nkind: IPv4Namespace\nmetadata:\n  name: ipns-2\n  namespace: default\nspec:\n  subnets:\n  - 10.0.0.0/16\nEOF\n\ncore@control-1 ~ $ kubectl apply -f ipns-2.yaml\nipv4namespace.vpc.githedgehog.com/ipns-2 created\n\ncore@control-1 ~ $ kubectl get ipns\nNAME      SUBNETS           AGE\ndefault   [\"10.0.0.0/16\"]   30m\nipns-2    [\"10.0.0.0/16\"]   8s\n

Let's assume that vpc-1 already exists and is attached to server-01 (see Creating and attaching VPCs). Now we can create vpc-3 with the same subnet as vpc-1 (but in the different IPv4Namespace) and attach it to the server-03:

core@control-1 ~ $ cat <<EOF > vpc-3.yaml\napiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPC\nmetadata:\n  name: vpc-1\n  namespace: default\nspec:\n  ipv4Namespace: ipns-2\n  subnets:\n    default:\n      dhcp:\n        enable: true\n        range:\n          start: 10.0.1.10\n      subnet: 10.0.1.0/24\n      vlan: \"2001\"\n  vlanNamespace: default\nEOF\n\ncore@control-1 ~ $ kubectl apply -f vpc-3.yaml\n

At that point you can setup networking on server-03 the same as you did for server-01 and server-02 in a previous section. Once you have configured networking, server-01 and server-03 have IP addresses from the same subnets.

"},{"location":"vlab/overview/","title":"Overview","text":"

It's possible to run Hedgehog Fabric in a fully virtual environment using QEMU/KVM and SONiC Virtual Switch (VS). It's a great way to try out Fabric and learn about its look and feel, API, and capabilities. It's not suitable for any data plane or performance testing, or for production use.

In the VLAB all switches start as empty VMs with only the ONIE image on them, and they go through the whole discovery, boot and installation process like on real hardware.

"},{"location":"vlab/overview/#overview_1","title":"Overview","text":"

The hhfab CLI provides a special command vlab to manage the virtual labs. It allows you to run sets of virtual machines to simulate the Fabric infrastructure including control node, switches, test servers and it automatically runs the installer to get Fabric up and running.

You can find more information about getting hhfab in the download section.

"},{"location":"vlab/overview/#system-requirements","title":"System Requirements","text":"

Currently, it's only tested on Ubuntu 22.04 LTS, but should work on any Linux distribution with QEMU/KVM support and fairly up-to-date packages.

The following packages needs to be installed: qemu-kvm swtpm-tools tpm2-tools socat. Docker is also required, to login into the OCI registry.

By default, the VLAB topology is Spine-Leaf with 2 spines, 2 MCLAG leaves and 1 non-MCLAG leaf. Optionally, you can choose to run the default Collapsed Core topology using flag --fabric-mode collapsed-core (or -m collapsed-core) which only consists of 2 switches.

You can calculate the system requirements based on the allocated resources to the VMs using the following table:

Device vCPU RAM Disk Control Node 6 6GB 100GB Test Server 2 768MB 10GB Switch 4 5GB 50GB

These numbers give approximately the following requirements for the default topologies:

Usually, none of the VMs will reach 100% utilization of the allocated resources, but as a rule of thumb you should make sure that you have at least allocated RAM and disk space for all VMs.

NVMe SSD for VM disks is highly recommended.

"},{"location":"vlab/overview/#installing-prerequisites","title":"Installing prerequisites","text":"

On Ubuntu 22.04 LTS you can install all required packages using the following commands:

curl -fsSL https://get.docker.com -o install-docker.sh\nsudo sh install-docker.sh\nsudo usermod -aG docker $USER\nnewgrp docker\n
sudo apt install -y qemu-kvm swtpm-tools tpm2-tools socat\nsudo usermod -aG kvm $USER\nnewgrp kvm\nkvm-ok\n

Good output of the kvm-ok command should look like this:

ubuntu@docs:~$ kvm-ok\nINFO: /dev/kvm exists\nKVM acceleration can be used\n
"},{"location":"vlab/overview/#next-steps","title":"Next steps","text":""},{"location":"vlab/running/","title":"Running VLAB","text":"

Make sure to follow the prerequisites and check system requirements in the VLAB Overview section before running VLAB.

"},{"location":"vlab/running/#initialize-vlab","title":"Initialize VLAB","text":"

First, initialize Fabricator for the VLAB by running hhfab init --preset vlab (or -p vlab). This command supports several customization options that are listed in the output of hhfab init --help. To tune the topology used for the VLAB, you can use the --fabric-mode (or -m) flag to choose between spine-leaf (default) and collapsed-core topologies. You can also configure the number of spines, leafs, connections, and so on. For example, flags --spines-count and --mclag-leafs-count allow you to set the number of spines and MCLAG leaves, respectively.

By default, the command creates 2 spines, 2 MCLAG leaves and 1 non-MCLAG leaf with 2 fabric connections (between each spine and leaf), 2 MCLAG peer links and 2 MCLAG session links as well as 2 loopbacks per leaf for implementing VPC Loopback workaround.

ubuntu@docs:~$ hhfab init -p vlab\n01:17:44 INF Generating wiring from gen flags\n01:17:44 INF Building wiring diagram fabricMode=spine-leaf chainControlLink=false controlLinksCount=0\n01:17:44 INF                     >>> spinesCount=2 fabricLinksCount=2\n01:17:44 INF                     >>> mclagLeafsCount=2 orphanLeafsCount=1\n01:17:44 INF                     >>> mclagSessionLinks=2 mclagPeerLinks=2\n01:17:44 INF                     >>> vpcLoopbacks=2\n01:17:44 WRN Wiring is not hydrated, hydrating reason=\"error validating wiring: ASN not set for switch leaf-01\"\n01:17:44 INF Initialized preset=vlab fabricMode=spine-leaf config=.hhfab/config.yaml wiring=.hhfab/wiring.yaml\n

Or if you want to run Collapsed Core topology with 2 MCLAG switches:

ubuntu@docs:~$ hhfab init -p vlab -m collapsed-core\n01:20:07 INF Generating wiring from gen flags\n01:20:07 INF Building wiring diagram fabricMode=collapsed-core chainControlLink=false controlLinksCount=0\n01:20:07 INF                     >>> mclagLeafsCount=2 orphanLeafsCount=0\n01:20:07 INF                     >>> mclagSessionLinks=2 mclagPeerLinks=2\n01:20:07 INF                     >>> vpcLoopbacks=2\n01:20:07 WRN Wiring is not hydrated, hydrating reason=\"error validating wiring: ASN not set for switch leaf-01\"\n01:20:07 INF Initialized preset=vlab fabricMode=collapsed-core config=.hhfab/config.yaml wiring=.hhfab/wiring.yaml\n

Or you can run custom topology with 2 spines, 4 MCLAG leaves and 2 non-MCLAG leaves using flags:

ubuntu@docs:~$ hhfab init -p vlab --mclag-leafs-count 4 --orphan-leafs-count 2\n01:21:53 INF Generating wiring from gen flags\n01:21:53 INF Building wiring diagram fabricMode=spine-leaf chainControlLink=false controlLinksCount=0\n01:21:53 INF                     >>> spinesCount=2 fabricLinksCount=2\n01:21:53 INF                     >>> mclagLeafsCount=4 orphanLeafsCount=2\n01:21:53 INF                     >>> mclagSessionLinks=2 mclagPeerLinks=2\n01:21:53 INF                     >>> vpcLoopbacks=2\n01:21:53 WRN Wiring is not hydrated, hydrating reason=\"error validating wiring: ASN not set for switch leaf-01\"\n01:21:53 INF Initialized preset=vlab fabricMode=spine-leaf config=.hhfab/config.yaml wiring=.hhfab/wiring.yaml\n

Additionally, you can pass extra Fabric configuration items using flags on init command or by passing a configuration file. For more information, refer to the Fabric Configuration section.

Once you have initialized the VLAB, download the artifacts and build the installer using hhfab build. This command automatically downloads all required artifacts from the OCI registry and builds the installer and all other prerequisites for running the VLAB.

"},{"location":"vlab/running/#build-the-installer-and-vlab","title":"Build the installer and VLAB","text":"
ubuntu@docs:~$ hhfab build\n01:23:33 INF Building component=base\n01:23:33 WRN Attention! Development mode enabled - this is not secure! Default users and keys will be created.\n...\n01:23:33 INF Building component=control-os\n01:23:33 INF Building component=k3s\n01:23:33 INF Downloading name=m.l.hhdev.io:31000/githedgehog/k3s:v1.27.4-k3s1 to=.hhfab/control-install\nCopying k3s-airgap-images-amd64.tar.gz  187.36 MiB / 187.36 MiB   \u2819   0.00 b/s done\nCopying k3s                               56.50 MiB / 56.50 MiB   \u2819   0.00 b/s done\n01:23:35 INF Building component=zot\n01:23:35 INF Downloading name=m.l.hhdev.io:31000/githedgehog/zot:v1.4.3 to=.hhfab/control-install\nCopying zot-airgap-images-amd64.tar.gz  19.24 MiB / 19.24 MiB   \u2838   0.00 b/s done\n01:23:35 INF Building component=misc\n01:23:35 INF Downloading name=m.l.hhdev.io:31000/githedgehog/fabricator/k9s:v0.27.4 to=.hhfab/control-install\nCopying k9s  57.75 MiB / 57.75 MiB   \u283c   0.00 b/s done\n...\n01:25:40 INF Planned bundle=control-install name=fabric-api-chart op=\"push fabric/charts/fabric-api:v0.23.0\"\n01:25:40 INF Planned bundle=control-install name=fabric-image op=\"push fabric/fabric:v0.23.0\"\n01:25:40 INF Planned bundle=control-install name=fabric-chart op=\"push fabric/charts/fabric:v0.23.0\"\n01:25:40 INF Planned bundle=control-install name=fabric-agent-seeder op=\"push fabric/agent/x86_64:latest\"\n01:25:40 INF Planned bundle=control-install name=fabric-agent op=\"push fabric/agent:v0.23.0\"\n...\n01:25:40 INF Recipe created bundle=control-install actions=67\n01:25:40 INF Creating recipe bundle=server-install\n01:25:40 INF Planned bundle=server-install name=toolbox op=\"file /opt/hedgehog/toolbox.tar\"\n01:25:40 INF Planned bundle=server-install name=toolbox-load op=\"exec ctr\"\n01:25:40 INF Planned bundle=server-install name=hhnet op=\"file /opt/bin/hhnet\"\n01:25:40 INF Recipe created bundle=server-install actions=3\n01:25:40 INF Building done took=2m6.813384532s\n01:25:40 INF Packing bundle=control-install target=control-install.tgz\n01:25:45 INF Packing bundle=server-install target=server-install.tgz\n01:25:45 INF Packing done took=5.67007384s\n

As soon as the build has completed, you can run the VLAB using hhfab vlab up. This command automatically starts all VMs and runs the installers on the control node and test servers. It takes some time for all VMs to come up and for the installer to finish. You can monitor progress in the output. If you stop the command, it will stop all VMs, and you can re-run it to get VMs back up and running.

"},{"location":"vlab/running/#run-vms-and-installers","title":"Run VMs and installers","text":"
ubuntu@docs:~$ hhfab vlab up\n01:29:13 INF Starting VLAB server... basedir=.hhfab/vlab-vms vm-size=\"\" dry-run=false\n01:29:13 INF VM id=0 name=control-1 type=control\n01:29:13 INF VM id=1 name=server-01 type=server\n01:29:13 INF VM id=2 name=server-02 type=server\n01:29:13 INF VM id=3 name=server-03 type=server\n01:29:13 INF VM id=4 name=server-04 type=server\n01:29:13 INF VM id=5 name=server-05 type=server\n01:29:13 INF VM id=6 name=server-06 type=server\n01:29:13 INF VM id=7 name=leaf-01 type=switch-vs\n01:29:13 INF VM id=8 name=leaf-02 type=switch-vs\n01:29:13 INF VM id=9 name=leaf-03 type=switch-vs\n01:29:13 INF VM id=10 name=spine-01 type=switch-vs\n01:29:13 INF VM id=11 name=spine-02 type=switch-vs\n01:29:13 INF Total VM resources cpu=\"38 vCPUs\" ram=\"36352 MB\" disk=\"410 GB\"\n...\n01:29:13 INF Preparing VM id=0 name=control-1 type=control\n01:29:13 INF Copying files  from=.hhfab/control-os/ignition.json to=.hhfab/vlab-vms/control-1/ignition.json\n01:29:13 INF Copying files  from=.hhfab/vlab-files/flatcar.img to=.hhfab/vlab-vms/control-1/os.img\n 947.56 MiB / 947.56 MiB [==========================================================] 5.13 GiB/s done\n01:29:14 INF Copying files  from=.hhfab/vlab-files/flatcar_efi_code.fd to=.hhfab/vlab-vms/control-1/efi_code.fd\n01:29:14 INF Copying files  from=.hhfab/vlab-files/flatcar_efi_vars.fd to=.hhfab/vlab-vms/control-1/efi_vars.fd\n01:29:14 INF Resizing VM image (may require sudo password) name=control-1\n01:29:17 INF Initializing TPM name=control-1\n...\n01:29:46 INF Installing VM name=control-1 type=control\n01:29:46 INF Installing VM name=server-01 type=server\n01:29:46 INF Installing VM name=server-02 type=server\n01:29:46 INF Installing VM name=server-03 type=server\n01:29:47 INF Installing VM name=server-04 type=server\n01:29:47 INF Installing VM name=server-05 type=server\n01:29:47 INF Installing VM name=server-06 type=server\n01:29:49 INF Running VM id=0 name=control-1 type=control\n01:29:49 INF Running VM id=1 name=server-01 type=server\n01:29:49 INF Running VM id=2 name=server-02 type=server\n01:29:49 INF Running VM id=3 name=server-03 type=server\n01:29:50 INF Running VM id=4 name=server-04 type=server\n01:29:50 INF Running VM id=5 name=server-05 type=server\n01:29:50 INF Running VM id=6 name=server-06 type=server\n01:29:50 INF Running VM id=7 name=leaf-01 type=switch-vs\n01:29:50 INF Running VM id=8 name=leaf-02 type=switch-vs\n01:29:51 INF Running VM id=9 name=leaf-03 type=switch-vs\n01:29:51 INF Running VM id=10 name=spine-01 type=switch-vs\n01:29:51 INF Running VM id=11 name=spine-02 type=switch-vs\n...\n01:30:41 INF VM installed name=server-06 type=server installer=server-install\n01:30:41 INF VM installed name=server-01 type=server installer=server-install\n01:30:41 INF VM installed name=server-02 type=server installer=server-install\n01:30:41 INF VM installed name=server-04 type=server installer=server-install\n01:30:41 INF VM installed name=server-03 type=server installer=server-install\n01:30:41 INF VM installed name=server-05 type=server installer=server-install\n...\n01:31:04 INF Running installer on VM name=control-1 type=control installer=control-install\n...\n01:35:15 INF Done took=3m39.586394608s\n01:35:15 INF VM installed name=control-1 type=control installer=control-install\n

Line VM installed name=control-1 from the installer's output means that the installer has finished. After this line has been displayed, you can get into the control node and other VMs to watch the Fabric coming up and switches getting provisioned.

"},{"location":"vlab/running/#configuring-vlab-vms","title":"Configuring VLAB VMs","text":"

By default, all test server VMs are isolated and have no connectivity to the host or the Internet. You can configure enable connectivity using hhfab vlab up --restrict-servers=false to allow the test servers to access the Internet and the host. When you enable connectivity, VMs get a default route pointing to the host, which means that in case of the VPC peering you need to configure test server VMs to use the VPC attachment as a default route (or just some specific subnets).

Additionally, you can configure the size of all VMs using hhfab vlab up --vm-size <size>. The flag allows you to choose from one of the presets (compact, default, full and huge) to get the control, switch, and server VMs of different sizes.

"},{"location":"vlab/running/#default-credentials","title":"Default credentials","text":"

Fabricator creates default users and keys for you to login into the control node and test servers as well as for the SONiC Virtual Switches.

Default user with passwordless sudo for the control node and test servers is core with password HHFab.Admin!. Admin user with full access and passwordless sudo for the switches is admin with password HHFab.Admin!. Read-only, non-sudo user with access only to the switch CLI for the switches is op with password HHFab.Op!.

"},{"location":"vlab/running/#accessing-the-vlab","title":"Accessing the VLAB","text":"

The hhfab vlab command provides ssh and serial subcommands to access the VMs. You can use ssh to get into the control node and test servers after the VMs are started. You can use serial to get into the switch VMs while they are provisioning and installing the software. After switches are installed you can use ssh to get into them.

You can select device you want to access or pass the name using the --vm flag.

ubuntu@docs:~$ hhfab vlab ssh\nUse the arrow keys to navigate: \u2193 \u2191 \u2192 \u2190  and / toggles search\nSSH to VM:\n  \ud83e\udd94 control-1\n  server-01\n  server-02\n  server-03\n  server-04\n  server-05\n  server-06\n  leaf-01\n  leaf-02\n  leaf-03\n  spine-01\n  spine-02\n\n----------- VM Details ------------\nID:             0\nName:           control-1\nReady:          true\nBasedir:        .hhfab/vlab-vms/control-1\n

On the control node you have access to kubectl, Fabric CLI, and k9s to manage the Fabric. You can find information about the switches provisioning by running kubectl get agents -o wide. It usually takes about 10-15 minutes for the switches to get installed.

After the switches are provisioned, the command returns something like this:

core@control-1 ~ $ kubectl get agents -o wide\nNAME       ROLE          DESCR           HWSKU                      ASIC   HEARTBEAT   APPLIED   APPLIEDG   CURRENTG   VERSION   SOFTWARE                ATTEMPT   ATTEMPTG   AGE\nleaf-01    server-leaf   VS-01 MCLAG 1   DellEMC-S5248f-P-25G-DPB   vs     30s         5m5s      4          4          v0.23.0   4.1.1-Enterprise_Base   5m5s      4          10m\nleaf-02    server-leaf   VS-02 MCLAG 1   DellEMC-S5248f-P-25G-DPB   vs     27s         3m30s     3          3          v0.23.0   4.1.1-Enterprise_Base   3m30s     3          10m\nleaf-03    server-leaf   VS-03           DellEMC-S5248f-P-25G-DPB   vs     18s         3m52s     4          4          v0.23.0   4.1.1-Enterprise_Base   3m52s     4          10m\nspine-01   spine         VS-04           DellEMC-S5248f-P-25G-DPB   vs     26s         3m59s     3          3          v0.23.0   4.1.1-Enterprise_Base   3m59s     3          10m\nspine-02   spine         VS-05           DellEMC-S5248f-P-25G-DPB   vs     19s         3m53s     4          4          v0.23.0   4.1.1-Enterprise_Base   3m53s     4          10m\n

The Heartbeat column shows how long ago the switch has sent the heartbeat to the control node. The Applied column shows how long ago the switch has applied the configuration. AppliedG shows the generation of the configuration applied. CurrentG shows the generation of the configuration the switch is supposed to run. Different values for AppliedG and CurrentG mean that the switch is in the process of applying the configuration.

At that point Fabric is ready and you can use kubectl and kubectl fabric to manage the Fabric. You can find more about managing the Fabric in the Running Demo and User Guide sections.

"},{"location":"vlab/running/#getting-main-fabric-objects","title":"Getting main Fabric objects","text":"

You can list the main Fabric objects by running kubectl get on the control node. You can find more details about using the Fabric in the User Guide, Fabric API and Fabric CLI sections.

For example, to get the list of switches, run:

core@control-1 ~ $ kubectl get switch\nNAME       ROLE          DESCR           GROUPS   LOCATIONUUID                           AGE\nleaf-01    server-leaf   VS-01 MCLAG 1            5e2ae08a-8ba9-599a-ae0f-58c17cbbac67   6h10m\nleaf-02    server-leaf   VS-02 MCLAG 1            5a310b84-153e-5e1c-ae99-dff9bf1bfc91   6h10m\nleaf-03    server-leaf   VS-03                    5f5f4ad5-c300-5ae3-9e47-f7898a087969   6h10m\nspine-01   spine         VS-04                    3e2c4992-a2e4-594b-bbd1-f8b2fd9c13da   6h10m\nspine-02   spine         VS-05                    96fbd4eb-53b5-5a4c-8d6a-bbc27d883030   6h10m\n

Similarly, to get the list of servers, run:

core@control-1 ~ $ kubectl get server\nNAME        TYPE      DESCR                        AGE\ncontrol-1   control   Control node                 6h10m\nserver-01             S-01 MCLAG leaf-01 leaf-02   6h10m\nserver-02             S-02 MCLAG leaf-01 leaf-02   6h10m\nserver-03             S-03 Unbundled leaf-01       6h10m\nserver-04             S-04 Bundled leaf-02         6h10m\nserver-05             S-05 Unbundled leaf-03       6h10m\nserver-06             S-06 Bundled leaf-03         6h10m\n

For connections, use:

core@control-1 ~ $ kubectl get connection\nNAME                                 TYPE           AGE\ncontrol-1--mgmt--leaf-01             management     6h11m\ncontrol-1--mgmt--leaf-02             management     6h11m\ncontrol-1--mgmt--leaf-03             management     6h11m\ncontrol-1--mgmt--spine-01            management     6h11m\ncontrol-1--mgmt--spine-02            management     6h11m\nleaf-01--mclag-domain--leaf-02       mclag-domain   6h11m\nleaf-01--vpc-loopback                vpc-loopback   6h11m\nleaf-02--vpc-loopback                vpc-loopback   6h11m\nleaf-03--vpc-loopback                vpc-loopback   6h11m\nserver-01--mclag--leaf-01--leaf-02   mclag          6h11m\nserver-02--mclag--leaf-01--leaf-02   mclag          6h11m\nserver-03--unbundled--leaf-01        unbundled      6h11m\nserver-04--bundled--leaf-02          bundled        6h11m\nserver-05--unbundled--leaf-03        unbundled      6h11m\nserver-06--bundled--leaf-03          bundled        6h11m\nspine-01--fabric--leaf-01            fabric         6h11m\nspine-01--fabric--leaf-02            fabric         6h11m\nspine-01--fabric--leaf-03            fabric         6h11m\nspine-02--fabric--leaf-01            fabric         6h11m\nspine-02--fabric--leaf-02            fabric         6h11m\nspine-02--fabric--leaf-03            fabric         6h11m\n

For IPv4 and VLAN namespaces, use:

core@control-1 ~ $ kubectl get ipns\nNAME      SUBNETS           AGE\ndefault   [\"10.0.0.0/16\"]   6h12m\n\ncore@control-1 ~ $ kubectl get vlanns\nNAME      AGE\ndefault   6h12m\n
"},{"location":"vlab/running/#reset-vlab","title":"Reset VLAB","text":"

To reset VLAB and start over just remove the .hhfab directory and run hhfab init again.

"},{"location":"vlab/running/#next-steps","title":"Next steps","text":""}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Introduction","text":"

Hedgehog Open Network Fabric is an open networking platform that brings the user experience enjoyed by so many in the public cloud to private environments. It comes without vendor lock-in.

Fabric is built around the concept of VPCs (Virtual Private Clouds), similar to the public clouds. It provides a multi-tenant API to define the user intent on network isolation and connectivity, which gets automatically transformed into configuration for switches and software appliances.

You can read more about its concepts and architecture in the documentation.

You can find how to download and try Fabric on the self-hosted fully virtualized lab or on hardware.

"},{"location":"architecture/fabric/","title":"Hedgehog Network Fabric","text":"

The Hedgehog Open Network Fabric is an open-source network architecture that provides connectivity between virtual and physical workloads and provides a way to achieve network isolation between different groups of workloads using standard BGP EVPN and VXLAN technology. The fabric provides a standard Kubernetes interface to manage the elements in the physical network and provides a mechanism to configure virtual networks and define attachments to these virtual networks. The Hedgehog Fabric provides isolation between different groups of workloads by placing them in different virtual networks called VPC's. To achieve this, it defines different abstractions starting from the physical network where a set of Connection objects defines how a physical server on the network connects to a physical switch on the fabric.

"},{"location":"architecture/fabric/#underlay-network","title":"Underlay Network","text":"

The Hedgehog Fabric currently supports two underlay network topologies.

"},{"location":"architecture/fabric/#collapsed-core","title":"Collapsed Core","text":"

A collapsed core topology is just a pair of switches connected in a MCLAG configuration with no other network elements. All workloads attach to these two switches.

The leaves in this setup are configured to be in a MCLAG pair and servers can either be connected to both switches as a MCLAG port channel or as orphan ports connected to only one switch. Both the leaves peer to external networks using BGP and act as gateway for workloads attached to them. The configuration of the underlay in the collapsed core is very simple and is ideal for very small deployments.

"},{"location":"architecture/fabric/#spine-leaf","title":"Spine-Leaf","text":"

A spine-leaf topology is a standard Clos network with workloads attaching to leaf switches and the spines providing connectivity between different leaves.

This kind of topology is useful for bigger deployments and provides all the advantages of a typical Clos network. The underlay network is established using eBGP where each leaf has a separate ASN and peers will all spines in the network. RFC7938 was used as the reference for establishing the underlay network.

"},{"location":"architecture/fabric/#overlay-network","title":"Overlay Network","text":"

The overlay network runs on top the underlay network to create a virtual network. The overlay network isolates control and data plane traffic between different virtual networks and the underlay network. Visualization is achieved in the Hedgehog Fabric by encapsulating workload traffic over VXLAN tunnels that are source and terminated on the leaf switches in the network. The fabric uses BGP-EVPN/VXLAN to enable the creation and management of virtual networks on top of the physical one. The fabric supports multiple virtual networks over the same underlay network to support multi-tenancy. Each virtual network in the Hedgehog Fabric is identified by a VPC. The following subsections contain a high-level overview of how VPCs are implemented in the Hedgehog Fabric and its associated objects.

"},{"location":"architecture/fabric/#vpc","title":"VPC","text":"

The previous subsections have described what a VPC is, and how to attach workloads to a specific VPC. Here comes a description of how VPCs are actually implemented on the network to provide the view of a private network.

"},{"location":"architecture/fabric/#vpc-peering","title":"VPC Peering","text":"

To enable communication between 2 different VPCs, one needs to configure a VPC peering policy. The Hedgehog Fabric supports two different peering modes.

"},{"location":"architecture/overview/","title":"Overview","text":"

Under construction.

"},{"location":"concepts/overview/","title":"Concepts","text":""},{"location":"concepts/overview/#introduction","title":"Introduction","text":"

Hedgehog Open Network Fabric is built on top of Kubernetes and uses Kubernetes API to manage its resources. It means that all user-facing APIs are Kubernetes Custom Resources (CRDs), so you can use standard Kubernetes tools to manage Fabric resources.

Hedgehog Fabric consists of the following components:

"},{"location":"concepts/overview/#fabric-api","title":"Fabric API","text":"

All infrastructure is represented as a set of Fabric resource (Kubernetes CRDs) and named Wiring Diagram. With this representation, Fabric defines switches, servers, control nodes, external systems and connections between them in a single place and then uses these definitions to deploy and manage the whole infrastructure. On top of the Wiring Diagram, Fabric provides a set of APIs to manage the VPCs and the connections between them and between VPCs and External systems.

"},{"location":"concepts/overview/#wiring-diagram-api","title":"Wiring Diagram API","text":"

Wiring Diagram consists of the following resources:

"},{"location":"concepts/overview/#user-facing-api","title":"User-facing API","text":""},{"location":"concepts/overview/#fabricator","title":"Fabricator","text":"

Installer builder and VLAB.

"},{"location":"concepts/overview/#das-boot","title":"Das Boot","text":"

Switch boot and installation.

"},{"location":"concepts/overview/#fabric","title":"Fabric","text":"

Control plane and switch agent.

"},{"location":"contribute/docs/","title":"Documentation","text":""},{"location":"contribute/docs/#getting-started","title":"Getting started","text":"

This documentation is done using MkDocs with multiple plugins enabled. It's based on the Markdown, you can find basic syntax overview here.

In order to contribute to the documentation, you'll need to have Git and Docker installed on your machine as well as any editor of your choice, preferably supporting Markdown preview. You can run the preview server using following command:

make serve\n

Now you can open continuously updated preview of your edits in browser at http://127.0.0.1:8000. Pages will be automatically updated while you're editing.

Additionally you can run

make build\n

to make sure that your changes will be built correctly and doesn't break documentation.

"},{"location":"contribute/docs/#workflow","title":"Workflow","text":"

If you want to quick edit any page in the documentation, you can press the Edit this page icon at the top right of the page. It'll open the page in the GitHub editor. You can edit it and create a pull request with your changes.

Please, never push to the master or release/* branches directly. Always create a pull request and wait for the review.

Each pull request will be automatically built and preview will be deployed. You can find the link to the preview in the comments in pull request.

"},{"location":"contribute/docs/#repository","title":"Repository","text":"

Documentation is organized in per-release branches:

Latest release branch is referenced as latest version in the documentation and will be used by default when you open the documentation.

"},{"location":"contribute/docs/#file-layout","title":"File layout","text":"

All documentation files are located in docs directory. Each file is a Markdown file with .md extension. You can create subdirectories to organize your files. Each directory can have a .pages file that overrides the default navigation order and titles.

For example, top-level .pages in this repository looks like this:

nav:\n  - index.md\n  - getting-started\n  - concepts\n  - Wiring Diagram: wiring\n  - Install & Upgrade: install-upgrade\n  - User Guide: user-guide\n  - Reference: reference\n  - Troubleshooting: troubleshooting\n  - ...\n  - release-notes\n  - contribute\n

Where you can add pages by file name like index.md and page title will be taked from the file (first line with #). Additionally, you can reference the whole directory to created nested section in navigation. You can also add custom titles by using : separator like Wiring Diagram: wiring where Wiring Diagram is a title and wiring is a file/directory name.

More details in the MkDocs Pages plugin.

"},{"location":"contribute/docs/#abbreaviations","title":"Abbreaviations","text":"

You can find abbreviations in includes/abbreviations.md file. You can add various abbreviations there and all usages of the defined words in the documentation will get a highlight.

For example, we have following in includes/abbreviations.md:

*[HHFab]: Hedgehog Fabricator - a tool for building Hedgehog Fabric\n

It'll highlight all usages of HHFab in the documentation and show a tooltip with the definition like this: HHFab.

"},{"location":"contribute/docs/#markdown-extensions","title":"Markdown extensions","text":"

We're using MkDocs Material theme with multiple extensions enabled. You can find detailed reference here, but here you can find some of the most useful ones.

To view code for examples, please, check the source code of this page.

"},{"location":"contribute/docs/#text-formatting","title":"Text formatting","text":"

Text can be deleted and replacement text added. This can also be combined into onea single operation. Highlighting is also possible and comments can be added inline.

Formatting can also be applied to blocks by putting the opening and closing tags on separate lines and adding new lines between the tags and the content.

Keyboard keys can be written like so:

Ctrl+Alt+Del

Amd inline icons/emojis can be added like this:

:fontawesome-regular-face-laugh-wink:\n:fontawesome-brands-twitter:{ .twitter }\n

"},{"location":"contribute/docs/#admonitions","title":"Admonitions","text":"

Admonitions, also known as call-outs, are an excellent choice for including side content without significantly interrupting the document flow. Different types of admonitions are available, each with a unique icon and color. Details can be found here.

Lorem ipsum

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor massa, nec semper lorem quam in massa.

"},{"location":"contribute/docs/#code-blocks","title":"Code blocks","text":"

Details can be found here.

Simple code block with line nums and highlighted lines:

bubble_sort.py
def bubble_sort(items):\n    for i in range(len(items)):\n        for j in range(len(items) - 1 - i):\n            if items[j] > items[j + 1]:\n                items[j], items[j + 1] = items[j + 1], items[j]\n

Code annotations:

theme:\n  features:\n    - content.code.annotate # (1)\n
  1. I'm a code annotation! I can contain code, formatted text, images, ... basically anything that can be written in Markdown.
"},{"location":"contribute/docs/#tabs","title":"Tabs","text":"

You can use Tabs to better organize content.

CC++ 
#include <stdio.h>\n\nint main(void) {\n  printf(\"Hello world!\\n\");\n  return 0;\n}\n
#include <iostream>\n\nint main(void) {\n  std::cout << \"Hello world!\" << std::endl;\n  return 0;\n}\n
"},{"location":"contribute/docs/#tables","title":"Tables","text":"Method Description GET Fetch resource PUT Update resource DELETE Delete resource"},{"location":"contribute/docs/#diagrams","title":"Diagrams","text":"

You can directly include Mermaid diagrams in your Markdown files. Details can be found here.

graph LR\n  A[Start] --> B{Error?};\n  B -->|Yes| C[Hmm...];\n  C --> D[Debug];\n  D --> B;\n  B ---->|No| E[Yay!];
sequenceDiagram\n  autonumber\n  Alice->>John: Hello John, how are you?\n  loop Healthcheck\n      John->>John: Fight against hypochondria\n  end\n  Note right of John: Rational thoughts!\n  John-->>Alice: Great!\n  John->>Bob: How about you?\n  Bob-->>John: Jolly good!
"},{"location":"contribute/overview/","title":"Overview","text":"

Under construction.

"},{"location":"getting-started/download/","title":"Download","text":""},{"location":"getting-started/download/#getting-access","title":"Getting access","text":"

Prior to General Availability, access to the full software is limited and requires Design Partner Agreement. Please submit a ticket with the request using Hedgehog Support Portal.

After that you will be provided with the credentials to access the software on GitHub Package. In order to use the software, log in to the registry using the following command:

docker login ghcr.io\n
"},{"location":"getting-started/download/#downloading-the-software","title":"Downloading the software","text":"

The main entry point for the software is the Hedgehog Fabricator CLI named hhfab. All software is published into the OCI registry GitHub Package including binaries, container images, or Helm charts. Download the latest stable hhfab binary from the GitHub Package using the following command:

curl -fsSL https://i.hhdev.io/hhfab | bash\n

Or download a specific version using the following command:

curl -fsSL https://i.hhdev.io/hhfab | VERSION=alpha-X bash\n

Use the VERSION environment variable to specify the version of the software to download. By default, the latest release is downloaded. You can pick a specific release series (e.g. alpha-2) or a specific release.

The download script requires ORAS to be installed. ORAS is used to download the binary from the OCI registry and can be installed using following command:

curl -fsSL https://i.hhdev.io/oras | bash\n

Currently only Linux x86 is supported for running hhfab.

"},{"location":"getting-started/download/#next-steps","title":"Next steps","text":""},{"location":"install-upgrade/build-wiring/","title":"Build Wiring Diagram","text":"

Under construction.

In the meantime, to have a look at working wiring diagram for Hedgehog Fabric, run the sample generator that produces VLAB-compatible wiring diagrams:

ubuntu@sl-dev:~$ hhfab wiring sample -h\nNAME:\n   hhfab wiring sample - sample wiring diagram (would work for vlab)\n\nUSAGE:\n   hhfab wiring sample [command options] [arguments...]\n\nOPTIONS:\n   --brief, -b                    brief output (only warn and error) (default: false)\n   --fabric-mode value, -m value  fabric mode (one of: collapsed-core, spine-leaf) (default: \"spine-leaf\")\n   --help, -h                     show help\n   --verbose, -v                  verbose output (includes debug) (default: false)\n\n   wiring generator options:\n\n   --chain-control-link         chain control links instead of all switches directly connected to control node if fabric mode is spine-leaf (default: false)\n   --control-links-count value  number of control links if chain-control-link is enabled (default: 0)\n   --fabric-links-count value   number of fabric links if fabric mode is spine-leaf (default: 0)\n   --mclag-leafs-count value    number of mclag leafs (should be even) (default: 0)\n   --mclag-peer-links value     number of mclag peer links for each mclag leaf (default: 0)\n   --mclag-session-links value  number of mclag session links for each mclag leaf (default: 0)\n   --orphan-leafs-count value   number of orphan leafs (default: 0)\n   --spines-count value         number of spines if fabric mode is spine-leaf (default: 0)\n   --vpc-loopbacks value        number of vpc loopbacks for each switch (default: 0)\n
"},{"location":"install-upgrade/config/","title":"Fabric Configuration","text":"

For more information about how to use hhfab init, run hhfab init --help.

"},{"location":"install-upgrade/config/#configure-switch-users","title":"Configure switch users","text":"

It's currently only possible by using a config yaml file for the hhfab init -c <config-file.yaml> command. You can specify users to be configured on the switches in the following format:

config:\n    ...\n    fabric:\n        ...\n        switchUsers:\n          - name: test\n            password: $5$oj/NxDtFw3eTyini$VHwdjWXSNYRxlFMu.1S5ZlGJbUF/CGmCAZIBroJlax4\n            role: operator\n

Where name is the username, password is the password hash created with openssl passwd -5 command, and role is the role of the user, one of admin or operator (read-only access to sonic-cli command on the switches).

"},{"location":"install-upgrade/config/#forward-switch-metrics-and-logs","title":"Forward switch metrics and logs","text":"

There is an option to enable Grafana Alloy on all switches to forward metrics and logs to the configured targets using Prometheus Remote-Write API and Loki API. If those APIs are available from Control Node(s), but not from the switches, it's possible to enable HTTP Proxy on Control Node(s) that will be used by Grafana Alloy running on the switches to access the configured targets. It could be done by passing --control-proxy=true to hhfab init.

Metrics includes port speeds, counters, errors, operational status, transceivers, fans, power supplies, temperature sensors, BGP neighbors, LLDP neighbors, and more. Logs include agent logs.

Configuring the exporters and targets is currently only possible by using a config yaml file for the hhfab init -c <config-file.yaml> command using the following format:

config:\n    ...\n    fabric:\n        ...\n        alloy:\n            agentScrapeIntervalSeconds: 120\n            controlProxyURL: http://172.30.1.1:31028\n            lokiTargets:\n                grafana_cloud: # target name, multiple targets can be configured\n                    basicAuth: # optional\n                        password: \"<password>\"\n                        username: \"<username>\"\n                    labels: # labels to be added to all logs\n                        env: env-1\n                    url: https://logs-prod-021.grafana.net/loki/api/v1/push\n                    useControlProxy: true # if the Loki API is not available from the switches directly, use the Control Node as a proxy\n            prometheusTargets:\n                grafana_cloud: # target name, multiple targets can be configured\n                    basicAuth: # optional\n                        password: \"<password>\"\n                        username: \"<username>\"\n                    labels: # labels to be added to all metrics\n                        env: env-1\n                    sendIntervalSeconds: 120\n                    url: https://prometheus-prod-36-prod-us-west-0.grafana.net/api/prom/push\n                    useControlProxy: true # if the Loki API is not available from the switches directly, use the Control Node as a proxy\n            unixExporterCollectors: # list of node-exporter collectors to enable, https://grafana.com/docs/alloy/latest/reference/components/prometheus.exporter.unix/#collectors-list\n                - cpu\n                - filesystem\n                - loadavg\n                - meminfo\n            unixExporterEnabled: true\n            unixScrapeIntervalSeconds: 120\n

For additional options, see the AlloyConfig struct in Fabric repo.

"},{"location":"install-upgrade/onie-update/","title":"ONIE Update/Upgrade","text":""},{"location":"install-upgrade/onie-update/#hedgehog-onie-honie-supported-systems","title":"Hedgehog ONIE (HONIE) Supported Systems","text":""},{"location":"install-upgrade/onie-update/#updating-onie","title":"Updating ONIE","text":""},{"location":"install-upgrade/onie-update/#via-usb","title":"Via USB","text":"

This example shows how to update a DELL S5248 to Hedgehog ONIE (HONIE).

Note: the USB port is on the back of the switch with the Management and Console.

  1. Prepare the USB stick by burning the honie-usb.img to a 4G or larger USB drive.

  2. Insert the USB drive into the switch. For example, burn the file to disk X of a macOS machine with sudo dd if=honie-usb.img of=/dev/rdiskX bs=1m.

  3. Boot into ONIE Installer

  4. ONIE will install the ONIE update and reboot. Here are some sample logs:

    ONIE: OS Install Mode ...\nPlatform\u00a0 : x86_64-dellemc_s5200_c3538-r0\nVersion \u00a0 : 3.40.1.1-7 <- Non HONIE version\nBuild Date: 2020-03-24T20:44-07:00\nInfo: Mounting EFI System on /boot/efi ...\nInfo: BIOS mode: UEFI\nInfo: Making NOS install boot mode persistent.\nInfo: Using eth0 MAC address: 3c:2c:30:66:f0:00\nInfo: eth0:\u00a0 Checking link... up.\nInfo: Trying DHCPv4 on interface: eth0\nWarning: Unable to configure interface using DHCPv4: eth0\nONIE: Using link-local IPv4 addr: eth0: 169.254.95.249/16\nStarting: klogd... done.\nStarting: dropbear ssh daemon... done.\nStarting: telnetd... done.\ndiscover: installer mode detected.\u00a0 Running installer.\nStarting: discover... done.\nPlease press Enter to activate this console. Info: eth0:\u00a0 Checking link... up.\nInfo: Trying DHCPv4 on interface: eth0\nWarning: Unable to configure interface using DHCPv4: eth0\nONIE: Using link-local IPv4 addr: eth0: 169.254.6.139/16\nONIE: Starting ONIE Service Discovery\nInfo: Attempting file://dev/sdb1/onie-installer-x86_64-dellemc_s5248f_c3538-r0 ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64-dellemc_s5248f_c3538-r0 ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64-dellemc_s5248f_c3538-r0.bin ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64-dellemc_s5248f_c3538.bin ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-dellemc_s5248f_c3538 ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-dellemc_s5248f_c3538.bin ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64-bcm ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64-bcm.bin ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64 ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer-x86_64.bin ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer ...\nInfo: Attempting file://dev/mmcblk0p1/onie-installer.bin ...\nONIE: Executing installer: file://dev/sdb1/onie-installer-x86_64-dellemc_s5248f_c3538-r0\nVerifying image checksum ... OK.\nPreparing image archive ... OK.\nONIE: Version \u00a0 \u00a0 \u00a0 : 3.40.1.1-8 <- HONIE Version\nONIE: Architecture\u00a0 : x86_64\nONIE: Machine \u00a0 \u00a0 \u00a0 : dellemc_s5200_c3538\nONIE: Machine Rev \u00a0 : 0\nONIE: Config Version: 1\nONIE: Build Date\u00a0 \u00a0 : 2023-12-15T23:43+00:00\nInstalling ONIE on: /dev/sda\nONIE: NOS install successful: file://dev/sdb1/onie-installer-x86_64-dellemc_s5248f_c3538-r0\nONIE: Rebooting...\ndiscover: installer mode detected.\nStopping: discover...start-stop-daemon: warning: killing process 665: No such process\nInfo: Unmounting kernel filesystems\numount: can't unmount /: Invalid argument\nThe system is going down NOW!\nSent SIGTERM to all processes\nSent SIGKILL to all processes\nRequesting system reboot\n

  5. The system is now ready for use.

"},{"location":"install-upgrade/overview/","title":"Install Fabric","text":"

Under construction.

"},{"location":"install-upgrade/overview/#prerequisites","title":"Prerequisites","text":""},{"location":"install-upgrade/overview/#main-steps","title":"Main steps","text":"

This chapter is dedicated to the Hedgehog Fabric installation on bare-metal control node(s) and switches, their preparation and configuration.

Get hhfab installed following instructions from the Download section.

The main steps to install Fabric are:

  1. Install hhfab on the machines with access to the Internet
    1. Prepare Wiring Diagram
    2. Select Fabric Configuration
    3. Build Control Node configuration and installer
  2. Install Control Node
    1. Install Flatcar Linux on the Control Node
    2. Upload and run Control Node installer on the Control Node
  3. Prepare supported switches
    1. Install Hedgehog ONIE (HONIE) on them
    2. Reboot them into ONIE Install Mode to have them automatically provisioned
"},{"location":"install-upgrade/overview/#build-control-node-configuration-and-installer","title":"Build Control Node configuration and installer","text":"

It's the only step that requires Internet access, to download artifacts and build the installer.

Once you've prepared the Wiring Diagram, initialize Fabricator by running hhfab init command and passing optional configuration into it as well as wiring diagram file(s) as flags. Additionally, there are a lot of customizations available as flags, e.g. to setup default credentials, keys and etc. For more details on the command invocation, refer to hhfab init --help.

The --dev option activates the development mode which enables default credentials and keys for the Control Node and switches:

Alternatively, you can pass your own credentials and keys using --authorized-key and --control-password-hash flags. Generate a password hash with command openssl passwd -5. Further customization items are available in the config file and can be passed using the --config flag.

hhfab init --preset lab --dev --wiring file1.yaml --wiring file2.yaml\nhhfab build\n

As a result, you will get the following files in the .hhfab directory or the one you've passed using --basedir flag:

More details on configuring the Fabric are available in the Configuration section.

"},{"location":"install-upgrade/overview/#install-control-node","title":"Install Control Node","text":"

Control Node installation is fully air-gapped and doesn't require Internet access.

Download the latest stable Flatcar Container Linux ISO and boot into it (using IPMI attaching media, USB stick or any other way).

Once you've booted into the Flatcar installer, upload the file ignition.json built during the previous step to the system and run the Flatcar installation:

sudo flatcar-install -d /dev/sda -i ignition.json\n

Where /dev/sda is a disk you want to install Control Node to and ignition.json is the control-os/ignition.json file from previous step uploaded to the Flatcar installer.

Once the installation is finished, reboot the machine and wait for it to boot into the installed Flatcar Linux.

At that point, you should get into the installed Flatcar Linux using the dev or provided credentials with user core and you can now install Hedgehog Open Network Fabric on it. Download control-install.tgz to the just installed Control Node (for example, by using scp) and run it.

tar xzf control-install.tgz && cd control-install && sudo ./hhfab-recipe run\n

The command prints the logs generated while installing Fabric (including logs from the Kubernetes cluster, miscellaneous OCI registry misc components, and more). At the end, you should observe lines similar to the following:

...\n01:34:45 INF Running name=reloader-image op=\"push fabricator/reloader:v1.0.40\"\n01:34:47 INF Running name=reloader-chart op=\"push fabricator/charts/reloader:1.0.40\"\n01:34:47 INF Running name=reloader-install op=\"file /var/lib/rancher/k3s/server/manifests/hh-reloader-install.yaml\"\n01:34:47 INF Running name=reloader-wait op=\"wait deployment/reloader-reloader\"\ndeployment.apps/reloader-reloader condition met\n01:35:15 INF Done took=3m39.586394608s\n

At that point, you can start interacting with the Fabric using kubectl, kubectl fabric and k9s, all preinstalled as part of the Control Node installer.

You can now get HONIE installed on your switches and reboot them into ONIE Install Mode to have them automatically provisioned from the Control Node.

"},{"location":"install-upgrade/requirements/","title":"System Requirements","text":""},{"location":"install-upgrade/requirements/#non-ha-minimal-setup-1-control-node","title":"Non-HA (minimal) setup - 1 Control Node","text":" Minimal Recommended CPU 4 8 RAM 12 GB 16 GB Disk 100 GB 250 GB"},{"location":"install-upgrade/requirements/#future-ha-setup-3-control-nodes-per-node","title":"(Future) HA setup - 3+ Control Nodes (per node)","text":" Minimal Recommended CPU 4 8 RAM 12 GB 16 GB Disk 100 GB 250 GB"},{"location":"install-upgrade/requirements/#device-participating-in-the-hedgehog-fabric-eg-switch","title":"Device participating in the Hedgehog Fabric (e.g. switch)","text":" Minimal Recommended CPU 1 2 RAM 1 GB 1.5 GB Disk 5 GB 10 GB"},{"location":"install-upgrade/supported-devices/","title":"Supported Devices","text":""},{"location":"install-upgrade/supported-devices/#spine","title":"Spine","text":""},{"location":"install-upgrade/supported-devices/#leaf","title":"Leaf","text":""},{"location":"reference/api/","title":"API Reference","text":""},{"location":"reference/api/#packages","title":"Packages","text":""},{"location":"reference/api/#agentgithedgehogcomv1alpha2","title":"agent.githedgehog.com/v1alpha2","text":"

Package v1alpha2 contains API Schema definitions for the agent v1alpha2 API group. This is the internal API group for the switch and control node agents. Not intended to be modified by the user.

"},{"location":"reference/api/#resource-types","title":"Resource Types","text":""},{"location":"reference/api/#adminstatus","title":"AdminStatus","text":"

Underlying type: string

Appears in: - SwitchStateInterface

"},{"location":"reference/api/#agent","title":"Agent","text":"

Agent is an internal API object used by the controller to pass all relevant information to the agent running on a specific switch in order to fully configure it and manage its lifecycle. It is not intended to be used directly by users. Spec of the object isn't user-editable, it is managed by the controller. Status of the object is updated by the agent and is used by the controller to track the state of the agent and the switch it is running on. Name of the Agent object is the same as the name of the switch it is running on and it's created in the same namespace as the Switch object.

Field Description Default Validation apiVersion string agent.githedgehog.com/v1alpha2 kind string Agent metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. status AgentStatus Status is the observed state of the Agent"},{"location":"reference/api/#agentstatus","title":"AgentStatus","text":"

AgentStatus defines the observed state of the agent running on a specific switch and includes information about the switch itself as well as the state of the agent and applied configuration.

Appears in: - Agent

Field Description Default Validation version string Current running agent version installID string ID of the agent installation, used to track NOS re-installs runID string ID of the agent run, used to track NOS reboots lastHeartbeat Time Time of the last heartbeat from the agent lastAttemptTime Time Time of the last attempt to apply configuration lastAttemptGen integer Generation of the last attempt to apply configuration lastAppliedTime Time Time of the last successful configuration application lastAppliedGen integer Generation of the last successful configuration application state SwitchState Detailed switch state updated with each heartbeat conditions Condition array Conditions of the agent, includes readiness marker for use with kubectl wait"},{"location":"reference/api/#bgpmessages","title":"BGPMessages","text":"

Appears in: - SwitchStateBGPNeighbor

Field Description Default Validation received BGPMessagesCounters sent BGPMessagesCounters"},{"location":"reference/api/#bgpmessagescounters","title":"BGPMessagesCounters","text":"

Appears in: - BGPMessages

Field Description Default Validation capability integer keepalive integer notification integer open integer routeRefresh integer update integer"},{"location":"reference/api/#bgpneighborsessionstate","title":"BGPNeighborSessionState","text":"

Underlying type: string

Appears in: - SwitchStateBGPNeighbor

"},{"location":"reference/api/#bgppeertype","title":"BGPPeerType","text":"

Underlying type: string

Appears in: - SwitchStateBGPNeighbor

"},{"location":"reference/api/#operstatus","title":"OperStatus","text":"

Underlying type: string

Appears in: - SwitchStateInterface

"},{"location":"reference/api/#switchstate","title":"SwitchState","text":"

Appears in: - AgentStatus

Field Description Default Validation nos SwitchStateNOS Information about the switch and NOS interfaces object (keys:string, values:SwitchStateInterface) Switch interfaces state (incl. physical, management and port channels) breakouts object (keys:string, values:SwitchStateBreakout) Breakout ports state (port -> breakout state) bgpNeighbors object (keys:string, values:map[string]SwitchStateBGPNeighbor) State of all BGP neighbors (VRF -> neighbor address -> state) platform SwitchStatePlatform State of the switch platform (fans, PSUs, sensors) criticalResources SwitchStateCRM State of the critical resources (ACLs, routes, etc.)"},{"location":"reference/api/#switchstatebgpneighbor","title":"SwitchStateBGPNeighbor","text":"

Appears in: - SwitchState

Field Description Default Validation connectionsDropped integer enabled boolean establishedTransitions integer lastEstablished Time lastRead Time lastResetReason string lastResetTime Time lastWrite Time localAS integer messages BGPMessages peerAS integer peerGroup string peerPort integer peerType BGPPeerType remoteRouterID string sessionState BGPNeighborSessionState shutdownMessage string prefixes object (keys:string, values:SwitchStateBGPNeighborPrefixes)"},{"location":"reference/api/#switchstatebgpneighborprefixes","title":"SwitchStateBGPNeighborPrefixes","text":"

Appears in: - SwitchStateBGPNeighbor

Field Description Default Validation received integer receivedPrePolicy integer sent integer"},{"location":"reference/api/#switchstatebreakout","title":"SwitchStateBreakout","text":"

Appears in: - SwitchState

Field Description Default Validation mode string members string array status string"},{"location":"reference/api/#switchstatecrm","title":"SwitchStateCRM","text":"

Appears in: - SwitchState

Field Description Default Validation aclStats SwitchStateCRMACLStats stats SwitchStateCRMStats"},{"location":"reference/api/#switchstatecrmacldetails","title":"SwitchStateCRMACLDetails","text":"

Appears in: - SwitchStateCRMACLInfo

Field Description Default Validation groupsAvailable integer groupsUsed integer tablesAvailable integer tablesUsed integer"},{"location":"reference/api/#switchstatecrmaclinfo","title":"SwitchStateCRMACLInfo","text":"

Appears in: - SwitchStateCRMACLStats

Field Description Default Validation lag SwitchStateCRMACLDetails port SwitchStateCRMACLDetails rif SwitchStateCRMACLDetails switch SwitchStateCRMACLDetails vlan SwitchStateCRMACLDetails"},{"location":"reference/api/#switchstatecrmaclstats","title":"SwitchStateCRMACLStats","text":"

Appears in: - SwitchStateCRM

Field Description Default Validation egress SwitchStateCRMACLInfo ingress SwitchStateCRMACLInfo"},{"location":"reference/api/#switchstatecrmstats","title":"SwitchStateCRMStats","text":"

Appears in: - SwitchStateCRM

Field Description Default Validation dnatEntriesAvailable integer dnatEntriesUsed integer fdbEntriesAvailable integer fdbEntriesUsed integer ipmcEntriesAvailable integer ipmcEntriesUsed integer ipv4NeighborsAvailable integer ipv4NeighborsUsed integer ipv4NexthopsAvailable integer ipv4NexthopsUsed integer ipv4RoutesAvailable integer ipv4RoutesUsed integer ipv6NeighborsAvailable integer ipv6NeighborsUsed integer ipv6NexthopsAvailable integer ipv6NexthopsUsed integer ipv6RoutesAvailable integer ipv6RoutesUsed integer nexthopGroupMembersAvailable integer nexthopGroupMembersUsed integer nexthopGroupsAvailable integer nexthopGroupsUsed integer snatEntriesAvailable integer snatEntriesUsed integer"},{"location":"reference/api/#switchstateinterface","title":"SwitchStateInterface","text":"

Appears in: - SwitchState

Field Description Default Validation enabled boolean adminStatus AdminStatus operStatus OperStatus mac string lastChanged Time speed string counters SwitchStateInterfaceCounters transceiver SwitchStateTransceiver lldpNeighbors SwitchStateLLDPNeighbor array"},{"location":"reference/api/#switchstateinterfacecounters","title":"SwitchStateInterfaceCounters","text":"

Appears in: - SwitchStateInterface

Field Description Default Validation inBitsPerSecond float inDiscards integer inErrors integer inPktsPerSecond float inUtilization integer lastClear Time outBitsPerSecond float outDiscards integer outErrors integer outPktsPerSecond float outUtilization integer"},{"location":"reference/api/#switchstatelldpneighbor","title":"SwitchStateLLDPNeighbor","text":"

Appears in: - SwitchStateInterface

Field Description Default Validation chassisID string systemName string systemDescription string portID string portDescription string manufacturer string model string serialNumber string"},{"location":"reference/api/#switchstatenos","title":"SwitchStateNOS","text":"

SwitchStateNOS contains information about the switch and NOS received from the switch itself by the agent

Appears in: - SwitchState

Field Description Default Validation asicVersion string ASIC name, such as \"broadcom\" or \"vs\" buildCommit string NOS build commit buildDate string NOS build date builtBy string NOS build user configDbVersion string NOS config DB version, such as \"version_4_2_1\" distributionVersion string Distribution version, such as \"Debian 10.13\" hardwareVersion string Hardware version, such as \"X01\" hwskuVersion string Hwsku version, such as \"DellEMC-S5248f-P-25G-DPB\" kernelVersion string Kernel version, such as \"5.10.0-21-amd64\" mfgName string Manufacturer name, such as \"Dell EMC\" platformName string Platform name, such as \"x86_64-dellemc_s5248f_c3538-r0\" productDescription string NOS product description, such as \"Enterprise SONiC Distribution by Broadcom - Enterprise Base package\" productVersion string NOS product version, empty for Broadcom SONiC serialNumber string Switch serial number softwareVersion string NOS software version, such as \"4.2.0-Enterprise_Base\" uptime string Switch uptime, such as \"21:21:27 up 1 day, 23:26, 0 users, load average: 1.92, 1.99, 2.00 \""},{"location":"reference/api/#switchstateplatform","title":"SwitchStatePlatform","text":"

Appears in: - SwitchState

Field Description Default Validation fans object (keys:string, values:SwitchStatePlatformFan) psus object (keys:string, values:SwitchStatePlatformPSU) temperature object (keys:string, values:SwitchStatePlatformTemperature)"},{"location":"reference/api/#switchstateplatformfan","title":"SwitchStatePlatformFan","text":"

Appears in: - SwitchStatePlatform

Field Description Default Validation direction string speed float presense boolean status boolean"},{"location":"reference/api/#switchstateplatformpsu","title":"SwitchStatePlatformPSU","text":"

Appears in: - SwitchStatePlatform

Field Description Default Validation inputCurrent float inputPower float inputVoltage float outputCurrent float outputPower float outputVoltage float presense boolean status boolean"},{"location":"reference/api/#switchstateplatformtemperature","title":"SwitchStatePlatformTemperature","text":"

Appears in: - SwitchStatePlatform

Field Description Default Validation temperature float alarms string highThreshold float criticalHighThreshold float lowThreshold float criticalLowThreshold float"},{"location":"reference/api/#switchstatetransceiver","title":"SwitchStateTransceiver","text":"

Appears in: - SwitchStateInterface

Field Description Default Validation description string cableClass string formFactor string connectorType string present string cableLength float operStatus string temperature float voltage float serialNumber string vendor string vendorPart string vendorOUI string vendorRev string"},{"location":"reference/api/#dhcpgithedgehogcomv1alpha2","title":"dhcp.githedgehog.com/v1alpha2","text":"

Package v1alpha2 contains API Schema definitions for the dhcp v1alpha2 API group. It is the primary internal API group for the intended Hedgehog DHCP server configuration and storing leases as well as making them available to the end user through API. Not intended to be modified by the user.

"},{"location":"reference/api/#resource-types_1","title":"Resource Types","text":""},{"location":"reference/api/#dhcpallocated","title":"DHCPAllocated","text":"

DHCPAllocated is a single allocated IP with expiry time and hostname from DHCP requests, it's effectively a DHCP lease

Appears in: - DHCPSubnetStatus

Field Description Default Validation ip string Allocated IP address expiry Time Expiry time of the lease hostname string Hostname from DHCP request"},{"location":"reference/api/#dhcpsubnet","title":"DHCPSubnet","text":"

DHCPSubnet is the configuration (spec) for the Hedgehog DHCP server and storage for the leases (status). It's primary internal API group, but it makes allocated IPs / leases information available to the end user through API. Not intended to be modified by the user.

Field Description Default Validation apiVersion string dhcp.githedgehog.com/v1alpha2 kind string DHCPSubnet metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec DHCPSubnetSpec Spec is the desired state of the DHCPSubnet status DHCPSubnetStatus Status is the observed state of the DHCPSubnet"},{"location":"reference/api/#dhcpsubnetspec","title":"DHCPSubnetSpec","text":"

DHCPSubnetSpec defines the desired state of DHCPSubnet

Appears in: - DHCPSubnet

Field Description Default Validation subnet string Full VPC subnet name (including VPC name), such as \"vpc-0/default\" cidrBlock string CIDR block to use for VPC subnet, such as \"10.10.10.0/24\" gateway string Gateway, such as 10.10.10.1 startIP string Start IP from the CIDRBlock to allocate IPs, such as 10.10.10.10 endIP string End IP from the CIDRBlock to allocate IPs, such as 10.10.10.99 vrf string VRF name to identify specific VPC (will be added to DHCP packets by DHCP relay in suboption 151), such as \"VrfVvpc-1\" as it's named on switch circuitID string VLAN ID to identify specific subnet withing the VPC, such as \"Vlan1000\" as it's named on switch pxeURL string PXEURL (optional) to identify the pxe server to use to boot hosts connected to this segment such as http://10.10.10.99/bootfilename or tftp://10.10.10.99/bootfilename, http query strings are not supported"},{"location":"reference/api/#dhcpsubnetstatus","title":"DHCPSubnetStatus","text":"

DHCPSubnetStatus defines the observed state of DHCPSubnet

Appears in: - DHCPSubnet

Field Description Default Validation allocated object (keys:string, values:DHCPAllocated) Allocated is a map of allocated IPs with expiry time and hostname from DHCP requests"},{"location":"reference/api/#vpcgithedgehogcomv1alpha2","title":"vpc.githedgehog.com/v1alpha2","text":"

Package v1alpha2 contains API Schema definitions for the vpc v1alpha2 API group. It is public API group for the VPCs and Externals APIs. Intended to be used by the user.

"},{"location":"reference/api/#resource-types_2","title":"Resource Types","text":""},{"location":"reference/api/#external","title":"External","text":"

External object represents an external system connected to the Fabric and available to the specific IPv4Namespace. Users can do external peering with the external system by specifying the name of the External Object without need to worry about the details of how external system is attached to the Fabric.

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string External metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec ExternalSpec Spec is the desired state of the External status ExternalStatus Status is the observed state of the External"},{"location":"reference/api/#externalattachment","title":"ExternalAttachment","text":"

ExternalAttachment is a definition of how specific switch is connected with external system (External object). Effectively it represents BGP peering between the switch and external system including all needed configuration.

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string ExternalAttachment metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec ExternalAttachmentSpec Spec is the desired state of the ExternalAttachment status ExternalAttachmentStatus Status is the observed state of the ExternalAttachment"},{"location":"reference/api/#externalattachmentneighbor","title":"ExternalAttachmentNeighbor","text":"

ExternalAttachmentNeighbor defines the BGP neighbor configuration for the external attachment

Appears in: - ExternalAttachmentSpec

Field Description Default Validation asn integer ASN is the ASN of the BGP neighbor ip string IP is the IP address of the BGP neighbor to peer with"},{"location":"reference/api/#externalattachmentspec","title":"ExternalAttachmentSpec","text":"

ExternalAttachmentSpec defines the desired state of ExternalAttachment

Appears in: - ExternalAttachment

Field Description Default Validation external string External is the name of the External object this attachment belongs to connection string Connection is the name of the Connection object this attachment belongs to (essentialy the name of the switch/port) switch ExternalAttachmentSwitch Switch is the switch port configuration for the external attachment neighbor ExternalAttachmentNeighbor Neighbor is the BGP neighbor configuration for the external attachment"},{"location":"reference/api/#externalattachmentstatus","title":"ExternalAttachmentStatus","text":"

ExternalAttachmentStatus defines the observed state of ExternalAttachment

Appears in: - ExternalAttachment

"},{"location":"reference/api/#externalattachmentswitch","title":"ExternalAttachmentSwitch","text":"

ExternalAttachmentSwitch defines the switch port configuration for the external attachment

Appears in: - ExternalAttachmentSpec

Field Description Default Validation vlan integer VLAN (optional) is the VLAN ID used for the subinterface on a switch port specified in the connection, set to 0 if no VLAN is used ip string IP is the IP address of the subinterface on a switch port specified in the connection"},{"location":"reference/api/#externalpeering","title":"ExternalPeering","text":"

ExternalPeering is the Schema for the externalpeerings API

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string ExternalPeering metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec ExternalPeeringSpec Spec is the desired state of the ExternalPeering status ExternalPeeringStatus Status is the observed state of the ExternalPeering"},{"location":"reference/api/#externalpeeringspec","title":"ExternalPeeringSpec","text":"

ExternalPeeringSpec defines the desired state of ExternalPeering

Appears in: - ExternalPeering

Field Description Default Validation permit ExternalPeeringSpecPermit Permit defines the peering policy - which VPC and External to peer with and which subnets/prefixes to permit"},{"location":"reference/api/#externalpeeringspecexternal","title":"ExternalPeeringSpecExternal","text":"

ExternalPeeringSpecExternal defines the External-side of the configuration to peer with

Appears in: - ExternalPeeringSpecPermit

Field Description Default Validation name string Name is the name of the External to peer with prefixes ExternalPeeringSpecPrefix array Prefixes is the list of prefixes to permit from the External to the VPC"},{"location":"reference/api/#externalpeeringspecpermit","title":"ExternalPeeringSpecPermit","text":"

ExternalPeeringSpecPermit defines the peering policy - which VPC and External to peer with and which subnets/prefixes to permit

Appears in: - ExternalPeeringSpec

Field Description Default Validation vpc ExternalPeeringSpecVPC VPC is the VPC-side of the configuration to peer with external ExternalPeeringSpecExternal External is the External-side of the configuration to peer with"},{"location":"reference/api/#externalpeeringspecprefix","title":"ExternalPeeringSpecPrefix","text":"

ExternalPeeringSpecPrefix defines the prefix to permit from the External to the VPC

Appears in: - ExternalPeeringSpecExternal

Field Description Default Validation prefix string Prefix is the subnet to permit from the External to the VPC, e.g. 0.0.0.0/0 for any route including default route.It matches any prefix length less than or equal to 32 effectively permitting all prefixes within the specified one."},{"location":"reference/api/#externalpeeringspecvpc","title":"ExternalPeeringSpecVPC","text":"

ExternalPeeringSpecVPC defines the VPC-side of the configuration to peer with

Appears in: - ExternalPeeringSpecPermit

Field Description Default Validation name string Name is the name of the VPC to peer with subnets string array Subnets is the list of subnets to advertise from VPC to the External"},{"location":"reference/api/#externalpeeringstatus","title":"ExternalPeeringStatus","text":"

ExternalPeeringStatus defines the observed state of ExternalPeering

Appears in: - ExternalPeering

"},{"location":"reference/api/#externalspec","title":"ExternalSpec","text":"

ExternalSpec describes IPv4 namespace External belongs to and inbound/outbound communities which are used to filter routes from/to the external system.

Appears in: - External

Field Description Default Validation ipv4Namespace string IPv4Namespace is the name of the IPv4Namespace this External belongs to inboundCommunity string InboundCommunity is the inbound community to filter routes from the external system (e.g. 65102:5000) outboundCommunity string OutboundCommunity is theoutbound community that all outbound routes will be stamped with (e.g. 50000:50001)"},{"location":"reference/api/#externalstatus","title":"ExternalStatus","text":"

ExternalStatus defines the observed state of External

Appears in: - External

"},{"location":"reference/api/#ipv4namespace","title":"IPv4Namespace","text":"

IPv4Namespace represents a namespace for VPC subnets allocation. All VPC subnets withing a single IPv4Namespace are non-overlapping. Users can create multiple IPv4Namespaces to allocate same VPC subnets.

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string IPv4Namespace metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec IPv4NamespaceSpec Spec is the desired state of the IPv4Namespace status IPv4NamespaceStatus Status is the observed state of the IPv4Namespace"},{"location":"reference/api/#ipv4namespacespec","title":"IPv4NamespaceSpec","text":"

IPv4NamespaceSpec defines the desired state of IPv4Namespace

Appears in: - IPv4Namespace

Field Description Default Validation subnets string array Subnets is the list of subnets to allocate VPC subnets from, couldn't overlap between each other and with Fabric reserved subnets MaxItems: 20 MinItems: 1"},{"location":"reference/api/#ipv4namespacestatus","title":"IPv4NamespaceStatus","text":"

IPv4NamespaceStatus defines the observed state of IPv4Namespace

Appears in: - IPv4Namespace

"},{"location":"reference/api/#vpc","title":"VPC","text":"

VPC is Virtual Private Cloud, similar to the public cloud VPC it provides an isolated private network for the resources with support for multiple subnets each with user-provided VLANs and on-demand DHCP.

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string VPC metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec VPCSpec Spec is the desired state of the VPC status VPCStatus Status is the observed state of the VPC"},{"location":"reference/api/#vpcattachment","title":"VPCAttachment","text":"

VPCAttachment is the Schema for the vpcattachments API

Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string VPCAttachment metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec VPCAttachmentSpec Spec is the desired state of the VPCAttachment status VPCAttachmentStatus Status is the observed state of the VPCAttachment"},{"location":"reference/api/#vpcattachmentspec","title":"VPCAttachmentSpec","text":"

VPCAttachmentSpec defines the desired state of VPCAttachment

Appears in: - VPCAttachment

Field Description Default Validation subnet string Subnet is the full name of the VPC subnet to attach to, such as \"vpc-1/default\" connection string Connection is the name of the connection to attach to the VPC nativeVLAN boolean NativeVLAN is the flag to indicate if the native VLAN should be used for attaching the VPC subnet"},{"location":"reference/api/#vpcattachmentstatus","title":"VPCAttachmentStatus","text":"

VPCAttachmentStatus defines the observed state of VPCAttachment

Appears in: - VPCAttachment

"},{"location":"reference/api/#vpcdhcp","title":"VPCDHCP","text":"

VPCDHCP defines the on-demand DHCP configuration for the subnet

Appears in: - VPCSubnet

Field Description Default Validation relay string Relay is the DHCP relay IP address, if specified, DHCP server will be disabled enable boolean Enable enables DHCP server for the subnet range VPCDHCPRange Range (optional) is the DHCP range for the subnet if DHCP server is enabled pxeURL string PXEURL (optional) to identify the pxe server to use to boot hosts connected to this segment such as http://10.10.10.99/bootfilename or tftp://10.10.10.99/bootfilename, http query strings are not supported"},{"location":"reference/api/#vpcdhcprange","title":"VPCDHCPRange","text":"

VPCDHCPRange defines the DHCP range for the subnet if DHCP server is enabled

Appears in: - VPCDHCP

Field Description Default Validation start string Start is the start IP address of the DHCP range end string End is the end IP address of the DHCP range"},{"location":"reference/api/#vpcpeer","title":"VPCPeer","text":"

Appears in: - VPCPeeringSpec

Field Description Default Validation subnets string array Subnets is the list of subnets to advertise from current VPC to the peer VPC MaxItems: 10 MinItems: 1"},{"location":"reference/api/#vpcpeering","title":"VPCPeering","text":"

VPCPeering represents a peering between two VPCs with corresponding filtering rules. Minimal example of the VPC peering showing vpc-1 to vpc-2 peering with all subnets allowed:

spec:\n  permit:\n  - vpc-1: {}\n    vpc-2: {}\n
Field Description Default Validation apiVersion string vpc.githedgehog.com/v1alpha2 kind string VPCPeering metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec VPCPeeringSpec Spec is the desired state of the VPCPeering status VPCPeeringStatus Status is the observed state of the VPCPeering"},{"location":"reference/api/#vpcpeeringspec","title":"VPCPeeringSpec","text":"

VPCPeeringSpec defines the desired state of VPCPeering

Appears in: - VPCPeering

Field Description Default Validation remote string permit map[string]VPCPeer array Permit defines a list of the peering policies - which VPC subnets will have access to the peer VPC subnets. MaxItems: 10 MinItems: 1"},{"location":"reference/api/#vpcpeeringstatus","title":"VPCPeeringStatus","text":"

VPCPeeringStatus defines the observed state of VPCPeering

Appears in: - VPCPeering

"},{"location":"reference/api/#vpcspec","title":"VPCSpec","text":"

VPCSpec defines the desired state of VPC. At least one subnet is required.

Appears in: - VPC

Field Description Default Validation subnets object (keys:string, values:VPCSubnet) Subnets is the list of VPC subnets to configure ipv4Namespace string IPv4Namespace is the name of the IPv4Namespace this VPC belongs to (if not specified, \"default\" is used) vlanNamespace string VLANNamespace is the name of the VLANNamespace this VPC belongs to (if not specified, \"default\" is used) defaultIsolated boolean DefaultIsolated sets default bahivour for isolated mode for the subnets (disabled by default) defaultRestricted boolean DefaultRestricted sets default bahivour for restricted mode for the subnets (disabled by default) permit string array array Permit defines a list of the access policies between the subnets within the VPC - each policy is a list of subnets that have access to each other.It's applied on top of the subnet isolation flag and if subnet isn't isolated it's not required to have it in a permit list while if vpc is markedas isolated it's required to have it in a permit list to have access to other subnets. staticRoutes VPCStaticRoute array StaticRoutes is the list of additional static routes for the VPC"},{"location":"reference/api/#vpcstaticroute","title":"VPCStaticRoute","text":"

VPCStaticRoute defines the static route for the VPC

Appears in: - VPCSpec

Field Description Default Validation prefix string Prefix for the static route (mandatory), e.g. 10.42.0.0/24 nextHops string array NextHops for the static route (at least one is required), e.g. 10.99.0.0"},{"location":"reference/api/#vpcstatus","title":"VPCStatus","text":"

VPCStatus defines the observed state of VPC

Appears in: - VPC

"},{"location":"reference/api/#vpcsubnet","title":"VPCSubnet","text":"

VPCSubnet defines the VPC subnet configuration

Appears in: - VPCSpec

Field Description Default Validation subnet string Subnet is the subnet CIDR block, such as \"10.0.0.0/24\", should belong to the IPv4Namespace and be unique within the namespace dhcp VPCDHCP DHCP is the on-demand DHCP configuration for the subnet vlan string VLAN is the VLAN ID for the subnet, should belong to the VLANNamespace and be unique within the namespace isolated boolean Isolated is the flag to enable isolated mode for the subnet which means no access to and from the other subnets within the VPC restricted boolean Restricted is the flag to enable restricted mode for the subnet which means no access between hosts within the subnet itself"},{"location":"reference/api/#wiringgithedgehogcomv1alpha2","title":"wiring.githedgehog.com/v1alpha2","text":"

Package v1alpha2 contains API Schema definitions for the wiring v1alpha2 API group. It is public API group mainly for the underlay definition including Switches, Server, wiring between them and etc. Intended to be used by the user.

"},{"location":"reference/api/#resource-types_3","title":"Resource Types","text":""},{"location":"reference/api/#baseportname","title":"BasePortName","text":"

BasePortName defines the full name of the switch port

Appears in: - ConnExternalLink - ConnFabricLinkSwitch - ConnMgmtLinkServer - ConnMgmtLinkSwitch - ConnStaticExternalLinkSwitch - ServerToSwitchLink - SwitchToSwitchLink

Field Description Default Validation port string Port defines the full name of the switch port in the format of \"device/port\", such as \"spine-1/Ethernet1\".SONiC port name is used as a port name and switch name should be same as the name of the Switch object."},{"location":"reference/api/#connbundled","title":"ConnBundled","text":"

ConnBundled defines the bundled connection (port channel, single server to a single switch with multiple links)

Appears in: - ConnectionSpec

Field Description Default Validation links ServerToSwitchLink array Links is the list of server-to-switch links mtu integer MTU is the MTU to be configured on the switch port or port channel"},{"location":"reference/api/#conneslag","title":"ConnESLAG","text":"

ConnESLAG defines the ESLAG connection (port channel, single server to 2-4 switches with multiple links)

Appears in: - ConnectionSpec

Field Description Default Validation links ServerToSwitchLink array Links is the list of server-to-switch links MinItems: 2 mtu integer MTU is the MTU to be configured on the switch port or port channel fallback boolean Fallback is the optional flag that used to indicate one of the links in LACP port channel to be used as a fallback link"},{"location":"reference/api/#connexternal","title":"ConnExternal","text":"

ConnExternal defines the external connection (single switch to a single external device with a single link)

Appears in: - ConnectionSpec

Field Description Default Validation link ConnExternalLink Link is the external connection link"},{"location":"reference/api/#connexternallink","title":"ConnExternalLink","text":"

ConnExternalLink defines the external connection link

Appears in: - ConnExternal

Field Description Default Validation switch BasePortName"},{"location":"reference/api/#connfabric","title":"ConnFabric","text":"

ConnFabric defines the fabric connection (single spine to a single leaf with at least one link)

Appears in: - ConnectionSpec

Field Description Default Validation links FabricLink array Links is the list of spine-to-leaf links MinItems: 1"},{"location":"reference/api/#connfabriclinkswitch","title":"ConnFabricLinkSwitch","text":"

ConnFabricLinkSwitch defines the switch side of the fabric link

Appears in: - FabricLink

Field Description Default Validation port string Port defines the full name of the switch port in the format of \"device/port\", such as \"spine-1/Ethernet1\".SONiC port name is used as a port name and switch name should be same as the name of the Switch object. ip string IP is the IP address of the switch side of the fabric link (switch port configuration) Pattern: ^((25[0-5]|(2[0-4]|1\\d|[1-9]|)\\d)\\.?\\b){4}/([1-2]?[0-9]|3[0-2])$"},{"location":"reference/api/#connmclag","title":"ConnMCLAG","text":"

ConnMCLAG defines the MCLAG connection (port channel, single server to pair of switches with multiple links)

Appears in: - ConnectionSpec

Field Description Default Validation links ServerToSwitchLink array Links is the list of server-to-switch links MinItems: 2 mtu integer MTU is the MTU to be configured on the switch port or port channel fallback boolean Fallback is the optional flag that used to indicate one of the links in LACP port channel to be used as a fallback link"},{"location":"reference/api/#connmclagdomain","title":"ConnMCLAGDomain","text":"

ConnMCLAGDomain defines the MCLAG domain connection which makes two switches into a single logical switch or redundancy group and allows to use MCLAG connections to connect servers in a multi-homed way.

Appears in: - ConnectionSpec

Field Description Default Validation peerLinks SwitchToSwitchLink array PeerLinks is the list of peer links between the switches, used to pass server traffic between switch MinItems: 1 sessionLinks SwitchToSwitchLink array SessionLinks is the list of session links between the switches, used only to pass MCLAG control plane and BGPtraffic between switches MinItems: 1"},{"location":"reference/api/#connmgmt","title":"ConnMgmt","text":"

ConnMgmt defines the management connection (single control node/server to a single switch with a single link)

Appears in: - ConnectionSpec

Field Description Default Validation link ConnMgmtLink"},{"location":"reference/api/#connmgmtlink","title":"ConnMgmtLink","text":"

ConnMgmtLink defines the management connection link

Appears in: - ConnMgmt

Field Description Default Validation server ConnMgmtLinkServer Server is the server side of the management link switch ConnMgmtLinkSwitch Switch is the switch side of the management link"},{"location":"reference/api/#connmgmtlinkserver","title":"ConnMgmtLinkServer","text":"

ConnMgmtLinkServer defines the server side of the management link

Appears in: - ConnMgmtLink

Field Description Default Validation port string Port defines the full name of the switch port in the format of \"device/port\", such as \"spine-1/Ethernet1\".SONiC port name is used as a port name and switch name should be same as the name of the Switch object. ip string IP is the IP address of the server side of the management link (control node port configuration) Pattern: ^((25[0-5]|(2[0-4]|1\\d|[1-9]|)\\d)\\.?\\b){4}/([1-2]?[0-9]|3[0-2])$ mac string MAC is an optional MAC address of the control node port for the management link, if specified will be used tocreate a \"virtual\" link with the connection names on the control node"},{"location":"reference/api/#connmgmtlinkswitch","title":"ConnMgmtLinkSwitch","text":"

ConnMgmtLinkSwitch defines the switch side of the management link

Appears in: - ConnMgmtLink

Field Description Default Validation port string Port defines the full name of the switch port in the format of \"device/port\", such as \"spine-1/Ethernet1\".SONiC port name is used as a port name and switch name should be same as the name of the Switch object. ip string IP is the IP address of the switch side of the management link (switch port configuration) Pattern: ^((25[0-5]|(2[0-4]|1\\d|[1-9]|)\\d)\\.?\\b){4}/([1-2]?[0-9]|3[0-2])$ oniePortName string ONIEPortName is an optional ONIE port name of the switch side of the management link that's only used by the IPv6 Link Local discovery"},{"location":"reference/api/#connstaticexternal","title":"ConnStaticExternal","text":"

ConnStaticExternal defines the static external connection (single switch to a single external device with a single link)

Appears in: - ConnectionSpec

Field Description Default Validation link ConnStaticExternalLink Link is the static external connection link withinVPC string WithinVPC is the optional VPC name to provision the static external connection within the VPC VRF instead of default one to make resource available to the specific VPC"},{"location":"reference/api/#connstaticexternallink","title":"ConnStaticExternalLink","text":"

ConnStaticExternalLink defines the static external connection link

Appears in: - ConnStaticExternal

Field Description Default Validation switch ConnStaticExternalLinkSwitch Switch is the switch side of the static external connection link"},{"location":"reference/api/#connstaticexternallinkswitch","title":"ConnStaticExternalLinkSwitch","text":"

ConnStaticExternalLinkSwitch defines the switch side of the static external connection link

Appears in: - ConnStaticExternalLink

Field Description Default Validation port string Port defines the full name of the switch port in the format of \"device/port\", such as \"spine-1/Ethernet1\".SONiC port name is used as a port name and switch name should be same as the name of the Switch object. ip string IP is the IP address of the switch side of the static external connection link (switch port configuration) Pattern: ^((25[0-5]|(2[0-4]|1\\d|[1-9]|)\\d)\\.?\\b){4}/([1-2]?[0-9]|3[0-2])$ nextHop string NextHop is the next hop IP address for static routes that will be created for the subnets Pattern: ^((25[0-5]|(2[0-4]|1\\d|[1-9]|)\\d)\\.?\\b){4}$ subnets string array Subnets is the list of subnets that will get static routes using the specified next hop vlan integer VLAN is the optional VLAN ID to be configured on the switch port"},{"location":"reference/api/#connunbundled","title":"ConnUnbundled","text":"

ConnUnbundled defines the unbundled connection (no port channel, single server to a single switch with a single link)

Appears in: - ConnectionSpec

Field Description Default Validation link ServerToSwitchLink Link is the server-to-switch link mtu integer MTU is the MTU to be configured on the switch port or port channel"},{"location":"reference/api/#connvpcloopback","title":"ConnVPCLoopback","text":"

ConnVPCLoopback defines the VPC loopback connection (multiple port pairs on a single switch) that enables automated workaround named \"VPC Loopback\" that allow to avoid switch hardware limitations and traffic going through CPU in some cases

Appears in: - ConnectionSpec

Field Description Default Validation links SwitchToSwitchLink array Links is the list of VPC loopback links MinItems: 1"},{"location":"reference/api/#connection","title":"Connection","text":"

Connection object represents a logical and physical connections between any devices in the Fabric (Switch, Server and External objects). It's needed to define all physical and logical connections between the devices in the Wiring Diagram. Connection type is defined by the top-level field in the ConnectionSpec. Exactly one of them could be used in a single Connection object.

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string Connection metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec ConnectionSpec Spec is the desired state of the Connection status ConnectionStatus Status is the observed state of the Connection"},{"location":"reference/api/#connectionspec","title":"ConnectionSpec","text":"

ConnectionSpec defines the desired state of Connection

Appears in: - Connection

Field Description Default Validation unbundled ConnUnbundled Unbundled defines the unbundled connection (no port channel, single server to a single switch with a single link) bundled ConnBundled Bundled defines the bundled connection (port channel, single server to a single switch with multiple links) management ConnMgmt Management defines the management connection (single control node/server to a single switch with a single link) mclag ConnMCLAG MCLAG defines the MCLAG connection (port channel, single server to pair of switches with multiple links) eslag ConnESLAG ESLAG defines the ESLAG connection (port channel, single server to 2-4 switches with multiple links) mclagDomain ConnMCLAGDomain MCLAGDomain defines the MCLAG domain connection which makes two switches into a single logical switch for server multi-homing fabric ConnFabric Fabric defines the fabric connection (single spine to a single leaf with at least one link) vpcLoopback ConnVPCLoopback VPCLoopback defines the VPC loopback connection (multiple port pairs on a single switch) for automated workaround external ConnExternal External defines the external connection (single switch to a single external device with a single link) staticExternal ConnStaticExternal StaticExternal defines the static external connection (single switch to a single external device with a single link)"},{"location":"reference/api/#connectionstatus","title":"ConnectionStatus","text":"

ConnectionStatus defines the observed state of Connection

Appears in: - Connection

"},{"location":"reference/api/#fabriclink","title":"FabricLink","text":"

FabricLink defines the fabric connection link

Appears in: - ConnFabric

Field Description Default Validation spine ConnFabricLinkSwitch Spine is the spine side of the fabric link leaf ConnFabricLinkSwitch Leaf is the leaf side of the fabric link"},{"location":"reference/api/#location","title":"Location","text":"

Location defines the geographical position of the device in a datacenter

Appears in: - SwitchSpec

Field Description Default Validation location string aisle string row string rack string slot string"},{"location":"reference/api/#locationsig","title":"LocationSig","text":"

LocationSig contains signatures for the location UUID as well as the device location itself

Appears in: - SwitchSpec

Field Description Default Validation sig string uuidSig string"},{"location":"reference/api/#rack","title":"Rack","text":"

Rack is the Schema for the racks API

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string Rack metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec RackSpec status RackStatus"},{"location":"reference/api/#rackposition","title":"RackPosition","text":"

RackPosition defines the geographical position of the rack in a datacenter

Appears in: - RackSpec

Field Description Default Validation location string aisle string row string"},{"location":"reference/api/#rackspec","title":"RackSpec","text":"

RackSpec defines the properties of a rack which we are modelling

Appears in: - Rack

Field Description Default Validation position RackPosition"},{"location":"reference/api/#rackstatus","title":"RackStatus","text":"

RackStatus defines the observed state of Rack

Appears in: - Rack

"},{"location":"reference/api/#server","title":"Server","text":"

Server is the Schema for the servers API

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string Server metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec ServerSpec Spec is desired state of the server status ServerStatus Status is the observed state of the server"},{"location":"reference/api/#serverfacingconnectionconfig","title":"ServerFacingConnectionConfig","text":"

ServerFacingConnectionConfig defines any server-facing connection (unbundled, bundled, mclag, etc.) configuration

Appears in: - ConnBundled - ConnESLAG - ConnMCLAG - ConnUnbundled

Field Description Default Validation mtu integer MTU is the MTU to be configured on the switch port or port channel"},{"location":"reference/api/#serverspec","title":"ServerSpec","text":"

ServerSpec defines the desired state of Server

Appears in: - Server

Field Description Default Validation type ServerType Type is the type of server, could be control for control nodes or default (empty string) for everything else Enum: [control] description string Description is a description of the server profile string Profile is the profile of the server, name of the ServerProfile object to be used for this server, currently not used by the Fabric"},{"location":"reference/api/#serverstatus","title":"ServerStatus","text":"

ServerStatus defines the observed state of Server

Appears in: - Server

"},{"location":"reference/api/#servertoswitchlink","title":"ServerToSwitchLink","text":"

ServerToSwitchLink defines the server-to-switch link

Appears in: - ConnBundled - ConnESLAG - ConnMCLAG - ConnUnbundled

Field Description Default Validation server BasePortName Server is the server side of the connection switch BasePortName Switch is the switch side of the connection"},{"location":"reference/api/#servertype","title":"ServerType","text":"

Underlying type: string

ServerType is the type of server, could be control for control nodes or default (empty string) for everything else

Validation: - Enum: [control]

Appears in: - ServerSpec

"},{"location":"reference/api/#switch","title":"Switch","text":"

Switch is the Schema for the switches API

All switches should always have 1 labels defined: wiring.githedgehog.com/rack. It represents name of the rack it belongs to.

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string Switch metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec SwitchSpec Spec is desired state of the switch status SwitchStatus Status is the observed state of the switch"},{"location":"reference/api/#switchgroup","title":"SwitchGroup","text":"

SwitchGroup is the marker API object to group switches together, switch can belong to multiple groups

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string SwitchGroup metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec SwitchGroupSpec Spec is the desired state of the SwitchGroup status SwitchGroupStatus Status is the observed state of the SwitchGroup"},{"location":"reference/api/#switchgroupspec","title":"SwitchGroupSpec","text":"

SwitchGroupSpec defines the desired state of SwitchGroup

Appears in: - SwitchGroup

"},{"location":"reference/api/#switchgroupstatus","title":"SwitchGroupStatus","text":"

SwitchGroupStatus defines the observed state of SwitchGroup

Appears in: - SwitchGroup

"},{"location":"reference/api/#switchredundancy","title":"SwitchRedundancy","text":"

SwitchRedundancy is the switch redundancy configuration which includes name of the redundancy group switch belongs to and its type, used both for MCLAG and ESLAG connections. It defines how redundancy will be configured and handled on the switch as well as which connection types will be available. If not specified, switch will not be part of any redundancy group. If name isn't empty, type must be specified as well and name should be the same as one of the SwitchGroup objects.

Appears in: - SwitchSpec

Field Description Default Validation group string Group is the name of the redundancy group switch belongs to type RedundancyType Type is the type of the redundancy group, could be mclag or eslag"},{"location":"reference/api/#switchrole","title":"SwitchRole","text":"

Underlying type: string

SwitchRole is the role of the switch, could be spine, server-leaf or border-leaf or mixed-leaf

Validation: - Enum: [spine server-leaf border-leaf mixed-leaf virtual-edge]

Appears in: - SwitchSpec

"},{"location":"reference/api/#switchspec","title":"SwitchSpec","text":"

SwitchSpec defines the desired state of Switch

Appears in: - Switch

Field Description Default Validation role SwitchRole Role is the role of the switch, could be spine, server-leaf or border-leaf or mixed-leaf Enum: [spine server-leaf border-leaf mixed-leaf virtual-edge] Required: {} description string Description is a description of the switch profile string Profile is the profile of the switch, name of the SwitchProfile object to be used for this switch, currently not used by the Fabric location Location Location is the location of the switch, it is used to generate the location UUID and location signature locationSig LocationSig LocationSig is the location signature for the switch groups string array Groups is a list of switch groups the switch belongs to redundancy SwitchRedundancy Redundancy is the switch redundancy configuration including name of the redundancy group switch belongs to and its type, used both for MCLAG and ESLAG connections vlanNamespaces string array VLANNamespaces is a list of VLAN namespaces the switch is part of, their VLAN ranges could not overlap asn integer ASN is the ASN of the switch ip string IP is the IP of the switch that could be used to access it from other switches and control nodes in the Fabric vtepIP string VTEPIP is the VTEP IP of the switch protocolIP string ProtocolIP is used as BGP Router ID for switch configuration portGroupSpeeds object (keys:string, values:string) PortGroupSpeeds is a map of port group speeds, key is the port group name, value is the speed, such as '\"2\": 10G' portSpeeds object (keys:string, values:string) PortSpeeds is a map of port speeds, key is the port name, value is the speed portBreakouts object (keys:string, values:string) PortBreakouts is a map of port breakouts, key is the port name, value is the breakout configuration, such as \"1/55: 4x25G\""},{"location":"reference/api/#switchstatus","title":"SwitchStatus","text":"

SwitchStatus defines the observed state of Switch

Appears in: - Switch

"},{"location":"reference/api/#switchtoswitchlink","title":"SwitchToSwitchLink","text":"

SwitchToSwitchLink defines the switch-to-switch link

Appears in: - ConnMCLAGDomain - ConnVPCLoopback

Field Description Default Validation switch1 BasePortName Switch1 is the first switch side of the connection switch2 BasePortName Switch2 is the second switch side of the connection"},{"location":"reference/api/#vlannamespace","title":"VLANNamespace","text":"

VLANNamespace is the Schema for the vlannamespaces API

Field Description Default Validation apiVersion string wiring.githedgehog.com/v1alpha2 kind string VLANNamespace metadata ObjectMeta Refer to Kubernetes API documentation for fields of metadata. spec VLANNamespaceSpec Spec is the desired state of the VLANNamespace status VLANNamespaceStatus Status is the observed state of the VLANNamespace"},{"location":"reference/api/#vlannamespacespec","title":"VLANNamespaceSpec","text":"

VLANNamespaceSpec defines the desired state of VLANNamespace

Appears in: - VLANNamespace

Field Description Default Validation ranges VLANRange array Ranges is a list of VLAN ranges to be used in this namespace, couldn't overlap between each other and with Fabric reserved VLAN ranges MaxItems: 20 MinItems: 1"},{"location":"reference/api/#vlannamespacestatus","title":"VLANNamespaceStatus","text":"

VLANNamespaceStatus defines the observed state of VLANNamespace

Appears in: - VLANNamespace

"},{"location":"reference/cli/","title":"Fabric CLI","text":"

Under construction.

Currently Fabric CLI is represented by a kubectl plugin kubectl-fabric automatically installed on the Control Node. It is a wrapper around kubectl and Kubernetes client which allows to manage Fabric resources in a more convenient way. Fabric CLI only provides a subset of the functionality available via Fabric API and is focused on simplifying objects creation and some manipulation with the already existing objects while main get/list/update operations are expected to be done using kubectl.

core@control-1 ~ $ kubectl fabric\nNAME:\n   hhfctl - Hedgehog Fabric user client\n\nUSAGE:\n   hhfctl [global options] command [command options] [arguments...]\n\nVERSION:\n   v0.23.0\n\nCOMMANDS:\n   vpc                VPC commands\n   switch, sw, agent  Switch/Agent commands\n   connection, conn   Connection commands\n   switchgroup, sg    SwitchGroup commands\n   external           External commands\n   help, h            Shows a list of commands or help for one command\n\nGLOBAL OPTIONS:\n   --verbose, -v  verbose output (includes debug) (default: true)\n   --help, -h     show help\n   --version, -V  print the version\n
"},{"location":"reference/cli/#vpc","title":"VPC","text":"

Create VPC named vpc-1 with subnet 10.0.1.0/24 and VLAN 1001 with DHCP enabled and DHCP range starting from 10.0.1.10 (optional):

core@control-1 ~ $ kubectl fabric vpc create --name vpc-1 --subnet 10.0.1.0/24 --vlan 1001 --dhcp --dhcp-start 10.0.1.10\n

Attach previously created VPC to the server server-01 (which is connected to the Fabric using the server-01--mclag--leaf-01--leaf-02 Connection):

core@control-1 ~ $ kubectl fabric vpc attach --vpc-subnet vpc-1/default --connection server-01--mclag--leaf-01--leaf-02\n

To peer VPC with another VPC (e.g. vpc-2) use the following command:

core@control-1 ~ $ kubectl fabric vpc peer --vpc vpc-1 --vpc vpc-2\n
"},{"location":"release-notes/","title":"Release notes","text":""},{"location":"release-notes/#alpha-6","title":"Alpha-6","text":""},{"location":"release-notes/#observability","title":"Observability","text":""},{"location":"release-notes/#telemetry-prometheus-exporter","title":"Telemetry - Prometheus Exporter","text":""},{"location":"release-notes/#logging","title":"Logging","text":""},{"location":"release-notes/#agent-status-api-enhancements","title":"Agent Status API Enhancements","text":""},{"location":"release-notes/#networking-enhancements","title":"Networking enhancements","text":""},{"location":"release-notes/#other-improvements","title":"Other improvements","text":""},{"location":"release-notes/#bugs-fixed","title":"Bugs fixed","text":""},{"location":"release-notes/#alpha-5","title":"Alpha-5","text":""},{"location":"release-notes/#open-source","title":"Open Source","text":""},{"location":"release-notes/#dhcppxe-boot-support-for-multi-homed-connections","title":"DHCP/PXE boot support for multi-homed connections","text":""},{"location":"release-notes/#improvements","title":"Improvements","text":""},{"location":"release-notes/#alpha-4","title":"Alpha-4","text":""},{"location":"release-notes/#documentation","title":"Documentation","text":""},{"location":"release-notes/#host-connectivity-dual-homing-improvements","title":"Host connectivity dual homing improvements","text":""},{"location":"release-notes/#improved-vpc-security-policy-better-zero-trust","title":"Improved VPC security policy - better Zero Trust","text":""},{"location":"release-notes/#static-external-connection","title":"Static External Connection","text":""},{"location":"release-notes/#internal-improvements","title":"Internal Improvements","text":""},{"location":"release-notes/#known-issues","title":"Known Issues","text":""},{"location":"release-notes/#alpha-3","title":"Alpha-3","text":""},{"location":"release-notes/#sonic-support","title":"SONiC support","text":""},{"location":"release-notes/#multiple-ipv4-namespaces","title":"Multiple IPv4 namespaces","text":""},{"location":"release-notes/#hedgehog-fabric-dhcp-and-ipam-service","title":"Hedgehog Fabric DHCP and IPAM Service","text":""},{"location":"release-notes/#hedgehog-fabric-ntp-service","title":"Hedgehog Fabric NTP Service","text":""},{"location":"release-notes/#staticexternal-connections","title":"StaticExternal connections","text":""},{"location":"release-notes/#dhcp-relay-to-3rd-party-dhcp-service","title":"DHCP Relay to 3rd party DHCP service","text":"

Support for 3rd party DHCP server (DHCP Relay config) through the API

"},{"location":"release-notes/#alpha-2","title":"Alpha-2","text":""},{"location":"release-notes/#controller","title":"Controller","text":"

A single controller. No controller redundancy.

"},{"location":"release-notes/#controller-connectivity","title":"Controller connectivity","text":"

For CLOS/LEAF-SPINE fabrics, it is recommended that the controller connects to one or more leaf switches in the fabric on front-facing data ports. Connection to two or more leaf switches is recommended for redundancy and performance. No port break-out functionality is supported for controller connectivity.

Spine controller connectivity is not supported.

For Collapsed Core topology, the controller can connect on front-facing data ports, as described above, or on management ports. Note that every switch in the collapsed core topology must be connected to the controller.

Management port connectivity can also be supported for CLOS/LEAF-SPINE topology but requires all switches connected to the controllers via management ports. No chain booting is possible for this configuration.

"},{"location":"release-notes/#controller-requirements","title":"Controller requirements","text":""},{"location":"release-notes/#chain-booting","title":"Chain booting","text":"

Switches not directly connecting to the controllers can chain boot via the data network.

"},{"location":"release-notes/#topology-support","title":"Topology support","text":"

CLOS/LEAF-SPINE and Collapsed Core topologies are supported.

"},{"location":"release-notes/#leaf-roles-for-clos-topology","title":"LEAF Roles for CLOS topology","text":"

server leaf, border leaf, and mixed leaf modes are supported.

"},{"location":"release-notes/#collapsed-core-topology","title":"Collapsed Core Topology","text":"

Two ToR/LEAF switches with MCLAG server connection.

"},{"location":"release-notes/#server-multihoming","title":"Server multihoming","text":"

MCLAG-only.

"},{"location":"release-notes/#device-support","title":"Device support","text":""},{"location":"release-notes/#leafs","title":"LEAFs","text":""},{"location":"release-notes/#spines","title":"SPINEs","text":""},{"location":"release-notes/#underlay-configuration","title":"Underlay configuration:","text":"

Port speed, port group speed, port breakouts are configurable through the API

"},{"location":"release-notes/#vpc-overlay-implementation","title":"VPC (overlay) Implementation","text":"

VXLAN-based BGP eVPN.

"},{"location":"release-notes/#multi-subnet-vpcs","title":"Multi-subnet VPCs","text":"

A VPC consists of subnets, each with a user-specified VLAN for external host/server connectivity.

"},{"location":"release-notes/#multiple-ip-address-namespaces","title":"Multiple IP address namespaces","text":"

Multiple IP address namespaces are supported per fabric. Each VPC belongs to the corresponding IPv4 namespace. There are no subnet overlaps within a single IPv4 namespace. IP address namespaces can mutually overlap.

"},{"location":"release-notes/#vlan-namespace","title":"VLAN Namespace","text":"

VLAN Namespaces guarantee the uniqueness of VLANs for a set of participating devices. Each switch belongs to a list of VLAN namespaces with non-overlapping VLAN ranges. Each VPC belongs to the VLAN namespace. There are no VLAN overlaps within a single VLAN namespace.

This feature is useful when multiple VM-management domains (like separate VMware clusters connect to the fabric).

"},{"location":"release-notes/#switch-groups","title":"Switch Groups","text":"

Each switch belongs to a list of switch groups used for identifying redundancy groups for things like external connectivity.

"},{"location":"release-notes/#mutual-vpc-peering","title":"Mutual VPC Peering","text":"

VPC peering is supported and possible between a pair of VPCs that belong to the same IPv4 and VLAN namespaces.

"},{"location":"release-notes/#external-vpc-peering","title":"External VPC Peering","text":"

VPC peering provides the means of peering with external networking devices (edge routers, firewalls, or data center interconnects). VPC egress/ingress is pinned to a specific group of the border or mixed leaf switches. Multiple \u201cexternal systems\u201d with multiple devices/links in each of them are supported.

The user controls what subnets/prefixes to import and export from/to the external system.

No NAT function is supported for external peering.

"},{"location":"release-notes/#host-connectivity","title":"Host connectivity","text":"

Servers can be attached as Unbundled, Bundled (LAG) and MCLAG

"},{"location":"release-notes/#dhcp-service","title":"DHCP Service","text":"

VPC is provided with an optional DHCP service with simple IPAM

"},{"location":"release-notes/#local-vpc-peering-loopbacks","title":"Local VPC peering loopbacks","text":"

To enable local inter-vpc peering that allows routing of traffic between VPCs, local loopbacks are required to overcome silicon limitations.

"},{"location":"release-notes/#scale","title":"Scale","text":""},{"location":"release-notes/#software-versions","title":"Software versions","text":""},{"location":"release-notes/#known-limitations","title":"Known Limitations","text":""},{"location":"release-notes/#alpha-1","title":"Alpha-1","text":""},{"location":"troubleshooting/overview/","title":"Troubleshooting","text":"

Under construction.

"},{"location":"user-guide/connections/","title":"Connections","text":"

The Connection object represents logical and physical connections between any devices in the Fabric (Switch, Server and External objects). It's needed to define all connections between the devices in the Wiring Diagram.

There are multiple types of connections.

"},{"location":"user-guide/connections/#server-connections-user-facing","title":"Server connections (user-facing)","text":"

Server connections are used to connect workload servers to the switches.

"},{"location":"user-guide/connections/#unbundled","title":"Unbundled","text":"

Unbundled server connections are used to connect servers to a single switch using a single port.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: server-4--unbundled--s5248-02\n  namespace: default\nspec:\n  unbundled:\n    link: # Defines a single link between a server and a switch\n      server:\n        port: server-4/enp2s1\n      switch:\n        port: s5248-02/Ethernet3\n
"},{"location":"user-guide/connections/#bundled","title":"Bundled","text":"

Bundled server connections are used to connect servers to a single switch using multiple ports (port channel, LAG).

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: server-3--bundled--s5248-01\n  namespace: default\nspec:\n  bundled:\n    links: # Defines multiple links between a single server and a single switch\n    - server:\n        port: server-3/enp2s1\n      switch:\n        port: s5248-01/Ethernet3\n    - server:\n        port: server-3/enp2s2\n      switch:\n        port: s5248-01/Ethernet4\n
"},{"location":"user-guide/connections/#mclag","title":"MCLAG","text":"

MCLAG server connections are used to connect servers to a pair of switches using multiple ports (Dual-homing). Switches should be configured as an MCLAG pair which requires them to be in a single redundancy group of type mclag and a Connection with type mclag-domain between them. MCLAG switches should also have the same spec.ASN and spec.VTEPIP.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: server-1--mclag--s5248-01--s5248-02\n  namespace: default\nspec:\n  mclag:\n    links: # Defines multiple links between a single server and a pair of switches\n    - server:\n        port: server-1/enp2s1\n      switch:\n        port: s5248-01/Ethernet1\n    - server:\n        port: server-1/enp2s2\n      switch:\n        port: s5248-02/Ethernet1\n
"},{"location":"user-guide/connections/#eslag","title":"ESLAG","text":"

ESLAG server connections are used to connect servers to the 2-4 switches using multiple ports (Multi-homing). Switches should belong to the same redundancy group with type eslag, but contrary to the MCLAG case, no other configuration is required.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: server-1--eslag--s5248-01--s5248-02\n  namespace: default\nspec:\n  eslag:\n    links: # Defines multiple links between a single server and a 2-4 switches\n    - server:\n        port: server-1/enp2s1\n      switch:\n        port: s5248-01/Ethernet1\n    - server:\n        port: server-1/enp2s2\n      switch:\n        port: s5248-02/Ethernet1\n
"},{"location":"user-guide/connections/#switch-connections-fabric-facing","title":"Switch connections (fabric-facing)","text":"

Switch connections are used to connect switches to each other and provide any needed \"service\" connectivity to implement the Fabric features.

"},{"location":"user-guide/connections/#fabric","title":"Fabric","text":"

A Fabric Connections is used between specific spine and leaf, it covers all actual wires between a single pair.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: s5232-01--fabric--s5248-01\n  namespace: default\nspec:\n  fabric:\n    links: # Defines multiple links between a spine-leaf pair of switches with IP addresses\n    - leaf:\n        ip: 172.30.30.1/31\n        port: s5248-01/Ethernet48\n      spine:\n        ip: 172.30.30.0/31\n        port: s5232-01/Ethernet0\n    - leaf:\n        ip: 172.30.30.3/31\n        port: s5248-01/Ethernet56\n      spine:\n        ip: 172.30.30.2/31\n        port: s5232-01/Ethernet4\n
"},{"location":"user-guide/connections/#mclag-domain","title":"MCLAG-Domain","text":"

MCLAG-Domain connections define a pair of MCLAG switches with Session and Peer link between them. Switches should be configured as an MCLAG, pair which requires them to be in a single redundancy group of type mclag and Connection with type mclag-domain between them. MCLAG switches should also have the same spec.ASN and spec.VTEPIP.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: s5248-01--mclag-domain--s5248-02\n  namespace: default\nspec:\n  mclagDomain:\n    peerLinks: # Defines multiple links between a pair of MCLAG switches for Peer link\n    - switch1:\n        port: s5248-01/Ethernet72\n      switch2:\n        port: s5248-02/Ethernet72\n    - switch1:\n        port: s5248-01/Ethernet73\n      switch2:\n        port: s5248-02/Ethernet73\n    sessionLinks: # Defines multiple links between a pair of MCLAG switches for Session link\n    - switch1:\n        port: s5248-01/Ethernet74\n      switch2:\n        port: s5248-02/Ethernet74\n    - switch1:\n        port: s5248-01/Ethernet75\n      switch2:\n        port: s5248-02/Ethernet75\n
"},{"location":"user-guide/connections/#vpc-loopback","title":"VPC-Loopback","text":"

VPC-Loopback connections are required in order to implement a workaround for the local VPC peering (when both VPC are attached to the same switch), which is caused by a hardware limitation of the currently supported switches.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: s5248-01--vpc-loopback\n  namespace: default\nspec:\n  vpcLoopback:\n    links: # Defines multiple loopbacks on a single switch\n    - switch1:\n        port: s5248-01/Ethernet16\n      switch2:\n        port: s5248-01/Ethernet17\n    - switch1:\n        port: s5248-01/Ethernet18\n      switch2:\n        port: s5248-01/Ethernet19\n
"},{"location":"user-guide/connections/#management","title":"Management","text":"

Management connections define connections to the Control Node.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: control-1--mgmt--s5248-01-front\n  namespace: default\nspec:\n  management:\n    link: # Defines a single link between a control node and a switch\n      server:\n        ip: 172.30.20.0/31\n        port: control-1/enp2s1\n      switch:\n        ip: 172.30.20.1/31\n        port: s5248-01/Ethernet0\n
"},{"location":"user-guide/connections/#connecting-fabric-to-the-outside-world","title":"Connecting Fabric to the outside world","text":"

Connections in this section provide connectivity to the outside world. For example, they can be connections to the Internet, to other networks, or to some other systems such as DHCP, NTP, LMA, or AAA services.

"},{"location":"user-guide/connections/#staticexternal","title":"StaticExternal","text":"

StaticExternal connections provide a simple way to connect things like DHCP servers directly to the Fabric by connecting them to specific switch ports.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: third-party-dhcp-server--static-external--s5248-04\n  namespace: default\nspec:\n  staticExternal:\n    link:\n      switch:\n        port: s5248-04/Ethernet1 # Switch port to use\n        ip: 172.30.50.5/24 # IP address that will be assigned to the switch port\n        vlan: 1005 # Optional VLAN ID to use for the switch port; if 0, no VLAN is configured\n        subnets: # List of subnets to route to the switch port using static routes and next hop\n          - 10.99.0.1/24\n          - 10.199.0.100/32\n        nextHop: 172.30.50.1 # Next hop IP address to use when configuring static routes for the \"subnets\" list\n

Additionally, it's possible to configure StaticExternal within the VPC to provide access to the third-party resources within a specific VPC, with the rest of the YAML configuration remaining unchanged.

...\nspec:\n  staticExternal:\n    withinVPC: vpc-1 # VPC name to attach the static external to\n    link:\n      ...\n
"},{"location":"user-guide/connections/#external","title":"External","text":"

Connection to external systems, such as edge/provider routers using BGP peering and configuring Inbound/Outbound communities as well as granularly controlling what gets advertised and which routes are accepted.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: s5248-03--external--5835\n  namespace: default\nspec:\n  external:\n    link: # Defines a single link between a switch and an external system\n      switch:\n        port: s5248-03/Ethernet3\n
"},{"location":"user-guide/devices/","title":"Switches and Servers","text":"

All devices in the Hedgehog Fabric are divided into two groups: switches and servers, represented by the corresponding Switch and Server objects in the API. These objects are needed to define all participants of the Fabric and their roles in the Wiring Diagram as well as Connections between them.

"},{"location":"user-guide/devices/#switches","title":"Switches","text":"

Switches are the main building blocks of the Fabric. They are represented by Switch objects in the API. These objects consist of basic metadata like name, description, location, role, as well as port group speeds, port breakouts, ASN, IP addresses, and more.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Switch\nmetadata:\n  name: s5248-01\n  namespace: default\nspec:\n  asn: 65101 # ASN of the switch\n  description: leaf-1\n  ip: 172.30.10.100/32 # Switch IP that will be accessible from the Control Node\n  location:\n    location: gen--default--s5248-01\n  locationSig:\n    sig: <undefined>\n    uuidSig: <undefined>\n  portBreakouts: # Configures port breakouts for the switch\n    1/55: 4x25G\n  portGroupSpeeds: # Configures port group speeds for the switch\n    \"1\": 10G\n    \"2\": 10G\n  protocolIP: 172.30.11.100/32 # Used as BGP router ID\n  role: server-leaf # Role of the switch, one of server-leaf, border-leaf and mixed-leaf\n  vlanNamespaces: # Defines which VLANs could be used to attach servers\n  - default\n  vtepIP: 172.30.12.100/32\n  groups: # Defines which groups the switch belongs to\n  - some-group\n  redundancy: # Optional field to define that switch belongs to the redundancy group\n    group: eslag-1 # Name of the redundancy group\n    type: eslag # Type of the redundancy group, one of mclag or eslag\n

The SwitchGroup is just a marker at that point and doesn't have any configuration options.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: SwitchGroup\nmetadata:\n  name: border\n  namespace: default\nspec: {}\n
"},{"location":"user-guide/devices/#redundancy-groups","title":"Redundancy Groups","text":"

Redundancy groups are used to define the redundancy between switches. It's a regular SwitchGroup used by multiple switches and currently it could be MCLAG or ESLAG (EVPN MH / ESI). A switch can only belong to a single redundancy group.

MCLAG is only supported for pair of switches and ESLAG is supported for up to 4 switches. Multiple types of redundancy groups can be used in the fabric simultaneously.

Connections with types mclag and eslag are used to define the servers connections to switches. They are only supported if the switch belongs to a redundancy group with the corresponding type.

In order to define a MCLAG or ESLAG redundancy group, you need to create a SwitchGroup object and assign it to the switches using the redundancy field.

Example of switch configured for ESLAG:

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: SwitchGroup\nmetadata:\n  name: eslag-1\n  namespace: default\nspec: {}\n---\napiVersion: wiring.githedgehog.com/v1alpha2\nkind: Switch\nmetadata:\n  name: s5248-03\n  namespace: default\nspec:\n  ...\n  redundancy:\n    group: eslag-1\n    type: eslag\n  ...\n

And example of switch configured for MCLAG:

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: SwitchGroup\nmetadata:\n  name: mclag-1\n  namespace: default\nspec: {}\n---\napiVersion: wiring.githedgehog.com/v1alpha2\nkind: Switch\nmetadata:\n  name: s5248-01\n  namespace: default\nspec:\n  ...\n  redundancy:\n    group: mclag-1\n    type: mclag\n  ...\n

In case of MCLAG it's required to have a special connection with type mclag-domain that defines the peer and session links between switches. For more details, see Connections.

"},{"location":"user-guide/devices/#servers","title":"Servers","text":"

Servers include both control nodes and user's workload servers.

Control Node:

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Server\nmetadata:\n  name: control-1\n  namespace: default\nspec:\n  type: control # Type of the server, one of control or \"\" (empty) for regular workload server\n

Regular workload server:

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Server\nmetadata:\n  name: server-1\n  namespace: default\nspec:\n  description: MH s5248-01/E1 s5248-02/E1\n
"},{"location":"user-guide/external/","title":"External Peering","text":"

Hedgehog Fabric uses the Border Leaf concept to exchange VPC routes outside the Fabric and provide L3 connectivity. The External Peering feature allows you to set up an external peering endpoint and to enforce several policies between internal and external endpoints.

Note

Hedgehog Fabric does not operate Edge side devices.

"},{"location":"user-guide/external/#overview","title":"Overview","text":"

Traffic exits from the Fabric on Border Leaves that are connected with Edge devices. Border Leaves are suitable to terminate L2VPN connections, to distinguish VPC L3 routable traffic towards Edge devices, and to land VPC servers. Border Leaves (or Borders) can connect to several Edge devices.

Note

External Peering is only available on the switch devices that are capable for sub-interfaces.

"},{"location":"user-guide/external/#connect-border-leaf-to-edge-device","title":"Connect Border Leaf to Edge device","text":"

In order to distinguish VPC traffic, an Edge device should be able to: - Set up BGP IPv4 to advertise and receive routes from the Fabric - Connect to a Fabric Border Leaf over VLAN - Be able to mark egress routes towards the Fabric with BGP Communities - Be able to filter ingress routes from the Fabric by BGP Communities

All other filtering and processing of L3 Routed Fabric traffic should be done on the Edge devices.

"},{"location":"user-guide/external/#control-plane","title":"Control Plane","text":"

Fabric is sharing VPC routes with Edge devices via BGP. Peering is done over VLAN in IPv4 Unicast AFI/SAFI.

"},{"location":"user-guide/external/#data-plane","title":"Data Plane","text":"

VPC L3 routable traffic will be tagged with VLAN and sent to Edge device. Later processing of VPC traffic (NAT, PBR, etc) should happen on Edge devices.

"},{"location":"user-guide/external/#vpc-access-to-edge-device","title":"VPC access to Edge device","text":"

Each VPC within the Fabric can be allowed to access Edge devices. Additional filtering can be applied to the routes that the VPC can export to Edge devices and import from the Edge devices.

"},{"location":"user-guide/external/#api-and-implementation","title":"API and implementation","text":""},{"location":"user-guide/external/#external","title":"External","text":"

General configuration starts with the specification of External objects. Each object of External type can represent a set of Edge devices, or a single BGP instance on Edge device, or any other united Edge entities that can be described with the following configuration:

Each External should be bound to some VPC IP Namespace, otherwise prefixes overlap may happen.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: External\nmetadata:\n  name: default--5835\nspec:\n  ipv4Namespace: # VPC IP Namespace\n  inboundCommunity: # BGP Standard Community of routes from Edge devices\n  outboundCommunity: # BGP Standard Community required to be assigned on prefixes advertised from Fabric\n
"},{"location":"user-guide/external/#connection","title":"Connection","text":"

A Connection of type external is used to identify the switch port on Border leaf that is cabled with an Edge device.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: # specified or generated\nspec:\n  external:\n    link:\n      switch:\n        port: # SwitchName/EthernetXXX\n
"},{"location":"user-guide/external/#external-attachment","title":"External Attachment","text":"

External Attachment defines BGP Peering and traffic connectivity between a Border leaf and External. Attachments are bound to a Connection with type external and they specify an optional vlan that will be used to segregate particular Edge peering.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: ExternalAttachment\nmetadata:\n  name: #\nspec:\n  connection: # Name of the Connection with type external\n  external: # Name of the External to pick config\n  neighbor:\n    asn: # Edge device ASN\n    ip: # IP address of Edge device to peer with\n  switch:\n    ip: # IP address on the Border Leaf to set up BGP peering\n    vlan: # VLAN (optional) ID to tag control and data traffic, use 0 for untagged\n

Several External Attachment can be configured for the same Connection but for different vlan.

"},{"location":"user-guide/external/#external-vpc-peering","title":"External VPC Peering","text":"

To allow a specific VPC to have access to Edge devices, bind the VPC to a specific External object. To do so, define an External Peering object.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: ExternalPeering\nmetadata:\n  name: # Name of ExternalPeering\nspec:\n  permit:\n    external:\n      name: # External Name\n      prefixes: # List of prefixes (routes) to be allowed to pick up from External\n      - # IPv4 prefix\n    vpc:\n      name: # VPC Name\n      subnets: # List of VPC subnets name to be allowed to have access to External (Edge)\n      - # Name of the subnet within VPC\n

Prefixes is the list of subnets to permit from the External to the VPC. It matches any prefix length less than or equal to 32, effectively permitting all prefixes within the specified one. Use 0.0.0.0/0 for any route, including the default route.

This example allows any IPv4 prefix that came from External:

spec:\n  permit:\n    external:\n      name: ###\n      prefixes:\n      - prefix: 0.0.0.0/0 # Any route will be allowed including default route\n

This example allows all prefixes that match the default route, with any prefix length:

spec:\n  permit:\n    external:\n      name: ###\n      prefixes:\n      - prefix: 77.0.0.0/8 # Any route that belongs to the specified prefix is allowed (such as 77.0.0.0/8 or 77.1.2.0/24)\n
"},{"location":"user-guide/external/#examples","title":"Examples","text":"

This example shows how to peer with the External object with name HedgeEdge, given a Fabric VPC with name vpc-1 on the Border Leaf switchBorder that has a cable connecting it to an Edge device on the port Ethernet42. Specifying vpc-1 is required to receive any prefixes advertised from the External.

"},{"location":"user-guide/external/#fabric-api-configuration","title":"Fabric API configuration","text":""},{"location":"user-guide/external/#external_1","title":"External","text":"
# hhfctl external create --name HedgeEdge --ipns default --in 65102:5000 --out 5000:65102\n
apiVersion: vpc.githedgehog.com/v1alpha2\nkind: External\nmetadata:\n  name: HedgeEdge\n  namespace: default\nspec:\n  inboundCommunity: 65102:5000\n  ipv4Namespace: default\n  outboundCommunity: 5000:65102\n
"},{"location":"user-guide/external/#connection_1","title":"Connection","text":"

Connection should be specified in the wiring diagram.

###\n### switchBorder--external--HedgeEdge\n###\napiVersion: wiring.githedgehog.com/v1alpha2\nkind: Connection\nmetadata:\n  name: switchBorder--external--HedgeEdge\nspec:\n  external:\n    link:\n      switch:\n        port: switchBorder/Ethernet42\n
"},{"location":"user-guide/external/#externalattachment","title":"ExternalAttachment","text":"

Specified in wiring diagram

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: ExternalAttachment\nmetadata:\n  name: switchBorder--HedgeEdge\nspec:\n  connection: switchBorder--external--HedgeEdge\n  external: HedgeEdge\n  neighbor:\n    asn: 65102\n    ip: 100.100.0.6\n  switch:\n    ip: 100.100.0.1/24\n    vlan: 100\n
"},{"location":"user-guide/external/#externalpeering","title":"ExternalPeering","text":"
apiVersion: vpc.githedgehog.com/v1alpha2\nkind: ExternalPeering\nmetadata:\n  name: vpc-1--HedgeEdge\nspec:\n  permit:\n    external:\n      name: HedgeEdge\n      prefixes:\n      - prefix: 0.0.0.0/0\n    vpc:\n      name: vpc-1\n      subnets:\n      - default\n
"},{"location":"user-guide/external/#example-edge-side-bgp-configuration-based-on-sonic-os","title":"Example Edge side BGP configuration based on SONiC OS","text":"

Warning

Hedgehog does not recommend using the following configuration for production. It is only provided as an example of Edge Peer configuration.

Interface configuration:

interface Ethernet2.100\n encapsulation dot1q vlan-id 100\n description switchBorder--Ethernet42\n no shutdown\n ip vrf forwarding VrfHedge\n ip address 100.100.0.6/24\n

BGP configuration:

!\nrouter bgp 65102 vrf VrfHedge\n log-neighbor-changes\n timers 60 180\n !\n address-family ipv4 unicast\n  maximum-paths 64\n  maximum-paths ibgp 1\n  import vrf VrfPublic\n !\n neighbor 100.100.0.1\n  remote-as 65103\n  !\n  address-family ipv4 unicast\n   activate\n   route-map HedgeIn in\n   route-map HedgeOut out\n   send-community both\n !\n

Route Map configuration:

route-map HedgeIn permit 10\n match community Hedgehog\n!\nroute-map HedgeOut permit 10\n set community 65102:5000\n!\n\nbgp community-list standard HedgeIn permit 5000:65102\n
"},{"location":"user-guide/harvester/","title":"Using VPCs with Harvester","text":"

This section contains an example of how Hedgehog Fabric can be used with Harvester or any hypervisor on the servers connected to Fabric. It assumes that you have already installed Fabric and have some servers running Harvester attached to it.

You need to define a Server object for each server running Harvester and a Connection object for each server connection to the switches.

You can have multiple VPCs created and attached to the Connections to the servers to make them available to the VMs in Harvester or any other hypervisor.

"},{"location":"user-guide/harvester/#configure-harvester","title":"Configure Harvester","text":""},{"location":"user-guide/harvester/#add-a-cluster-network","title":"Add a Cluster Network","text":"

From the \"Cluster Networks/Configs\" side menu, create a new Cluster Network.

Here is a cleaned-up version of what the CRD looks like:

apiVersion: network.harvesterhci.io/v1beta1\nkind: ClusterNetwork\nmetadata:\n  name: testnet\n
"},{"location":"user-guide/harvester/#add-a-network-config","title":"Add a Network Config","text":"

Click \"Create Network Config\". Add your connections and select the bonding type.

The resulting CRD (cleaned up) looks like the following:

apiVersion: network.harvesterhci.io/v1beta1\nkind: VlanConfig\nmetadata:\n  name: testconfig\n  labels:\n    network.harvesterhci.io/clusternetwork: testnet\nspec:\n  clusterNetwork: testnet\n  uplink:\n    bondOptions:\n      miimon: 100\n      mode: 802.3ad\n    linkAttributes:\n      txQLen: -1\n    nics:\n      - enp5s0f0\n      - enp3s0f1\n
"},{"location":"user-guide/harvester/#add-vlan-based-vm-networks","title":"Add VLAN based VM Networks","text":"

Browse over to \"VM Networks\" and add one network for each VLAN you want to support. Assign them to the cluster network.

Here is what the CRDs will look like for both VLANs:

apiVersion: k8s.cni.cncf.io/v1\nkind: NetworkAttachmentDefinition\nmetadata:\n  labels:\n    network.harvesterhci.io/clusternetwork: testnet\n    network.harvesterhci.io/ready: 'true'\n    network.harvesterhci.io/type: L2VlanNetwork\n    network.harvesterhci.io/vlan-id: '1001'\n  name: testnet1001\n  namespace: default\nspec:\n  config: >-\n    {\"cniVersion\":\"0.3.1\",\"name\":\"testnet1001\",\"type\":\"bridge\",\"bridge\":\"testnet-br\",\"promiscMode\":true,\"vlan\":1001,\"ipam\":{}}\n
apiVersion: k8s.cni.cncf.io/v1\nkind: NetworkAttachmentDefinition\nmetadata:\n  name: testnet1000\n  labels:\n    network.harvesterhci.io/clusternetwork: testnet\n    network.harvesterhci.io/ready: 'true'\n    network.harvesterhci.io/type: L2VlanNetwork\n    network.harvesterhci.io/vlan-id: '1000'\n    #  key: string\n  namespace: default\nspec:\n  config: >-\n    {\"cniVersion\":\"0.3.1\",\"name\":\"testnet1000\",\"type\":\"bridge\",\"bridge\":\"testnet-br\",\"promiscMode\":true,\"vlan\":1000,\"ipam\":{}}\n
"},{"location":"user-guide/harvester/#using-the-vpcs","title":"Using the VPCs","text":"

Now you can choose the new VM Networks when creating a VM in Harvester, and have them created as part of the VPC.

"},{"location":"user-guide/overview/","title":"Overview","text":"

This chapter gives an overview of the main features of Hedgehog Fabric and their usage.

"},{"location":"user-guide/shrink-expand/","title":"Fabric Shrink/Expand","text":"

This section provides a brief overview of how to add or remove switches within the fabric using Hedgehog Fabric API, and how to manage connections between them.

Manipulating API objects is done with the assumption that target devices are correctly cabeled and connected.

This article operates terms that can be found in the Hedgehog Concepts, the User Guide documentation, and the Fabric API reference.

"},{"location":"user-guide/shrink-expand/#add-a-switch-to-the-existing-fabric","title":"Add a switch to the existing fabric","text":"

To be added to the Hedgehog Fabric, a switch should have a corresponding Switch object. An example on how to define this object is available in the User Guilde.

Note

If theSwitch will be used in ESLAG or MCLAG groups, appropriate groups should exist. Redundancy groups should be specified in the Switch object before creation.

After the Switch object has been created, you can define and create dedicated device Connections. The types of the connections may differ based on the Switch role given to the device. For more details, refer to Connections section.

Note

If the switch is facing a Control Node Connection on the front-panel port, the switch port should be described in a Management connection.

Note

Switch devices should be booted in ONIE or HONIE installation mode to install SONiC OS and configure the Fabric Agent.

"},{"location":"user-guide/shrink-expand/#remove-a-switch-from-the-existing-fabric","title":"Remove a switch from the existing fabric","text":"

Before you decommission a switch from the Hedgehog Fabric, several preparation steps are necessary.

Warning

Currently the Wiring diagram used for initial deployment is saved in /var/lib/rancher/k3s/server/manifests/hh-wiring.yaml on the Control node. Fabric will sustain objects within the original wiring diagram. In order to remove any object, first remove the dedicated API objects from this file. It is recommended to reapply hh-wiring.yaml after changing its internals.

"},{"location":"user-guide/vpcs/","title":"VPCs and Namespaces","text":""},{"location":"user-guide/vpcs/#vpc","title":"VPC","text":"

A Virtual Private Cloud (VPC) is similar to a public cloud VPC. It provides an isolated private network for the resources with support for multiple subnets, each with user-provided VLANs and on-demand DHCP.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPC\nmetadata:\n  name: vpc-1\n  namespace: default\nspec:\n  ipv4Namespace: default # Limits to which subnets could be used by VPC to guarantee non-overlapping IPv4 ranges\n  vlanNamespace: default # Limits to which switches VPC could be attached to guarantee non-overlapping VLANs\n\n  defaultIsolated: true # Sets default behavior for the current VPC subnets to be isolated\n  defaultRestricted: true # Sets default behavior for the current VPC subnets to be restricted\n\n  subnets:\n    default: # Each subnet is named, \"default\" subnet isn't required, but actively used by CLI\n      dhcp:\n        enable: true # On-demand DHCP server\n        range: # Optionally, start/end range could be specified, otherwise all available IPs are used\n          start: 10.10.1.10\n          end: 10.10.1.99\n        pxeURL: tftp://10.10.10.99/bootfilename # PXEURL (optional) to identify the PXE server to use to boot hosts; HTTP query strings are not supported\n      subnet: 10.10.1.0/24 # User-defined subnet from ipv4 namespace\n      gateway: 10.10.1.1 # User-defined gateway (optional, default is .1)\n      vlan: 1001 # User-defined VLAN from VLAN namespace\n      isolated: true # Makes subnet isolated from other subnets within the VPC (doesn't affect VPC peering)\n      restricted: true # Causes all hosts in the subnet to be isolated from each other\n\n    thrird-party-dhcp: # Another subnet\n      dhcp:\n        relay: 10.99.0.100/24 # Use third-party DHCP server (DHCP relay configuration), access to it could be enabled using StaticExternal connection\n      subnet: \"10.10.2.0/24\"\n      vlan: 1002\n\n    another-subnet: # Minimal configuration is just a name, subnet and VLAN\n      subnet: 10.10.100.0/24\n      vlan: 1100\n\n  permit: # Defines which VPCs could communicate to each other, applied on top of subnets \"isolated\" flag (doesn't affect VPC peering)\n    - [subnet-1, subnet-2, subnet-3] # 1, 2 and 3 subnets could communicate to each other\n    - [subnet-4, subnet-5] # Possible to define multiple lists\n\n  staticRoutes: # Optional, static routes to be added to the VPC\n    - prefix: 10.100.0.0/24 # Destination prefix\n      nextHops: # Next hop IP addresses\n        - 10.200.0.0\n
"},{"location":"user-guide/vpcs/#isolated-and-restricted-subnets-permit-lists","title":"Isolated and restricted subnets, permit lists","text":"

Subnets can be isolated and restricted, with the ability to define permit lists to allow communication between specific isolated subnets. The permit list is applied on top of the isolated flag and doesn't affect VPC peering.

Isolated subnet means that the subnet has no connectivity with other subnets within the VPC, but it could still be allowed by permit lists.

Restricted subnet means that all hosts in the subnet are isolated from each other within the subnet.

A Permit list is defined as a list of subnets that could communicate with each other.

"},{"location":"user-guide/vpcs/#third-party-dhcp-server","title":"Third-party DHCP server","text":"

In case you use a third-party DHCP server by configuring spec.subnets.<subnet>.dhcp.relay, additional information is added to the DHCP packet forwarded to the DHCP server to make it possible to identify the VPC and subnet. This information is added under the RelayAgentInfo (option 82) in the DHCP packet. The relay sets two suboptions in the packet:

"},{"location":"user-guide/vpcs/#vpcattachment","title":"VPCAttachment","text":"

A VPCAttachment represents a specific VPC subnet assignment to the Connection object which means a binding between an exact server port and a VPC. It basically leads to the VPC being available on the specific server port(s) on a subnet VLAN.

VPC could be attached to a switch that is part of the VLAN namespace used by the VPC.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPCAttachment\nmetadata:\n  name: vpc-1-server-1--mclag--s5248-01--s5248-02\n  namespace: default\nspec:\n  connection: server-1--mclag--s5248-01--s5248-02 # Connection name representing the server port(s)\n  subnet: vpc-1/default # VPC subnet name\n  nativeVLAN: true # Optional, if true the port will be configured as a native VLAN port (untagged)\n
"},{"location":"user-guide/vpcs/#vpcpeering","title":"VPCPeering","text":"

A VPCPeering enables VPC-to-VPC connectivity. There are two types of VPC peering:

VPC peering is only possible between VPCs attached to the same IPv4 namespace.

"},{"location":"user-guide/vpcs/#local-vpc-peering","title":"Local VPC peering","text":"
apiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPCPeering\nmetadata:\n  name: vpc-1--vpc-2\n  namespace: default\nspec:\n  permit: # Defines a pair of VPCs to peer\n  - vpc-1: {} # Meaning all subnets of two VPCs will be able to communicate with each other\n    vpc-2: {} # See \"Subnet filtering\" for more advanced configuration\n
"},{"location":"user-guide/vpcs/#remote-vpc-peering","title":"Remote VPC peering","text":"
apiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPCPeering\nmetadata:\n  name: vpc-1--vpc-2\n  namespace: default\nspec:\n  permit:\n  - vpc-1: {}\n    vpc-2: {}\n  remote: border # Indicates a switch group to implement the peering on\n
"},{"location":"user-guide/vpcs/#subnet-filtering","title":"Subnet filtering","text":"

It's possible to specify which specific subnets of the peering VPCs could communicate to each other using the permit field.

```yaml\napiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPCPeering\nmetadata:\n  name: vpc-1--vpc-2\n  namespace: default\nspec:\n  permit: # subnet-1 and subnet-2 of vpc-1 could communicate to subnet-3 of vpc-2 as well as subnet-4 of vpc-2 could communicate to subnet-5 and subnet-6 of vpc-2\n  - vpc-1:\n      subnets: [subnet-1, subnet-2]\n    vpc-2:\n      subnets: [subnet-3]\n  - vpc-1:\n      subnets: [subnet-4]\n    vpc-2:\n      subnets: [subnet-5, subnet-6]\n
"},{"location":"user-guide/vpcs/#ipv4namespace","title":"IPv4Namespace","text":"

An IPv4Namespace defines non-overlapping VLAN ranges for attaching servers. Each switch belongs to a list of VLAN namespaces with non-overlapping VLAN ranges.

apiVersion: vpc.githedgehog.com/v1alpha2\nkind: IPv4Namespace\nmetadata:\n  name: default\n  namespace: default\nspec:\n  subnets: # List of the subnets that VPCs can pick their subnets from\n  - 10.10.0.0/16\n
"},{"location":"user-guide/vpcs/#vlannamespace","title":"VLANNamespace","text":"

A VLANNamespace defines non-overlapping IPv4 ranges for VPC subnets. Each VPC belongs to a specific IPv4 namespace.

apiVersion: wiring.githedgehog.com/v1alpha2\nkind: VLANNamespace\nmetadata:\n  name: default\n  namespace: default\nspec:\n  ranges: # List of VLAN ranges that VPCs can pick their subnet VLANs from\n  - from: 1000\n    to: 2999\n
"},{"location":"vlab/demo/","title":"Demo on VLAB","text":"

The goal of this demo is to show how to use VPCs, attach and peer them and run test connectivity between the servers. Examples are based on the default VLAB topology.

You can find instructions on how to setup VLAB in the Overview and Running VLAB sections.

"},{"location":"vlab/demo/#default-topology","title":"Default topology","text":"

The default topology is Spine-Leaf with 2 spines, 2 MCLAG leaves and 1 non-MCLAG leaf. Optionally, you can choose to run the default Collapsed Core topology using flag --fabric-mode collapsed-core (or -m collapsed-core) which only consists of 2 switches.

For more details on customizing topologies see the Running VLAB section.

In the default topology, the following Control Node and Switch VMs are created:

graph TD\n    CN[Control Node]\n\n    S1[Spine 1]\n    S2[Spine 2]\n\n    L1[MCLAG Leaf 1]\n    L2[MCLAG Leaf 2]\n    L3[Leaf 3]\n\n    CN --> L1\n    CN --> L2\n\n    S1 --> L1\n    S1 --> L2\n    S2 --> L2\n    S2 --> L3

As well as the following test servers:

graph TD\n    L1[MCLAG Leaf 1]\n    L2[MCLAG Leaf 2]\n    L3[Leaf 3]\n    L4[Leaf 4]\n\n    TS1[Test Server 1]\n    TS2[Test Server 2]\n    TS3[Test Server 3]\n    TS4[Test Server 4]\n    TS5[Test Server 5]\n    TS6[Test Server 6]\n\n    TS1 --> L1\n    TS1 --> L2\n\n    TS2 --> L1\n    TS2 --> L2\n\n    TS3 --> L1\n    TS4 --> L2\n\n    TS5 --> L3\n    TS6 --> L4
"},{"location":"vlab/demo/#creating-and-attaching-vpcs","title":"Creating and attaching VPCs","text":"

You can create and attach VPCs to the VMs using the kubectl fabric vpc command on the Control Node or outside of the cluster using the kubeconfig. For example, run the following commands to create 2 VPCs with a single subnet each, a DHCP server enabled with its optional IP address range start defined, and to attach them to some of the test servers:

core@control-1 ~ $ kubectl get conn | grep server\nserver-01--mclag--leaf-01--leaf-02   mclag          5h13m\nserver-02--mclag--leaf-01--leaf-02   mclag          5h13m\nserver-03--unbundled--leaf-01        unbundled      5h13m\nserver-04--bundled--leaf-02          bundled        5h13m\nserver-05--unbundled--leaf-03        unbundled      5h13m\nserver-06--bundled--leaf-03          bundled        5h13m\n\ncore@control-1 ~ $ kubectl fabric vpc create --name vpc-1 --subnet 10.0.1.0/24 --vlan 1001 --dhcp --dhcp-start 10.0.1.10\n06:48:46 INF VPC created name=vpc-1\n\ncore@control-1 ~ $ kubectl fabric vpc create --name vpc-2 --subnet 10.0.2.0/24 --vlan 1002 --dhcp --dhcp-start 10.0.2.10\n06:49:04 INF VPC created name=vpc-2\n\ncore@control-1 ~ $ kubectl fabric vpc attach --vpc-subnet vpc-1/default --connection server-01--mclag--leaf-01--leaf-02\n06:49:24 INF VPCAttachment created name=vpc-1--default--server-01--mclag--leaf-01--leaf-02\n\ncore@control-1 ~ $ kubectl fabric vpc attach --vpc-subnet vpc-2/default --connection server-02--mclag--leaf-01--leaf-02\n06:49:34 INF VPCAttachment created name=vpc-2--default--server-02--mclag--leaf-01--leaf-02\n

The VPC subnet should belong to an IPv4Namespace, the default one in the VLAB is 10.0.0.0/16:

core@control-1 ~ $ kubectl get ipns\nNAME      SUBNETS           AGE\ndefault   [\"10.0.0.0/16\"]   5h14m\n

After you created the VPCs and VPCAttachments, you can check the status of the agents to make sure that the requested configuration was applied to the switches:

core@control-1 ~ $ kubectl get agents\nNAME       ROLE          DESCR           APPLIED   APPLIEDG   CURRENTG   VERSION\nleaf-01    server-leaf   VS-01 MCLAG 1   2m2s      5          5          v0.23.0\nleaf-02    server-leaf   VS-02 MCLAG 1   2m2s      4          4          v0.23.0\nleaf-03    server-leaf   VS-03           112s      5          5          v0.23.0\nspine-01   spine         VS-04           16m       3          3          v0.23.0\nspine-02   spine         VS-05           18m       4          4          v0.23.0\n

In this example, the values in columns APPLIED and APPLIEDG are equal which means that the requested configuration has been applied.

"},{"location":"vlab/demo/#setting-up-networking-on-test-servers","title":"Setting up networking on test servers","text":"

You can use hhfab vlab ssh on the host to SSH into the test servers and configure networking there. For example, for both server-01 (MCLAG attached to both leaf-01 and leaf-02) we need to configure a bond with a VLAN on top of it and for the server-05 (single-homed unbundled attached to leaf-03) we need to configure just a VLAn and they both will get an IP address from the DHCP server. You can use the ip command to configure networking on the servers or use the little helper preinstalled by Fabricator on test servers.

For server-01:

core@server-01 ~ $ hhnet cleanup\ncore@server-01 ~ $ hhnet bond 1001 enp2s1 enp2s2\n10.0.1.10/24\ncore@server-01 ~ $ ip a\n...\n3: enp2s1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000\n    link/ether 06:5a:e8:38:3b:ea brd ff:ff:ff:ff:ff:ff permaddr 0c:20:12:fe:01:01\n4: enp2s2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000\n    link/ether 06:5a:e8:38:3b:ea brd ff:ff:ff:ff:ff:ff permaddr 0c:20:12:fe:01:02\n6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link/ether 06:5a:e8:38:3b:ea brd ff:ff:ff:ff:ff:ff\n    inet6 fe80::45a:e8ff:fe38:3bea/64 scope link\n       valid_lft forever preferred_lft forever\n7: bond0.1001@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link/ether 06:5a:e8:38:3b:ea brd ff:ff:ff:ff:ff:ff\n    inet 10.0.1.10/24 metric 1024 brd 10.0.1.255 scope global dynamic bond0.1001\n       valid_lft 86396sec preferred_lft 86396sec\n    inet6 fe80::45a:e8ff:fe38:3bea/64 scope link\n       valid_lft forever preferred_lft forever\n

And for server-02:

core@server-02 ~ $ hhnet cleanup\ncore@server-02 ~ $ hhnet bond 1002 enp2s1 enp2s2\n10.0.2.10/24\ncore@server-02 ~ $ ip a\n...\n3: enp2s1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000\n    link/ether 5e:10:b1:f7:d0:4c brd ff:ff:ff:ff:ff:ff permaddr 0c:20:12:fe:02:01\n4: enp2s2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000\n    link/ether 5e:10:b1:f7:d0:4c brd ff:ff:ff:ff:ff:ff permaddr 0c:20:12:fe:02:02\n8: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link/ether 5e:10:b1:f7:d0:4c brd ff:ff:ff:ff:ff:ff\n    inet6 fe80::5c10:b1ff:fef7:d04c/64 scope link\n       valid_lft forever preferred_lft forever\n9: bond0.1002@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000\n    link/ether 5e:10:b1:f7:d0:4c brd ff:ff:ff:ff:ff:ff\n    inet 10.0.2.10/24 metric 1024 brd 10.0.2.255 scope global dynamic bond0.1002\n       valid_lft 86185sec preferred_lft 86185sec\n    inet6 fe80::5c10:b1ff:fef7:d04c/64 scope link\n       valid_lft forever preferred_lft forever\n
"},{"location":"vlab/demo/#testing-connectivity-before-peering","title":"Testing connectivity before peering","text":"

You can test connectivity between the servers before peering the switches using the ping command:

core@server-01 ~ $ ping 10.0.2.10\nPING 10.0.2.10 (10.0.2.10) 56(84) bytes of data.\nFrom 10.0.1.1 icmp_seq=1 Destination Net Unreachable\nFrom 10.0.1.1 icmp_seq=2 Destination Net Unreachable\nFrom 10.0.1.1 icmp_seq=3 Destination Net Unreachable\n^C\n--- 10.0.2.10 ping statistics ---\n3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2003ms\n
core@server-02 ~ $ ping 10.0.1.10\nPING 10.0.1.10 (10.0.1.10) 56(84) bytes of data.\nFrom 10.0.2.1 icmp_seq=1 Destination Net Unreachable\nFrom 10.0.2.1 icmp_seq=2 Destination Net Unreachable\nFrom 10.0.2.1 icmp_seq=3 Destination Net Unreachable\n^C\n--- 10.0.1.10 ping statistics ---\n3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2004ms\n
"},{"location":"vlab/demo/#peering-vpcs-and-testing-connectivity","title":"Peering VPCs and testing connectivity","text":"

To enable connectivity between the VPCs, peer them using kubectl fabric vpc peer:

core@control-1 ~ $ kubectl fabric vpc peer --vpc vpc-1 --vpc vpc-2\n07:04:58 INF VPCPeering created name=vpc-1--vpc-2\n

Make sure to wait until the peering is applied to the switches using kubectl get agents command. After that, you can test connectivity between the servers again:

core@server-01 ~ $ ping 10.0.2.10\nPING 10.0.2.10 (10.0.2.10) 56(84) bytes of data.\n64 bytes from 10.0.2.10: icmp_seq=1 ttl=62 time=6.25 ms\n64 bytes from 10.0.2.10: icmp_seq=2 ttl=62 time=7.60 ms\n64 bytes from 10.0.2.10: icmp_seq=3 ttl=62 time=8.60 ms\n^C\n--- 10.0.2.10 ping statistics ---\n3 packets transmitted, 3 received, 0% packet loss, time 2004ms\nrtt min/avg/max/mdev = 6.245/7.481/8.601/0.965 ms\n
core@server-02 ~ $ ping 10.0.1.10\nPING 10.0.1.10 (10.0.1.10) 56(84) bytes of data.\n64 bytes from 10.0.1.10: icmp_seq=1 ttl=62 time=5.44 ms\n64 bytes from 10.0.1.10: icmp_seq=2 ttl=62 time=6.66 ms\n64 bytes from 10.0.1.10: icmp_seq=3 ttl=62 time=4.49 ms\n^C\n--- 10.0.1.10 ping statistics ---\n3 packets transmitted, 3 received, 0% packet loss, time 2004ms\nrtt min/avg/max/mdev = 4.489/5.529/6.656/0.886 ms\n

If you delete the VPC peering with kubectl delete applied to the relevant object and wait for the agent to apply the configuration on the switches, you can observe that connectivity is lost again:

core@control-1 ~ $ kubectl delete vpcpeering/vpc-1--vpc-2\nvpcpeering.vpc.githedgehog.com \"vpc-1--vpc-2\" deleted\n
core@server-01 ~ $ ping 10.0.2.10\nPING 10.0.2.10 (10.0.2.10) 56(84) bytes of data.\nFrom 10.0.1.1 icmp_seq=1 Destination Net Unreachable\nFrom 10.0.1.1 icmp_seq=2 Destination Net Unreachable\nFrom 10.0.1.1 icmp_seq=3 Destination Net Unreachable\n^C\n--- 10.0.2.10 ping statistics ---\n3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2004ms\n

You can see duplicate packets in the output of the ping command between some of the servers. This is expected behavior and is caused by the limitations in the VLAB environment.

core@server-01 ~ $ ping 10.0.5.10\nPING 10.0.5.10 (10.0.5.10) 56(84) bytes of data.\n64 bytes from 10.0.5.10: icmp_seq=1 ttl=62 time=9.58 ms\n64 bytes from 10.0.5.10: icmp_seq=1 ttl=62 time=9.58 ms (DUP!)\n64 bytes from 10.0.5.10: icmp_seq=2 ttl=62 time=6.99 ms\n64 bytes from 10.0.5.10: icmp_seq=2 ttl=62 time=6.99 ms (DUP!)\n64 bytes from 10.0.5.10: icmp_seq=3 ttl=62 time=9.59 ms\n64 bytes from 10.0.5.10: icmp_seq=3 ttl=62 time=9.60 ms (DUP!)\n^C\n--- 10.0.5.10 ping statistics ---\n3 packets transmitted, 3 received, +3 duplicates, 0% packet loss, time 2003ms\nrtt min/avg/max/mdev = 6.987/8.720/9.595/1.226 ms\n
"},{"location":"vlab/demo/#using-vpcs-with-overlapping-subnets","title":"Using VPCs with overlapping subnets","text":"

First, create a second IPv4Namespace with the same subnet as the default one:

core@control-1 ~ $ kubectl get ipns\nNAME      SUBNETS           AGE\ndefault   [\"10.0.0.0/16\"]   24m\n\ncore@control-1 ~ $ cat <<EOF > ipns-2.yaml\napiVersion: vpc.githedgehog.com/v1alpha2\nkind: IPv4Namespace\nmetadata:\n  name: ipns-2\n  namespace: default\nspec:\n  subnets:\n  - 10.0.0.0/16\nEOF\n\ncore@control-1 ~ $ kubectl apply -f ipns-2.yaml\nipv4namespace.vpc.githedgehog.com/ipns-2 created\n\ncore@control-1 ~ $ kubectl get ipns\nNAME      SUBNETS           AGE\ndefault   [\"10.0.0.0/16\"]   30m\nipns-2    [\"10.0.0.0/16\"]   8s\n

Let's assume that vpc-1 already exists and is attached to server-01 (see Creating and attaching VPCs). Now we can create vpc-3 with the same subnet as vpc-1 (but in the different IPv4Namespace) and attach it to the server-03:

core@control-1 ~ $ cat <<EOF > vpc-3.yaml\napiVersion: vpc.githedgehog.com/v1alpha2\nkind: VPC\nmetadata:\n  name: vpc-1\n  namespace: default\nspec:\n  ipv4Namespace: ipns-2\n  subnets:\n    default:\n      dhcp:\n        enable: true\n        range:\n          start: 10.0.1.10\n      subnet: 10.0.1.0/24\n      vlan: \"2001\"\n  vlanNamespace: default\nEOF\n\ncore@control-1 ~ $ kubectl apply -f vpc-3.yaml\n

At that point you can setup networking on server-03 the same as you did for server-01 and server-02 in a previous section. Once you have configured networking, server-01 and server-03 have IP addresses from the same subnets.

"},{"location":"vlab/overview/","title":"Overview","text":"

It's possible to run Hedgehog Fabric in a fully virtual environment using QEMU/KVM and SONiC Virtual Switch (VS). It's a great way to try out Fabric and learn about its look and feel, API, and capabilities. It's not suitable for any data plane or performance testing, or for production use.

In the VLAB all switches start as empty VMs with only the ONIE image on them, and they go through the whole discovery, boot and installation process like on real hardware.

"},{"location":"vlab/overview/#overview_1","title":"Overview","text":"

The hhfab CLI provides a special command vlab to manage the virtual labs. It allows you to run sets of virtual machines to simulate the Fabric infrastructure including control node, switches, test servers and it automatically runs the installer to get Fabric up and running.

You can find more information about getting hhfab in the download section.

"},{"location":"vlab/overview/#system-requirements","title":"System Requirements","text":"

Currently, it's only tested on Ubuntu 22.04 LTS, but should work on any Linux distribution with QEMU/KVM support and fairly up-to-date packages.

The following packages needs to be installed: qemu-kvm swtpm-tools tpm2-tools socat. Docker is also required, to login into the OCI registry.

By default, the VLAB topology is Spine-Leaf with 2 spines, 2 MCLAG leaves and 1 non-MCLAG leaf. Optionally, you can choose to run the default Collapsed Core topology using flag --fabric-mode collapsed-core (or -m collapsed-core) which only consists of 2 switches.

You can calculate the system requirements based on the allocated resources to the VMs using the following table:

Device vCPU RAM Disk Control Node 6 6GB 100GB Test Server 2 768MB 10GB Switch 4 5GB 50GB

These numbers give approximately the following requirements for the default topologies:

Usually, none of the VMs will reach 100% utilization of the allocated resources, but as a rule of thumb you should make sure that you have at least allocated RAM and disk space for all VMs.

NVMe SSD for VM disks is highly recommended.

"},{"location":"vlab/overview/#installing-prerequisites","title":"Installing prerequisites","text":"

On Ubuntu 22.04 LTS you can install all required packages using the following commands:

curl -fsSL https://get.docker.com -o install-docker.sh\nsudo sh install-docker.sh\nsudo usermod -aG docker $USER\nnewgrp docker\n
sudo apt install -y qemu-kvm swtpm-tools tpm2-tools socat\nsudo usermod -aG kvm $USER\nnewgrp kvm\nkvm-ok\n

Good output of the kvm-ok command should look like this:

ubuntu@docs:~$ kvm-ok\nINFO: /dev/kvm exists\nKVM acceleration can be used\n
"},{"location":"vlab/overview/#next-steps","title":"Next steps","text":""},{"location":"vlab/running/","title":"Running VLAB","text":"

Make sure to follow the prerequisites and check system requirements in the VLAB Overview section before running VLAB.

"},{"location":"vlab/running/#initialize-vlab","title":"Initialize VLAB","text":"

First, initialize Fabricator for the VLAB by running hhfab init --preset vlab (or -p vlab). This command supports several customization options that are listed in the output of hhfab init --help. To tune the topology used for the VLAB, you can use the --fabric-mode (or -m) flag to choose between spine-leaf (default) and collapsed-core topologies. You can also configure the number of spines, leafs, connections, and so on. For example, flags --spines-count and --mclag-leafs-count allow you to set the number of spines and MCLAG leaves, respectively.

By default, the command creates 2 spines, 2 MCLAG leaves and 1 non-MCLAG leaf with 2 fabric connections (between each spine and leaf), 2 MCLAG peer links and 2 MCLAG session links as well as 2 loopbacks per leaf for implementing VPC Loopback workaround.

ubuntu@docs:~$ hhfab init -p vlab\n01:17:44 INF Generating wiring from gen flags\n01:17:44 INF Building wiring diagram fabricMode=spine-leaf chainControlLink=false controlLinksCount=0\n01:17:44 INF                     >>> spinesCount=2 fabricLinksCount=2\n01:17:44 INF                     >>> mclagLeafsCount=2 orphanLeafsCount=1\n01:17:44 INF                     >>> mclagSessionLinks=2 mclagPeerLinks=2\n01:17:44 INF                     >>> vpcLoopbacks=2\n01:17:44 WRN Wiring is not hydrated, hydrating reason=\"error validating wiring: ASN not set for switch leaf-01\"\n01:17:44 INF Initialized preset=vlab fabricMode=spine-leaf config=.hhfab/config.yaml wiring=.hhfab/wiring.yaml\n

Or if you want to run Collapsed Core topology with 2 MCLAG switches:

ubuntu@docs:~$ hhfab init -p vlab -m collapsed-core\n01:20:07 INF Generating wiring from gen flags\n01:20:07 INF Building wiring diagram fabricMode=collapsed-core chainControlLink=false controlLinksCount=0\n01:20:07 INF                     >>> mclagLeafsCount=2 orphanLeafsCount=0\n01:20:07 INF                     >>> mclagSessionLinks=2 mclagPeerLinks=2\n01:20:07 INF                     >>> vpcLoopbacks=2\n01:20:07 WRN Wiring is not hydrated, hydrating reason=\"error validating wiring: ASN not set for switch leaf-01\"\n01:20:07 INF Initialized preset=vlab fabricMode=collapsed-core config=.hhfab/config.yaml wiring=.hhfab/wiring.yaml\n

Or you can run custom topology with 2 spines, 4 MCLAG leaves and 2 non-MCLAG leaves using flags:

ubuntu@docs:~$ hhfab init -p vlab --mclag-leafs-count 4 --orphan-leafs-count 2\n01:21:53 INF Generating wiring from gen flags\n01:21:53 INF Building wiring diagram fabricMode=spine-leaf chainControlLink=false controlLinksCount=0\n01:21:53 INF                     >>> spinesCount=2 fabricLinksCount=2\n01:21:53 INF                     >>> mclagLeafsCount=4 orphanLeafsCount=2\n01:21:53 INF                     >>> mclagSessionLinks=2 mclagPeerLinks=2\n01:21:53 INF                     >>> vpcLoopbacks=2\n01:21:53 WRN Wiring is not hydrated, hydrating reason=\"error validating wiring: ASN not set for switch leaf-01\"\n01:21:53 INF Initialized preset=vlab fabricMode=spine-leaf config=.hhfab/config.yaml wiring=.hhfab/wiring.yaml\n

Additionally, you can pass extra Fabric configuration items using flags on init command or by passing a configuration file. For more information, refer to the Fabric Configuration section.

Once you have initialized the VLAB, download the artifacts and build the installer using hhfab build. This command automatically downloads all required artifacts from the OCI registry and builds the installer and all other prerequisites for running the VLAB.

"},{"location":"vlab/running/#build-the-installer-and-vlab","title":"Build the installer and VLAB","text":"
ubuntu@docs:~$ hhfab build\n01:23:33 INF Building component=base\n01:23:33 WRN Attention! Development mode enabled - this is not secure! Default users and keys will be created.\n...\n01:23:33 INF Building component=control-os\n01:23:33 INF Building component=k3s\n01:23:33 INF Downloading name=m.l.hhdev.io:31000/githedgehog/k3s:v1.27.4-k3s1 to=.hhfab/control-install\nCopying k3s-airgap-images-amd64.tar.gz  187.36 MiB / 187.36 MiB   \u2819   0.00 b/s done\nCopying k3s                               56.50 MiB / 56.50 MiB   \u2819   0.00 b/s done\n01:23:35 INF Building component=zot\n01:23:35 INF Downloading name=m.l.hhdev.io:31000/githedgehog/zot:v1.4.3 to=.hhfab/control-install\nCopying zot-airgap-images-amd64.tar.gz  19.24 MiB / 19.24 MiB   \u2838   0.00 b/s done\n01:23:35 INF Building component=misc\n01:23:35 INF Downloading name=m.l.hhdev.io:31000/githedgehog/fabricator/k9s:v0.27.4 to=.hhfab/control-install\nCopying k9s  57.75 MiB / 57.75 MiB   \u283c   0.00 b/s done\n...\n01:25:40 INF Planned bundle=control-install name=fabric-api-chart op=\"push fabric/charts/fabric-api:v0.23.0\"\n01:25:40 INF Planned bundle=control-install name=fabric-image op=\"push fabric/fabric:v0.23.0\"\n01:25:40 INF Planned bundle=control-install name=fabric-chart op=\"push fabric/charts/fabric:v0.23.0\"\n01:25:40 INF Planned bundle=control-install name=fabric-agent-seeder op=\"push fabric/agent/x86_64:latest\"\n01:25:40 INF Planned bundle=control-install name=fabric-agent op=\"push fabric/agent:v0.23.0\"\n...\n01:25:40 INF Recipe created bundle=control-install actions=67\n01:25:40 INF Creating recipe bundle=server-install\n01:25:40 INF Planned bundle=server-install name=toolbox op=\"file /opt/hedgehog/toolbox.tar\"\n01:25:40 INF Planned bundle=server-install name=toolbox-load op=\"exec ctr\"\n01:25:40 INF Planned bundle=server-install name=hhnet op=\"file /opt/bin/hhnet\"\n01:25:40 INF Recipe created bundle=server-install actions=3\n01:25:40 INF Building done took=2m6.813384532s\n01:25:40 INF Packing bundle=control-install target=control-install.tgz\n01:25:45 INF Packing bundle=server-install target=server-install.tgz\n01:25:45 INF Packing done took=5.67007384s\n

As soon as the build has completed, you can run the VLAB using hhfab vlab up. This command automatically starts all VMs and runs the installers on the control node and test servers. It takes some time for all VMs to come up and for the installer to finish. You can monitor progress in the output. If you stop the command, it will stop all VMs, and you can re-run it to get VMs back up and running.

"},{"location":"vlab/running/#run-vms-and-installers","title":"Run VMs and installers","text":"
ubuntu@docs:~$ hhfab vlab up\n01:29:13 INF Starting VLAB server... basedir=.hhfab/vlab-vms vm-size=\"\" dry-run=false\n01:29:13 INF VM id=0 name=control-1 type=control\n01:29:13 INF VM id=1 name=server-01 type=server\n01:29:13 INF VM id=2 name=server-02 type=server\n01:29:13 INF VM id=3 name=server-03 type=server\n01:29:13 INF VM id=4 name=server-04 type=server\n01:29:13 INF VM id=5 name=server-05 type=server\n01:29:13 INF VM id=6 name=server-06 type=server\n01:29:13 INF VM id=7 name=leaf-01 type=switch-vs\n01:29:13 INF VM id=8 name=leaf-02 type=switch-vs\n01:29:13 INF VM id=9 name=leaf-03 type=switch-vs\n01:29:13 INF VM id=10 name=spine-01 type=switch-vs\n01:29:13 INF VM id=11 name=spine-02 type=switch-vs\n01:29:13 INF Total VM resources cpu=\"38 vCPUs\" ram=\"36352 MB\" disk=\"410 GB\"\n...\n01:29:13 INF Preparing VM id=0 name=control-1 type=control\n01:29:13 INF Copying files  from=.hhfab/control-os/ignition.json to=.hhfab/vlab-vms/control-1/ignition.json\n01:29:13 INF Copying files  from=.hhfab/vlab-files/flatcar.img to=.hhfab/vlab-vms/control-1/os.img\n 947.56 MiB / 947.56 MiB [==========================================================] 5.13 GiB/s done\n01:29:14 INF Copying files  from=.hhfab/vlab-files/flatcar_efi_code.fd to=.hhfab/vlab-vms/control-1/efi_code.fd\n01:29:14 INF Copying files  from=.hhfab/vlab-files/flatcar_efi_vars.fd to=.hhfab/vlab-vms/control-1/efi_vars.fd\n01:29:14 INF Resizing VM image (may require sudo password) name=control-1\n01:29:17 INF Initializing TPM name=control-1\n...\n01:29:46 INF Installing VM name=control-1 type=control\n01:29:46 INF Installing VM name=server-01 type=server\n01:29:46 INF Installing VM name=server-02 type=server\n01:29:46 INF Installing VM name=server-03 type=server\n01:29:47 INF Installing VM name=server-04 type=server\n01:29:47 INF Installing VM name=server-05 type=server\n01:29:47 INF Installing VM name=server-06 type=server\n01:29:49 INF Running VM id=0 name=control-1 type=control\n01:29:49 INF Running VM id=1 name=server-01 type=server\n01:29:49 INF Running VM id=2 name=server-02 type=server\n01:29:49 INF Running VM id=3 name=server-03 type=server\n01:29:50 INF Running VM id=4 name=server-04 type=server\n01:29:50 INF Running VM id=5 name=server-05 type=server\n01:29:50 INF Running VM id=6 name=server-06 type=server\n01:29:50 INF Running VM id=7 name=leaf-01 type=switch-vs\n01:29:50 INF Running VM id=8 name=leaf-02 type=switch-vs\n01:29:51 INF Running VM id=9 name=leaf-03 type=switch-vs\n01:29:51 INF Running VM id=10 name=spine-01 type=switch-vs\n01:29:51 INF Running VM id=11 name=spine-02 type=switch-vs\n...\n01:30:41 INF VM installed name=server-06 type=server installer=server-install\n01:30:41 INF VM installed name=server-01 type=server installer=server-install\n01:30:41 INF VM installed name=server-02 type=server installer=server-install\n01:30:41 INF VM installed name=server-04 type=server installer=server-install\n01:30:41 INF VM installed name=server-03 type=server installer=server-install\n01:30:41 INF VM installed name=server-05 type=server installer=server-install\n...\n01:31:04 INF Running installer on VM name=control-1 type=control installer=control-install\n...\n01:35:15 INF Done took=3m39.586394608s\n01:35:15 INF VM installed name=control-1 type=control installer=control-install\n

Line VM installed name=control-1 from the installer's output means that the installer has finished. After this line has been displayed, you can get into the control node and other VMs to watch the Fabric coming up and switches getting provisioned.

"},{"location":"vlab/running/#configuring-vlab-vms","title":"Configuring VLAB VMs","text":"

By default, all test server VMs are isolated and have no connectivity to the host or the Internet. You can configure enable connectivity using hhfab vlab up --restrict-servers=false to allow the test servers to access the Internet and the host. When you enable connectivity, VMs get a default route pointing to the host, which means that in case of the VPC peering you need to configure test server VMs to use the VPC attachment as a default route (or just some specific subnets).

Additionally, you can configure the size of all VMs using hhfab vlab up --vm-size <size>. The flag allows you to choose from one of the presets (compact, default, full and huge) to get the control, switch, and server VMs of different sizes.

"},{"location":"vlab/running/#default-credentials","title":"Default credentials","text":"

Fabricator creates default users and keys for you to login into the control node and test servers as well as for the SONiC Virtual Switches.

Default user with passwordless sudo for the control node and test servers is core with password HHFab.Admin!. Admin user with full access and passwordless sudo for the switches is admin with password HHFab.Admin!. Read-only, non-sudo user with access only to the switch CLI for the switches is op with password HHFab.Op!.

"},{"location":"vlab/running/#accessing-the-vlab","title":"Accessing the VLAB","text":"

The hhfab vlab command provides ssh and serial subcommands to access the VMs. You can use ssh to get into the control node and test servers after the VMs are started. You can use serial to get into the switch VMs while they are provisioning and installing the software. After switches are installed you can use ssh to get into them.

You can select device you want to access or pass the name using the --vm flag.

ubuntu@docs:~$ hhfab vlab ssh\nUse the arrow keys to navigate: \u2193 \u2191 \u2192 \u2190  and / toggles search\nSSH to VM:\n  \ud83e\udd94 control-1\n  server-01\n  server-02\n  server-03\n  server-04\n  server-05\n  server-06\n  leaf-01\n  leaf-02\n  leaf-03\n  spine-01\n  spine-02\n\n----------- VM Details ------------\nID:             0\nName:           control-1\nReady:          true\nBasedir:        .hhfab/vlab-vms/control-1\n

On the control node you have access to kubectl, Fabric CLI, and k9s to manage the Fabric. You can find information about the switches provisioning by running kubectl get agents -o wide. It usually takes about 10-15 minutes for the switches to get installed.

After the switches are provisioned, the command returns something like this:

core@control-1 ~ $ kubectl get agents -o wide\nNAME       ROLE          DESCR           HWSKU                      ASIC   HEARTBEAT   APPLIED   APPLIEDG   CURRENTG   VERSION   SOFTWARE                ATTEMPT   ATTEMPTG   AGE\nleaf-01    server-leaf   VS-01 MCLAG 1   DellEMC-S5248f-P-25G-DPB   vs     30s         5m5s      4          4          v0.23.0   4.1.1-Enterprise_Base   5m5s      4          10m\nleaf-02    server-leaf   VS-02 MCLAG 1   DellEMC-S5248f-P-25G-DPB   vs     27s         3m30s     3          3          v0.23.0   4.1.1-Enterprise_Base   3m30s     3          10m\nleaf-03    server-leaf   VS-03           DellEMC-S5248f-P-25G-DPB   vs     18s         3m52s     4          4          v0.23.0   4.1.1-Enterprise_Base   3m52s     4          10m\nspine-01   spine         VS-04           DellEMC-S5248f-P-25G-DPB   vs     26s         3m59s     3          3          v0.23.0   4.1.1-Enterprise_Base   3m59s     3          10m\nspine-02   spine         VS-05           DellEMC-S5248f-P-25G-DPB   vs     19s         3m53s     4          4          v0.23.0   4.1.1-Enterprise_Base   3m53s     4          10m\n

The Heartbeat column shows how long ago the switch has sent the heartbeat to the control node. The Applied column shows how long ago the switch has applied the configuration. AppliedG shows the generation of the configuration applied. CurrentG shows the generation of the configuration the switch is supposed to run. Different values for AppliedG and CurrentG mean that the switch is in the process of applying the configuration.

At that point Fabric is ready and you can use kubectl and kubectl fabric to manage the Fabric. You can find more about managing the Fabric in the Running Demo and User Guide sections.

"},{"location":"vlab/running/#getting-main-fabric-objects","title":"Getting main Fabric objects","text":"

You can list the main Fabric objects by running kubectl get on the control node. You can find more details about using the Fabric in the User Guide, Fabric API and Fabric CLI sections.

For example, to get the list of switches, run:

core@control-1 ~ $ kubectl get switch\nNAME       ROLE          DESCR           GROUPS   LOCATIONUUID                           AGE\nleaf-01    server-leaf   VS-01 MCLAG 1            5e2ae08a-8ba9-599a-ae0f-58c17cbbac67   6h10m\nleaf-02    server-leaf   VS-02 MCLAG 1            5a310b84-153e-5e1c-ae99-dff9bf1bfc91   6h10m\nleaf-03    server-leaf   VS-03                    5f5f4ad5-c300-5ae3-9e47-f7898a087969   6h10m\nspine-01   spine         VS-04                    3e2c4992-a2e4-594b-bbd1-f8b2fd9c13da   6h10m\nspine-02   spine         VS-05                    96fbd4eb-53b5-5a4c-8d6a-bbc27d883030   6h10m\n

Similarly, to get the list of servers, run:

core@control-1 ~ $ kubectl get server\nNAME        TYPE      DESCR                        AGE\ncontrol-1   control   Control node                 6h10m\nserver-01             S-01 MCLAG leaf-01 leaf-02   6h10m\nserver-02             S-02 MCLAG leaf-01 leaf-02   6h10m\nserver-03             S-03 Unbundled leaf-01       6h10m\nserver-04             S-04 Bundled leaf-02         6h10m\nserver-05             S-05 Unbundled leaf-03       6h10m\nserver-06             S-06 Bundled leaf-03         6h10m\n

For connections, use:

core@control-1 ~ $ kubectl get connection\nNAME                                 TYPE           AGE\ncontrol-1--mgmt--leaf-01             management     6h11m\ncontrol-1--mgmt--leaf-02             management     6h11m\ncontrol-1--mgmt--leaf-03             management     6h11m\ncontrol-1--mgmt--spine-01            management     6h11m\ncontrol-1--mgmt--spine-02            management     6h11m\nleaf-01--mclag-domain--leaf-02       mclag-domain   6h11m\nleaf-01--vpc-loopback                vpc-loopback   6h11m\nleaf-02--vpc-loopback                vpc-loopback   6h11m\nleaf-03--vpc-loopback                vpc-loopback   6h11m\nserver-01--mclag--leaf-01--leaf-02   mclag          6h11m\nserver-02--mclag--leaf-01--leaf-02   mclag          6h11m\nserver-03--unbundled--leaf-01        unbundled      6h11m\nserver-04--bundled--leaf-02          bundled        6h11m\nserver-05--unbundled--leaf-03        unbundled      6h11m\nserver-06--bundled--leaf-03          bundled        6h11m\nspine-01--fabric--leaf-01            fabric         6h11m\nspine-01--fabric--leaf-02            fabric         6h11m\nspine-01--fabric--leaf-03            fabric         6h11m\nspine-02--fabric--leaf-01            fabric         6h11m\nspine-02--fabric--leaf-02            fabric         6h11m\nspine-02--fabric--leaf-03            fabric         6h11m\n

For IPv4 and VLAN namespaces, use:

core@control-1 ~ $ kubectl get ipns\nNAME      SUBNETS           AGE\ndefault   [\"10.0.0.0/16\"]   6h12m\n\ncore@control-1 ~ $ kubectl get vlanns\nNAME      AGE\ndefault   6h12m\n
"},{"location":"vlab/running/#reset-vlab","title":"Reset VLAB","text":"

To reset VLAB and start over just remove the .hhfab directory and run hhfab init again.

"},{"location":"vlab/running/#next-steps","title":"Next steps","text":""}]} \ No newline at end of file diff --git a/alpha-6/sitemap.xml b/alpha-6/sitemap.xml index 823987d..a70d7de 100755 --- a/alpha-6/sitemap.xml +++ b/alpha-6/sitemap.xml @@ -2,137 +2,137 @@ https://docs.githedgehog.com/alpha-6/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/architecture/fabric/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/architecture/overview/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/concepts/overview/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/contribute/docs/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/contribute/overview/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/getting-started/download/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/install-upgrade/build-wiring/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/install-upgrade/config/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/install-upgrade/onie-update/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/install-upgrade/overview/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/install-upgrade/requirements/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/install-upgrade/supported-devices/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/reference/api/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/reference/cli/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/release-notes/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/troubleshooting/overview/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/user-guide/connections/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/user-guide/devices/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/user-guide/external/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/user-guide/harvester/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/user-guide/overview/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/user-guide/shrink-expand/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/user-guide/vpcs/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/vlab/demo/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/vlab/overview/ - 2024-05-08 + 2024-06-18 daily https://docs.githedgehog.com/alpha-6/vlab/running/ - 2024-05-08 + 2024-06-18 daily \ No newline at end of file diff --git a/alpha-6/sitemap.xml.gz b/alpha-6/sitemap.xml.gz index 8660d4d..814e093 100755 Binary files a/alpha-6/sitemap.xml.gz and b/alpha-6/sitemap.xml.gz differ diff --git a/alpha-6/user-guide/vpcs/index.html b/alpha-6/user-guide/vpcs/index.html index 04a8d42..7b33929 100755 --- a/alpha-6/user-guide/vpcs/index.html +++ b/alpha-6/user-guide/vpcs/index.html @@ -1360,13 +1360,13 @@

default: # Each subnet is named, "default" subnet isn't required, but actively used by CLI dhcp: enable: true # On-demand DHCP server - range: # Optionally, start/end range could be specified + range: # Optionally, start/end range could be specified, otherwise all available IPs are used start: 10.10.1.10 end: 10.10.1.99 - pxeURL: tftp://10.10.10.99/bootfilename # PXEURL (optional) to identify the PXE server to use to boot hosts; HTTP query strings are not supported + pxeURL: tftp://10.10.10.99/bootfilename # PXEURL (optional) to identify the PXE server to use to boot hosts; HTTP query strings are not supported subnet: 10.10.1.0/24 # User-defined subnet from ipv4 namespace gateway: 10.10.1.1 # User-defined gateway (optional, default is .1) - vlan: "1001" # User-defined VLAN from VLAN namespace + vlan: 1001 # User-defined VLAN from VLAN namespace isolated: true # Makes subnet isolated from other subnets within the VPC (doesn't affect VPC peering) restricted: true # Causes all hosts in the subnet to be isolated from each other @@ -1374,11 +1374,11 @@

dhcp: relay: 10.99.0.100/24 # Use third-party DHCP server (DHCP relay configuration), access to it could be enabled using StaticExternal connection subnet: "10.10.2.0/24" - vlan: "1002" + vlan: 1002 another-subnet: # Minimal configuration is just a name, subnet and VLAN subnet: 10.10.100.0/24 - vlan: "1100" + vlan: 1100 permit: # Defines which VPCs could communicate to each other, applied on top of subnets "isolated" flag (doesn't affect VPC peering) - [subnet-1, subnet-2, subnet-3] # 1, 2 and 3 subnets could communicate to each other @@ -1501,7 +1501,7 @@

VLANNamespace

Last update: - May 7, 2024 + June 5, 2024
Created: