Development.md

Development

This document describes how to get started with developing CAPMOX.

Table of contents

• About CAPMOX
• CAPMOX Dependencies
• Getting Started
  • Proxmox VE API Token
• Setting up a development environment
  • Running Tilt
• Make Targets
  • Modifying API Definitions
• Manual Capmox Setup
  • Deploying CAPMOX
  • Running CAPMOX
  • Uninstalling CAPMOX

About CAPMOX

CAPMOX is a Kubernetes Cluster API provider for Proxmox Virtual Environment.

This project aims to follow the Kubernetes Operator pattern.

It uses Controllers, which provide a reconcile function responsible for synchronizing resources until the desired state is reached on the cluster.

CAPMOX dependencies

The following dependencies are required to setup a development environment:

• git
• make
• Go v1.22
• Kubebuilder (only required for making new controllers)
• Docker (required for Kind)
• Tilt
• Kubectl
• kind
• clusterctl

Our Makefile will set up the following dependencies automatically:

• kustomize
• controller-gen
• setup-envtest

Getting Started

You’ll need a Kubernetes cluster to run against. You can use kind to get a local cluster for testing or run against a remote cluster. It is possible to substitute Docker with Podman, and kind with minikube but various assumptions in cluster-api's Tiltfile do not hold. We strongly advise against this approach.

Note: Your controller will automatically use the current context in your kubeconfig file (i.e. whatever cluster kubectl cluster-info shows).

If you're having trouble setting any of this up, check the Troubleshooting docs.

Proxmox VE API Token

Cluster-api-provider-proxmox requires a running Proxmox VE instance and an API token for access. See the Proxmox wiki for more information.

How to setup a development environment

Running Tilt

• Create a directory and cd into it

• Clone cluster-api

• Clone cluster-api-ipam-provider-in-cluster

• Clone cluster-api-provider-proxmox

• You should now have a directory containing the following git repositories:

./cluster-api
./cluster-api-ipam-provider-in-cluster
./cluster-api-provider-proxmox

• Change directory to cluster-api: cd cluster-api. This directory is the working directory for Tilt.
• Create a file called tilt-settings.json with the following contents:

{
  "default_registry": "ghcr.io/ionos-cloud",
  "provider_repos": ["../cluster-api-provider-proxmox/", "../cluster-api-ipam-provider-in-cluster/"],
  "enable_providers": ["ipam-in-cluster", "proxmox", "kubeadm-bootstrap", "kubeadm-control-plane"],
  "allowed_contexts": ["minikube"],
  "kustomize_substitutions": {},
  "extra_args": {
    "proxmox": ["--v=4"]
  },
  "debug": {
    "proxmox": {
      "port": 30000
    }
  }
}

This file instructs Tilt to use the cluster-api-provider-proxmox and ipam-provider-in-cluster repositories. allowed_contexts is used to add allowed clusters other than kind (which is always implicitly enabled).

The optional debug section will start CAPMOX with a delve remote debugger listening on port 30000. You will need to forward the port to localhost, e.g. by running kubectl port-forward deployments/capmox-controller-manager 30000:30000 -n capmox-system.

• Change directory to cluster-api-ipam-provider-in-cluster cd ../cluster-api-ipam-provider-in-cluster.

• If you don't have a cluster, create a new kind cluster:

kind create cluster --name capi-test

• cluster-api-provider-proxmox uses environment variables to connect to Proxmox VE. These need to be set in the shell which spawns Tilt. Tilt will pass these to the respective Kubernetes pods created. All variables are documented in ../cluster-api-provider-proxmox/envfile.example. Copy ../cluster-api-provider-proxmox/envfile.example to ../cluster-api-provider-proxmox/envfile and make changes pertaining to your configuration. For documentation on environment variables, see usage

• If you already had a kind cluster, add this line to ../cluster-api-provider-proxmox/envfile:

CAPI_KIND_CLUSTER_NAME=<yourclustername>

• Start tilt with the following command (with CWD still being ./cluster-api):

. ../cluster-api-provider-proxmox/envfile && tilt up

Press the space key to open the Tilt web interface in your browser. Check that all resources turn green and there are no warnings. You can click on (Tiltfile) to see all the resources.

Congratulations you now have CAPMOX running via Tilt. If you make any code changes you should see that CAPMOX is automatically rebuilt. For help deploying your first cluster with CAPMOX, see usage.

Make targets

NOTE: Run make --help for more information on all potential make targets

More information can be found via the Kubebuilder Documentation

Modifying API definitions

If you are editing the API definitions, generate the manifests such as CRs or CRDs using:

make manifests

Manual CAPMOX setup

Deploying CAPMOX to kind

1. Install CAPMOX's Custom Resources into kind:

kubectl apply -f config/samples/

2. Build and push your image to the location specified by IMG:

make docker-build docker-push IMG=<some-registry>/cluster-api-provider-proxmox:tag

3. Deploy the controller to the cluster with the image specified by IMG:

make deploy IMG=<some-registry>/cluster-api-provider-proxmox:tag

Running CAPMOX

1. Install the CRDs into the cluster:

make install

2. Run your controller (this will run in the foreground, so switch to a new terminal if you want to leave it running):

make run

NOTE: You can also run this in one step by running: make install run

Uninstalling CAPMOX

To delete the CRDs from the cluster:

make uninstall

To undeploy the controller from the cluster:

make undeploy