Network binding plugin: Address compute container resource overhead

Signed-off-by: Orel Misan <[email protected]>

design-proposals/network-binding-plugin/network-binding-plugin.md

@@ -281,6 +281,57 @@ This solution was selected since it provides the cluster admin more control in r
 
 For the alternative solutions please see [Appendix G](#appendix-g-alternatives-to-plugin-sidecar-container-resource-specification)
 
+##### Compute Container Resource Overhead
+
+For some plugins, an additional resource consumption can be expected from the virt-launcher pod compute container.
+For example, there could be need to execute an additional binary in the compute container.
+Since this binary has its own CPU and memory limits, they should be somehow accounted for.
+Another example could be increased resource consumption of the virt-stack resulting from using the plugin.
+
+Suggested solution:
+
+Additional API for compute container resource overhead:
+
+The network binding plugin API in the KubeVirt CR could receive an additional input field to specify the resource requirements overhead for the compute container:
+
+```yaml
+apiVersion: kubevirt.io/v1
+kind: KubeVirt
+metadata:
+  name: kubevirt
+  namespace: kubevirt
+spec:
+  configuration:
+    network:
+      binding:
+        mynetbindingplugin:
+          sidecarImage: quay.io/kubevirt/mynetbindingplugin
+          computeResourceOverhead:
+            requests:
+              cpu: 200m
+              memory: 20Mi
+```
+
+If specified, KubeVirt will add the resource overhead specified in `computeResourceOverhead` to the compute container of the virt-launcher pod.
+The specified resource overhead will be added to the compute container per unique usage of the plugin (not per NIC using the binding).
+
+For example, assuming there is a plugin registered with a 200Mi memory overhead for the `compute` container, and
+there are two interfaces using it.
+`virt-controller` will only add 200Mi of memory to the `compute` container.
+
+Pros:
+- Cluster-wide definition of compute resource overhead per plugin.
+- Additional resources could be requested other than CPU and memory.
+- The resource overhead specification is visible to cluster admins.
+
+Cons:
+- Requires an API change.
+- When upgrading KubeVirt / network binding plugin versions, the compute resource overhea specification might require adjustments.
+
+This solution was selected since it provides the cluster admin more control in regard to resource allocation.
+
+For the alternative solutions please see [Appendix H](#appendix-h-alternatives-to-compute-container-resource-overhead-specification)
+
 #### Configure Pod netns
 
 The CNI plugin has privileged access to the pod network namespace and
@@ -1262,3 +1313,24 @@ Cons:
 
 This solution provides flexibility for plugin authors while keeping the network binding plugin API in KubeVirt small.
 The requirement to maintain certificates for the webhook could be mitigated using tools such as [cert-manager](https://cert-manager.io/).
+
+# Appendix H:  Alternatives to compute container resource overhead specification
+
+1. Manually setting the VM's resources:
+
+The user can override KubeVirt's algorithms and set resource requirements.
+
+Pros:
+- Already implemented.
+
+Cons:
+- Error prune
+- The user does not take into account the overhead considerations KubeVirt takes when templating a virt-launcher pod.
+
+2. Mutating webhook
+
+Shipping a mutating webhook as part of the plugin's deliverables.
+The webhook will add the required resource overhead for the compute container.
+
+The compute container's name could be identified using `kubectl.kubernetes.io/default-container` annotation's
+value on the virt-launcher pod.