Separate clusters from trust zones

[markgoddard]

Summary

Currently we have a model in which a trust zone/domain contains exactly one Kubernetes cluster. With SPIFFE/SPIRE it is possible to have a trust domain that includes multiple Kubernetes clusters, as well as other types of hosts (Linux, Windows, etc.). In this case the SPIRE server(s) for the trust zone/domain would reside in one of the clusters, or even outside of the clusters.

CLI UI

New CLI commands will be added:

cofidectl cluster list
cofidectl cluster add <name> --trust-zone <trust zone> --kubernetes-cluster <cluster> --kubernetes-context <context> --profile <profile>
cofidectl cluster helm override <name> --input-file <values-file>
cofidectl cluster helm values <name> --output-file <values-file>
cofidectl cluster status <name>

Consider whether the cluster name is scoped to the trust zone or global.

Data model

A new Cluster message will be added. Some of the TrustZone fields will need to move into the Cluster message.

Consider whether a TrustZone should have a clusters field containing a list of Cluster messages. This would be convenient, but probably not how it would be represented in a database.

We could start by retaining the 1:1 trust zone/cluster mapping (as a restriction on the list length), while moving the cluster-specific information into the new field.

Current:

message TrustZone {
  string name = 1;
  string trust_domain = 2;
  optional string kubernetes_cluster = 3;
  optional string kubernetes_context = 4;
  optional proto.trust_provider.v1alpha1.TrustProvider trust_provider = 5;
  optional string bundle_endpoint_url = 6;
  optional string bundle = 7;
  repeated proto.federation.v1alpha1.Federation federations = 8;
  repeated proto.ap_binding.v1alpha1.APBinding attestation_policies = 9;
  optional string jwt_issuer = 10;
  optional google.protobuf.Struct extra_helm_values = 11;
  optional BundleEndpointProfile bundle_endpoint_profile = 12;
  optional string profile = 13;
}

Proposed:

message TrustZone {
  string name = 1;
  string trust_domain = 2;
  optional string bundle_endpoint_url = 3;
  optional string bundle = 4;
  repeated proto.federation.v1alpha1.Federation federations = 5;
  repeated proto.ap_binding.v1alpha1.APBinding attestation_policies = 6;
  optional string jwt_issuer = 7;
  optional google.protobuf.Struct extra_helm_values = 8;
  optional BundleEndpointProfile bundle_endpoint_profile = 9;
  // Consider whether this is a good idea:
  repeated Cluster clusters = 10;
}

message Cluster {
  optional string name = 1;
  optional string trust_zone = 2;
  optional string kubernetes_cluster = 2;
  optional string kubernetes_context = 3;
  optional proto.trust_provider.v1alpha1.TrustProvider trust_provider = 4;
  optional google.protobuf.Struct extra_helm_values = 5;
  optional string profile = 6;
}

Data source

New methods would need to be added to the DataSource interface:

	GetCluster(string name) (*clusterpb.Cluster, error)
	ListClusters() ([]*clusterpb.Cluster, error)
	AddCluster(*clusterpb.Cluster, tzName string) (*clusterpb.Cluster, error)
	UpdateCluster(*clusterpb.Cluster) error

Provisioning

The Provision interface is at a high enough level that it would not need to change. However implementations of the interface would need to support the new Cluster message. If we support multiple clusters per trust zone, then this would add additional complexity to the provisioning process, particularly if some of the clusters do not include any SPIRE servers.

Non-Kubernetes clusters

Not to be implemented for this issue, but while implementing the feature consider the implications for non-Kubernetes clusters.

[jsnctl]

Consider whether a TrustZone should have a clusters field containing a list of Cluster messages. This would be convenient, but probably not how it would be represented in a database.

Would the alternative be a collection ClusterID (or similar) values that disambiguate between individual cluster objects, and the TrustZone just stores those IDs? One upside of this could be migrating a Cluster from TrustZone A -> TrustZoneB would be just a case of deleting and adding an ID instead of the full object (and maps to a fairly standard data model normalisation setup too)

[markgoddard]

Consider whether a TrustZone should have a clusters field containing a list of Cluster messages. This would be convenient, but probably not how it would be represented in a database.

Would the alternative be a collection ClusterID (or similar) values that disambiguate between individual cluster objects, and the TrustZone just stores those IDs? One upside of this could be migrating a Cluster from TrustZone A -> TrustZoneB would be just a case of deleting and adding an ID instead of the full object (and maps to a fairly standard data model normalisation setup too)

This makes sense, similar to the attestation policy binding approach.