Custom CA

[kfox1111]

Until k8s has a native way of adding CA's to the containers chain of trust, we need a way to do it on the spire containers anyway.

[marcofranssen]

Could you clarify this issue a bit further @kfox1111? It lacks context for anyone not been part of the discussion.

My assumption on what I think this is about as a solution:

We are doing this using cert-manager as a upstream ca to have that shared root of trust at the cluster level using a selfsigned CA on cert-manager level.

If you want trust beyond the cluster you could configure cert-manager to utilize PCA on Amazon e.g. to have a CA that is trusted beyond the cluster. Using cert-manager you can make any intermediate CAs on the cluster level, we do this for various components that need to manage their own certificates e.g. spire-ca.

What we can do is create an example of that if that is what this issue is about.

[kfox1111]

Other direction. When an organization has their own self signed CA, often for internal use, spire needs to talk to servers with that signature. For example, spire -> mysql, or spire -> federated http based endpoint. This means spire needs to be able to trust the CA for that organization. Currently thats only possible if the root chain of trust within the containers includes the orgs CA. This is currently not configurable.

There is not currently a native way to do this in k8s. It's being worked on though: kubernetes/enhancements#3257

There is a cert-manager subproject that could help in the mean time: https://cert-manager.io/docs/trust/trust-manager/ we could explore if we think that we want to add a dependency for this use case.

[kfox1111]

It looks like trust-manager just sets up the configmap though with the whole ca bundle. Its still left up to the user how to inject it into the containers. So we still likely need to add a feature to the chart in that case.

[marcofranssen]

It looks like trust-manager just sets up the configmap though with the whole ca bundle. Its still left up to the user how to inject it into the containers. So we still likely need to add a feature to the chart in that case.

You might be able to use this CA cert to create another cert-manager issuer based on that CA to do the Spire usecase.

[kfox1111]

It has nothing to do with issuers. it has to do with the container root chain of trust.

[marcofranssen]

It has nothing to do with issuers. it has to do with the container root chain of trust.

As a workarround you can use the cert-manager issuers to get the secrets created for each container until there is better official support in k8s.

[kfox1111]

how do you use cert manager to inject your corperate ca into the spire-server container?

[cccsss01]

Happy to look into this.

[faisal-memon]

Thanks @cccsss01. Assigned to you.

[kfox1111]

Some relevant bits:

https://go.dev/src/crypto/x509/root_unix.go lines 19 - 25 define:

	// the SSL certificate file. If set this overrides the system default.
	certFileEnv = "SSL_CERT_FILE"

	// certDirEnv is the environment variable which identifies which directory
	// to check for SSL certificate files. If set this overrides the system default.
	// It is a colon separated list of directories.
	// See https://www.openssl.org/docs/man1.0.2/man1/c_rehash.html.
	certDirEnv = "SSL_CERT_DIR"

Some places golang looks on various linux distro's:
https://go.dev/src/crypto/x509/root_linux.go

It would be nice if we could reuse this logic, but the container contains its own ca at the top of the list.... so it would take precident over all other distros....

We may want an enum of something like:
hostDistro: [redhat, centos, ubuntu, other]
so the user can specify what distro they have and we can mount in a path that is well known for that distro, with a fallthrough of letting the user mount their own with extraVolumes/extraVolumeMounts.

In any case, we set either SSL_CERT_DIR or SSL_CERT_FILE to tell spire-agent where to find it?

[cccsss01]

I know hostPath is being used in other places, but it didn't look like the OIDC provider is using hostPath as of yet. I think it's worth investigating what could be done w/o hostPath, set envVariable to the external volume/mount?

[kfox1111]

I know hostPath is being used in other places, but it didn't look like the OIDC provider is using hostPath as of yet. I think it's worth investigating what could be done w/o hostPath, set envVariable to the external volume/mount?

the oidc provider shouldn't need it as its exporting only.

Think the two pieces that may need overrides are the agent and the server. The agent has hostpaths already, so those should be good to go.

the server probably should have its own configmap/secret I think, as it probably shouldn't ever have hostPath.

This task should probably be made into two pr's? one for the agent (higher priority) and one for the server (lower priority) I think?

[cccsss01]

You'd know best I'm still getting caught up with whrre I left off, with the new csi driver and spiffe helper.

[edwbuck]

@kfox1111 Your commentary here leaves the request unclear. Can we move from this discussion to some sort of proposed action?

[kfox1111]

We need a mechanism to allow the user to inject a ca file or ca path into the spire-agent and spire-server containers via configmap/secret

In the case of the spire-agent, the ability to hostPath in the hosts ca's would be useful as well.