Machine ID: SPIFFE support in tbot

[strideynet]

Dependent on #38181

Part of #36205

As per RFD https://github.com/gravitational/teleport.e/blob/master/rfd/0018e-machine-id-workload-identity-mvp.md

Introduces a service type and output type for generating SPIFFE SVIDs. There's a few improvements I'd like to make in follow up PRs before I remove the feature flag for this.

Example config:

version: v2
auth_server: redacted:443
certificate_ttl: 24h
onboarding:
  join_method: token
  token: redacted
storage:
  type: directory
  path: /Users/noah/code/gravitational/teleport-scratch/workload-id/storage
# outputs will be filled in during the completion of an access guide.
outputs:
- type: spiffe-svid
  destination:
    type: directory
    path: /Users/noah/code/gravitational/teleport-scratch/workload-id/aws-roles-anywhere
  svid:
    path: /ra/my-role
- type: spiffe-svid
  destination:
    type: directory
    path: /Users/noah/code/gravitational/teleport-scratch/workload-id/output
  svid:
    path: /bart
    sans:
      ip:
       - 10.0.0.0
      dns:
       - google.com
       - example.com
services:
  - type: spiffe-workload-api
    listen: unix:///Users/noah/code/gravitational/workload-identity-experiment/workload.sock
    svids:
      - path: /bar
      - path: /foo

changelog: SPIFFE SVID generation introduced to tbot - experimental

[ibeckermayer]

Flagging to double check whether we want to add any non-default KeepaliveParams here:

teleport/lib/service/service.go

Lines 5965 to 5981 in fe345d4

	grpc.KeepaliveParams(keepalive.ServerParameters{
	// Using an aggressive idle timeout here since this gRPC server
	// currently only hosts the join service, which has no need for
	// long-lived idle connections.
	//
	// The reason for introducing this is that teleport clients
	// before #17685 is fixed will hold connections open
	// indefinitely if they encounter an error during the joining
	// process, and this seems like the best way for the server to
	// forcibly close those connections.
	//
	// If another gRPC service is added here in the future, it
	// should be alright to increase or remove this idle timeout as
	// necessary once the client fix has been released and widely
	// available for some time.
	MaxConnectionIdle: 10 * time.Second,
	}),

[ibeckermayer]

Consider whether we want the hard Stop vs GracefulStop here

[strideynet]

Good catch, I had GracefulStop originally but the long-lived nature of the Workload API streaming RPCs means that GracefulStop will just hang as these connections don't close.

[ibeckermayer]

Elsewhere we wait for the channel associated with the rootReloadBroadcaster for deciding that we're undergoing a cert renewal, should we do the same here (or is this something else / I'm just confused)?

teleport/lib/tbot/service_spiffe_workload_api.go

Lines 273 to 279 in fe345d4

	reloadCh, unsubscribe := s.rootReloadBroadcaster.subscribe()
	defer unsubscribe()
	for {
	select {
	case <-ctx.Done():
	return nil
	case <-reloadCh:

[strideynet]

reloadCh is fulfilling the same role here. Essentially, we have another goroutine which watches rootReloadBroadcaster and then fetches the new CA bundle and then sends a message on reloadCh. This just allows us to do a little bit of work before letting the RPCs know and prevents all the RPCs having to complete that work individually.

[timothyb89]

This is looking really good! I played around a bit with various role RBAC combinations and tried the socket out with spiffe-helper, all with no issues.

(Oddly spiffe-helper can only dump the first cert so it's not a great test if multiple are requested, ah well)

[timothyb89]

No real comments, just neat to see this test use the actual SPIFFE client on our socket 🙂

[strideynet]

Easiest way :D I've found my testing life has gotten 90% easier since focussing on trying to write less unit tests, and more high-level real world tests. We're very lucky in Machine ID that we're not restrained by trying to reproduce user input and doing things programatically is inline with the product.

[ryanclark]

bot

[public-teleport-github-review-bot]

@strideynet See the table below for backport results.

Branch	Result
branch/v15	Failed