Introduction of the optional authentication mechanism for Labgrid (i.e. Oauth2) for securing gRPC communication between coordinator, client, exporter

[JacekBartynowski]

Hello,

I work for ARM Ltd.
We are planning to use the Labgrid for our testing and releases purposes (part of the build and CI/CD pipelines).

Currently, the main obstacle from our perspective (commercial organization) is the lack of an authentication/authorization.
I am aware that the Labgrid is an open-source, publicly available product that can be adapted by many organizations/projects, and the authentication/authorization mechanism may not be the desired thing.

I would like to get your opinions about the potential introduction of the authentication mechanism to secure a communication between the labgird-client(s), labgrid-cooridnator, and labgrid-exporter. The gRPC protocol provides many options to configure and use authentication.

The potential authentication mechanism would probably be OAuth2 with the external authentication provider (like Github, Google, Microsoft, Facebook, etc.), the mechanism would require to:

• Obtain the SSO (single-sign-on) token from the external authentication provider (OAuth2, OpenID based solution)
• Obtain SSL certificates (file system, remote storage solution like vault, etc.)

The scope of changes would probably include:

• adding selector mechanism to apply/use/configure the authentication or skip it
• adding functionality to obtain SSO token (OAuth2, OpenID standard)
• adding functionality to obtain SSL certificate(s)
• changes in labgrid/remote/client.py
• changes in labgrid/remote/coordinator.py
• changes in labgrid/remote/exporter.py

If possible, I would like to learn your opinions and views on:

• Possibility of adding optional (i.e. configurable via command line arguments) authentication mechanism
• Selection of the potential authentication mechanism
• Probable timeline of reviewing/accepting and releasing the modifications related to the authentication
• Maybe you consider any other authentication mechanism in the past?
  ?

[Emantor]

Hi,

I'd like you to know that we have read your questions and need to discuss this internally before posting an answer, thanks for your patience.

[jluebbe]

For now, we've tried to keep the complexity in labgrid low by relying on network- and SSH-level authentication to restrict access to our labgrid instance. I understand that in some scenarios, not all users of a labgrid instance should have access to all places.

At a high level, we have mainly three kinds of connections that might need authentication:

1. client to coordinator via gRPC (for an interactive user or a CI run)
2. exporter to coordinator
3. client to exporter via SSH

For 1., I think token based authentication makes a lot of sense. The token can be passed as headers in gRPC and checked by the coordinator. I don't really have an opinion how the coordinator would check the token and then authorized the client for specific resources, so I'd be open to suggestions. The client could use a system-wide configuration to select some Python backend module which is then used to acquire a token from SSO (or from somewhere else).

For 2., SSO doesn't really seem applicable, as the exporter runs as a service, without any specific (human) user being involved. So, perhaps a fixed token or TLS client certificates would be more appropriate.

In a scenario with authentication, direct TCP connection to PDUs or other resource would likely be block by a firewall, so all actual control of the DUT would happen via SSH. Either by executing commands (fastboot, dd, imx-usb-loader, ...), uploading and running labgrid Python agents or by configuring port forwards. Perhaps this can be solved by assigning short-lived SSH certificates to clients after they have locked a place which allows targeted access to only that related resources on that exporter. There are open questions, though:

• use one or multiple UNIX users on the exporter?
• create UNIX users dynamically and remove them after unlock (perhaps using systemd's homed or something similar)
• restrict access to devices in /dev using ACLs or UNIX groups?

[JacekBartynowski]

@Emantor @jluebbe
Would it be possible to assign me (user: JacekBartynowski@github) privileges to create MR?
Alternatively, I could post a patch/diff file.

I tested the client, exporter, coordinator changes with optional authentication (securing channels with SSL certificates/keys + adding middleware - sample for the official gRPC repository).

[JacekBartynowski]

Could you please take a look at the attached patch file (output from the 'git diff' command; the base for the patch is the commit: 0304ec6 Fri Nov 1, branch 'master'): grpc_authentication_proposal_secure_grpc_channels.txt

The most significant modifications:

• adding new command line input argument '-A' to the coordinator, client, and exporter;
  when enabled, the coordinator, client, and exporter use the secure channels (SSL encryption), it allows using the authentication based on the JWT token added to HTTP/2 headers;
  this option is disabled by default (users should not notice any change, by default the Labgrid would still use insecure gRPC channels)

• adding self-signed certificates for local development and debugging purposes (certificates directory in 'labgrid/remote' directory)

• adding an initial version of the authentication code based on the examples available in the official gRPC repository
  The authentication employs standard gRPC authentication instruments such as grpc.AuthMetadataPlugin and grpc.aio.ServerInterceptor

We're going to use the Python 'entry-points' to deliver the authentication part of the code as a plugin; the work is still in progress.

Would you accept:

• adding new command line argument: '-A' to use gRPC.secure channels
• adding SSL certificates (I can add input argument to specify a path to the directory with the certificates)
• using Python 'entry-points' to handle and customize the gRPC authentication (defining custom gRPC plugins and interceptor)
  grpc_authentication_proposal_secure_grpc_channels.txt

[Emantor]

@Emantor @jluebbe Would it be possible to assign me (user: JacekBartynowski@github) privileges to create MR? Alternatively, I could post a patch/diff file.

You should be able to open a Pull Requests without additional permissions.

[JacekBartynowski]

here is a problem I am facing (when trying to push my commit) (I cloned using ssh protocol):

ERROR: Permission to labgrid-project/labgrid.git dened to JacekBartynowski

UPDATE: I forked the repository - I will use the fork to submit the proposal (apologies for the confusion)

[JacekBartynowski]

I managed to create a MR for potential changes:
#1541

• adding new cmd input argument
• introducing of gRPC secure channel (SSL encryption) - disabled by default
• very basic authentication using gRPC code (grpc.AuthMetaPlugin + grpc.aio.ServerInterceptor)

The authentication part will be re-worked (decoupling; probably using abstract interface + concrete implementation in additional package integrated via 'entry points')

[JacekBartynowski]

Hi @jluebbe
Concerning your questions from the previous message (~2 weeks ago)

There are open questions, though:
* use one or multiple UNIX users on the exporter?

* create UNIX users dynamically and remove them after unlock (perhaps using systemd's homed or something similar)

* restrict access to devices in `/dev` using ACLs or UNIX groups?

We would like to use the OAuth2/OIDC (Open ID Connect) tokens with configurable roles + scopes. This would allow us to minimize the number of configuration changes and avoid configuring UNIX users or ACLs.

We are aiming at generating tokens (JWT OAuth) with different privileges and for different purposes (so only selected, authenticated users would be able to delete the place, etc.).
The tokens would be used and decoded by the custom authentication plugins, derived from the grpc.AuthMetadataPlugin and grpc.aio.ServerInterceptor classes.
The user's details would be read by the custom plugin from the .netrc file present on the machine (client).

The usage of the standard Python 'entry-points' instrumentation would allow us to instantiate and use such plugins delivered as separate Python packages (installed within the labgrid virtual environment) - effectively avoiding polluting the labgrid code base with the company's specific implementations.

The proposed authentication mechanism (using encrypted gRPC channels + passing JWT tokens in HTTP/2 headers):
#1541

is based on the sample solutions available in the official gRPC repository:
https://github.com/grpc/grpc/tree/master/examples/python/auth

We believe this is an effective and lightweight way to implement authentication.

Would it be possible to get your opinion about this proposed solution: #1541?