Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tls.2] Docker socket protected with TLS #1792

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions examples/clusters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,10 @@ Once the nodes are up and running, it will run the appcelerator/ampadmin image t
| DockerChannel | channel for Docker installation | stable | edge |
| DockerPlugins | space separated list of plugins to install | | rexray/ebs |
| InstallApplication | install AMP | yes | no |
| NFSEndpoint | enable a NFSv4 service inside the VPC | no | yes |
| EnableSystemPrune | Enable Docker system prune | yes | no |
| MonitoringPort | Public port for the dashboard | 8080 | |
| EnableTLS | Docker socket secured with TLS | yes | |

## Output

Expand All @@ -47,8 +50,12 @@ The output of the stack lists the DNS name of the ELB in front of the manager no
| --------- | ----------- |
| VpcId | VPC ID |
| DNSTarget | public facing endpoint for the cluster, It can be used for ssh access, https access to swarm services and configuration of the remote server in the CLI |
| InternalDockerHost | Docker host for services requiring access to the secured API |
| InternalRegistryTarget | internal endpoint for the registry service |
| MetricsURL | URL for cluster health dashboard |
| NFSEndpoint | NFSv4 Endpoint |
| InternalPKITarget | internal endpoint for the PKI service |


## Custom AMI

Expand All @@ -69,3 +76,77 @@ An option of the template is the inclusion of a Docker registry.
It includes a S3 bucket as registry backend, and an autoscaling group of registry containers.
The registry is composed of non swarm nodes and is not part of the swarm.
The registry is only available from the VPC, all Docker swarm nodes are configured with the internal endpoint of the registry as mirror registry.

## Docker socket protected with TLS

When the EnableTLS option is set to yes, all swarm nodes are started with a certificate to protect the Docker socket. It is then available on port 2376 (instead of 2375 when not secured).
An autoscaling group with a single instance runs a cfssl docker container that is providing a CA, and serves an API for certificate generation. It is used by all the nodes to get the server certificate (for the Docker daemon) and a client certificate (for the Docker CLI).
The Manager external ELB has a listener on port 2376 that allows external access to the Docker engine API on the manager nodes (round robin). This is by default blocked by the security group, but can be open for a range of IP if a direct access is needed.
To be able to authenticate, you need the CA certificate, a key and a certificate. This is served by the cfssl container, but is not available from outside of the VPC (for security reason). This can be implemented as an AMP API, that would offer the interfaces with the PKS, generating and providing these 3 pem files to a client, with the added value of authorization. The API is not yet implemented.

#### Using the client certificate on a node of the swarm

Services can use the certificate available on the swarm. For instance, core services requiring access to the API on manager node can be scheduled on non manager nodes, and use the client certificate to connect to the API on the manager node. For that, the service has to mount the certificate from the host.

The certificate is available as well as its private key in /etc/docker: client.cert and client.key.

#### How to get the pem files for a Docker client

Identify the CA URL, it's the DNS name of the CA ELB. You can get it from the AWS console or from the status of the cluster (amp -s CLUSTER_URL cluster status). The URL should include the scheme.

The procedure below can be done only if you open the PKS service outside of the VPC, this is done by adding a rule to the PKS security group (look for CASecurityGroup).

From you client (usually a server from outside the swarm), do:
```
docker run --rm cfssl/cfssl:latest info -remote=CA_URL | jq -r .certificate
```

Paste the result on your machine in `~/.docker/ca.pem`.

Then, prepare a JSON file with the CSR.

```
{
"CN": "USERNAME",
"hosts": [
"$(hostname)"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "US",
"L": "Santa Clara",
"O": "Axway",
"OU": "AMP",
"ST": "California"
}
]
}
```


Submit the CSR and save the response:

```
docker run --rm -v $PWD/csr.json:/csr.json cfssl/cfssl:latest gencert -remote=CA_URL -profile client /csr.json > response.json
```

Extract the key on your machine:
```
jq -r .key < response.json > ~/.docker/client.cert
```

Extract the certificate on your machine:
```
jq -r .cert < response.json > ~/.docker/client.cert
```

You can now use the Docker CLI by setting these variables:
```
export DOCKER_TLS_VERIFY=1
export DOCKER_HOST=MANAGER_EXTERNAL_ELB_DNS_NAME:2376
docker info
```
Loading