Add ssh capability between deployed containers (odpi-egeria-lab)

[planetf1]

When building demos in our coco pharma environment, on occasion we need to similar things happening on different systems - outside the scope of egeria operations. In particular this is often in the context of running a demo 'script' via a notebook

For example

• Files being created/deleted in a filesystem
• Manipulation of a third party tool (for example, postgres) to simulate activity
• manipulating files being they are made available to the server, and there there is no client access (intentionally, ie with content packs)
• Directly running egeria samples and utilities
• viewing logs, files, perhaps grepping out for errors or demonstrating what one might expect to see - not just for egeria

In some cases there are also API possibilities (files, postgres), sometimes not (egeria utilities -- if we want to demonstrate the utility itself, rather than the API)

Ensuring ssh access between these containers USED FOR A DEMO, particularly from the jupyter environment to the other containers, will make it possible to more quickly develop these scenarios, often before more fundamental changes (adding a file server, finding the best python libraries to use etc) is in place - ie more adaptable.

Note that access to the k8s cli is another option (kubectl exec) but in general I would err on using ssh as it's more understandable for most -- unless it is k8s itself being demoed.

Also note that our containers DO NOT RUN AS ROOT, so there are some limits in what can be run. Also containers in general are cut down in what commands are installed.

[planetf1]

Investigating:

• Using ssh is complicated by the fact we'd have to run another daemon in container, which would mean some changes to the initialization for the minimal image which can get messy. The full image adds init framework - easier - but doubles the size of the container ..
• Adding a sidecar container is simpler, but increases footpring
• kubectl may be the better option. However appears not to be in the default distro
• Getting kubectl via curl is fine, but this can't be run in the Dockerfile as this is RUN at build time, not run time -- so we wouldn't have the right architecture binary ....

Adding '-v10' to any kubectl command prints out the equivilent curl command to use. For example
kubectl exec lab-odpi-egeria-lab-core-0 -- /bin/ls becomes

I0210 18:52:39.051518   38389 round_trippers.go:466] curl -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: kubectl/v1.23.3 (darwin/amd64) kubernetes/816c97a" 'https://c108-e.eu-gb.containers.cloud.ibm.com:31413/api/v1/namespaces/lab/pods/lab-odpi-egeria-lab-core-0'
I0210 18:52:39.432094   38389 round_trippers.go:466] curl -v -XPOST  -H "X-Stream-Protocol-Version: v4.channel.k8s.io" -H "X-Stream-Protocol-Version: v3.channel.k8s.io" -H "X-Stream-Protocol-Version: v2.channel.k8s.io" -H "X-Stream-Protocol-Version: channel.k8s.io" -H "User-Agent: kubectl/v1.23.3 (darwin/amd64) kubernetes/816c97a" 'https://c108-e.eu-gb.containers.cloud.ibm.com:31413/api/v1/namespaces/lab/pods/lab-odpi-egeria-lab-core-0/exec?command=%2Fbin%2Fls&container=egeria&stderr=true&stdout=true'

This is cumbersome.

The intent of this issue was to look at how to invoke commands from the Jupyter container to the egeria - or other containers. If we restrict the change to jupyter only we can now just consider issuing the commands from Python.

Therefore the easiest root is actually to use https://github.com/kubernetes-client/python pip install kubernetes since the required credentials should be available under /var/run/secrets, placed there automatically by k8s