Environment Internal Metadata API and CLI

[akarasulu]

What is it?

This is about making a REST API that is only accessible from within an environment to get environment metadata and to have the environment push events up the stack to its peers and the Bazaar. It will have two parts:

• A REST API of various endpoints hosted at the gateway (or some fixed preset IP) in each environment segment
• A CLI program or script which mirror the REST API for convenience while scripting, and should be available in the base templates

No authentication is needed since the containers making the REST calls are within the

Why?

The facility is a general one. It can be used for any function. Some ideas of what we would like to use it for include:

• service discovery
• environment awareness
• container syslog service
• application alerts
• enabling scripts, playbooks, and blueprints to make decisions about how to configure themselves

This was asked for before several times but it never happened.

Precedents

This is not a unique idea. AWS uses this with it's Metadata API. It even has a REST API and a CLI component just as was stated here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

Also note that this API uses a non-routable special address to expose the rest API: http://169.254.169.254/latest/meta-data/

[dilshat]

We have an experimental endpoint that provides information about the environment.
From within a container it is possible to request the information to be put inside the containers filesystem ( it gets dumped to /root/env file).
The endpoint is GET https://10.10.10.1:8443/rest/v1/environments/{CONTAINER-IP}/info, where CONTAINER_IP is container's eth0 interface ip.

[akarasulu]

This is a good start. We just also need to support endpoints to send stuff from the container. IP can be used to key off of like this in the rest URI path. The dumping to the file system is a bad code smell but I guess we can use this info in any way. But I guess this just dumps a huge file with all env info instead of a granualar api ... not a problem for now.

We just need to be able to use this interface for applications to send events especially when building long running environment blueprint apps. Can you talk to @absidish and @tjamakeev about this to share your ideas. Thanks.

[akarasulu]

I like the idea of depositing the token, with TTL and only using the depositing to an address thing once to make sure it is secure. Can we revisit this stuff when you are around? I want to make sure we can enhance this feature to be robust and support all the things we want to do with it.

[akarasulu]

It will be nice also to document exactly how this works now and how it will work later in the future.

[dilshat]

The idea with token based access is this (it is just design but can be implemented based on this):

1. Client can reside inside ANY container
2. Client needs to know container IP from eth0 interface
3. Client requests token from Console by the REST endpoint: GET https://10.10.10.1:8443/rest/v1/environments/{CONTAINER-IP}/token
4. Token gets deposited into the agreed location on the container's filesystem
5. Token has TTL and is bound to the container's environment. Console allows to read information of and optionally perform actions only within the context of the container's parent environment