Add IPv6 support

[hellais]

In order to support IPv6 on AWS we had to do a major rework of the networking configuration of the VPC and ECS clusters.

In case you like to enjoy some some trivia about why IPv6 support is so weak in AWS, check these links:

• https://news.ycombinator.com/item?id=37608900
• https://github.com/DuckbillGroup/aws-ipv6-gaps
• [ECS] : ECS Support for IPv6 aws/containers-roadmap#1340

More details follow:

• ECS tasks network mode has been switched to awsvpc, which is the official way to support IPv6.
• Separate the networking stack into private and public. The routing table of the public network has direct IPv4 and IPv6 routes configured, while the private network uses NAT to route to the internet.
• For NAT to work we also need to allocate elastic IPs, one for each availability zone we would like to support.

In order to get the networking to work properly in ECS, we setup the container host to make use of the public network to have direct access to the internet, while the ECS container itself uses the private NATed network. This is the desired configuration because we don't want to directly expose the container to the internet, but rather map the container port to the public internet accessible endpoint via load balancer.

[hellais]

Taking note of some stuff:

% aws ec2 describe-instance-types --filters "Name=instance-type,Values=t3.*" --query "InstanceTypes[].{Type: InstanceType, MaxENI: NetworkInfo.MaximumNetworkInterfaces, IPv4addr: NetworkInfo.Ipv4AddressesPerInterface}" --output table
--------------------------------------
|        DescribeInstanceTypes       |
+----------+----------+--------------+
| IPv4addr | MaxENI   |    Type      |
+----------+----------+--------------+
|  15      |  4       |  t3.2xlarge  |
|  15      |  4       |  t3.xlarge   |
|  6       |  3       |  t3.medium   |
|  12      |  3       |  t3.large    |
|  2       |  2       |  t3.nano     |
|  2       |  2       |  t3.micro    |
|  4       |  3       |  t3.small    |
+----------+----------+--------------+

% aws ec2 describe-instance-types --filters "Name=instance-type,Values=t2.*" --query "InstanceTypes[].{Type: InstanceType, MaxENI: NetworkInfo.MaximumNetworkInterfaces, IPv4addr: NetworkInfo.Ipv4AddressesPerInterface}" --output table
--------------------------------------
|        DescribeInstanceTypes       |
+----------+----------+--------------+
| IPv4addr | MaxENI   |    Type      |
+----------+----------+--------------+
|  15      |  3       |  t2.2xlarge  |
|  12      |  3       |  t2.large    |
|  2       |  2       |  t2.micro    |
|  6       |  3       |  t2.medium   |
|  4       |  3       |  t2.small    |
|  15      |  3       |  t2.xlarge   |
|  2       |  2       |  t2.nano     |
+----------+----------+--------------+

Basically with t2.micro, we can only have 1 additional ENI, while with small-large we can only do 2 additional. Effectively if we switch to the awsvpc networking mode we will have to allocate a full small instance to running a maximum of 2 services each assuming we use small+ base instance size.