Multiple Volumes in ECS Compose pointing to same EFS Filesystem

[Rfferrao87]

I would like to propose the possibility of creating a single MountTarget for multiple Access Points in ECS compose. I am quite new to AWS and not sure if it would be a good practice, however I do not like having to create multiple EFS Filesystems to accomodate every volume defined in my compose file.

For example, I have the following compose file:

version: '3.7'

services:
  ftp-server:
    container_name: my-ftp-server
    environment:
      - FTP_PASS=123
      - FTP_USER=user
    image: garethflowers/ftp-server
    ports:
      - '20-21:20-21/tcp'
      - '40000-40002:40000-40002/tcp'
    volumes:
      - 'config-dir:/home/user/config'
      - 'data-dir:/home/user/data'

volumes:
  config-dir:
    external: true
    name: fs-0211e5a42036764bd
  data-dir:
    external: true
    name: fs-0211e5a42036764bd

Problem is, when I convert it to the Cloudformation template, it tries to create 2 Mount Targets for each volume, with the same subnet, in the same Filesystem. Example below:

AWSTemplateFormatVersion: 2010-09-09
Resources:
...
  ConfigdirAccessPoint:
    Properties:
      AccessPointTags:
      - Key: com.docker.compose.project
        Value: docker-ftp-server
      - Key: com.docker.compose.volume
        Value: config-dir
      - Key: Name
        Value: fs-0211e5a42036764bd
      FileSystemId: fs-0211e5a42036764bd
    Type: AWS::EFS::AccessPoint
  ConfigdirNFSMountTargetOnSubnet06d4cbe3216168915:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-06d4cbe3216168915
    Type: AWS::EFS::MountTarget
  ConfigdirNFSMountTargetOnSubnet007a8c32c41574882:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-007a8c32c41574882
    Type: AWS::EFS::MountTarget
  ConfigdirNFSMountTargetOnSubnet019c921272db60f58:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-019c921272db60f58
    Type: AWS::EFS::MountTarget
  ConfigdirNFSMountTargetOnSubnet045795de25fae67b0:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-045795de25fae67b0
    Type: AWS::EFS::MountTarget
  ConfigdirNFSMountTargetOnSubnet0a1e195edb8c3e37d:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-0a1e195edb8c3e37d
    Type: AWS::EFS::MountTarget
  ConfigdirNFSMountTargetOnSubnet0b9991c2f7782cadb:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-0b9991c2f7782cadb
    Type: AWS::EFS::MountTarget
  DatadirAccessPoint:
    Properties:
      AccessPointTags:
      - Key: com.docker.compose.project
        Value: docker-ftp-server
      - Key: com.docker.compose.volume
        Value: data-dir
      - Key: Name
        Value: fs-0211e5a42036764bd
      FileSystemId: fs-0211e5a42036764bd
    Type: AWS::EFS::AccessPoint
  DatadirNFSMountTargetOnSubnet06d4cbe3216168915:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-06d4cbe3216168915
    Type: AWS::EFS::MountTarget
  DatadirNFSMountTargetOnSubnet007a8c32c41574882:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-007a8c32c41574882
    Type: AWS::EFS::MountTarget
  DatadirNFSMountTargetOnSubnet019c921272db60f58:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-019c921272db60f58
    Type: AWS::EFS::MountTarget
  DatadirNFSMountTargetOnSubnet045795de25fae67b0:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-045795de25fae67b0
    Type: AWS::EFS::MountTarget
  DatadirNFSMountTargetOnSubnet0a1e195edb8c3e37d:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-0a1e195edb8c3e37d
    Type: AWS::EFS::MountTarget
  DatadirNFSMountTargetOnSubnet0b9991c2f7782cadb:
    Properties:
      FileSystemId: fs-0211e5a42036764bd
      SecurityGroups:
      - Ref: DefaultNetwork
      SubnetId: subnet-0b9991c2f7782cadb
    Type: AWS::EFS::MountTarget

The example above fails because of the aforementioned situation, so would it be pertinent to force multiple volumes, that are pointing to the same Filesystem ID, use the same Mount Targets when running ECS Compose?

[Adesoji1]

@Rfferrao87 The issue you described seems to be an architectural limitation due to the fact that in AWS, a single EFS (Elastic File System) filesystem can have multiple mount targets, but these must be in different subnets. If your compose file is trying to create two mount targets in the same subnet for the same EFS filesystem, it would indeed cause an error.

A possible solution to this could be to modify the ECS Compose conversion process to support using multiple access points on a single EFS filesystem. Each access point provides a specific directory on the filesystem and can enforce a certain POSIX user and group ID. This way, you can create a unique access point for each volume in your Docker Compose file, while still using the same mount target.

Here is the general outline of the steps you might take:

1. When converting from the Docker Compose file to CloudFormation, if multiple volumes are identified that refer to the same EFS filesystem, create a single mount target for them.

2. For each volume, create a unique EFS access point. Set the FileSystemId to your EFS filesystem ID and the Path to the specific directory for that volume.

3. Modify your Docker Compose file to mount the volume from the access point, not directly from the EFS filesystem.

[Rfferrao87]

@Adesoji1 I appreciate the idea, but I'm almost 100% sure I tried this before; problem is that Docker Compose's ECS Integration currently only supports linking to an external File System rather than an Access Point, at least as far as I'm aware. If there's another way to point directly to an Access Point I'd appreciate if you could share this method (regarding the docker-compose.yml syntax).