Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add limitation regarding EFS file system #333

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jason-seqera
Copy link
Contributor

@jason-seqera jason-seqera self-assigned this Dec 5, 2024
Copy link

netlify bot commented Dec 5, 2024

Deploy Preview for seqera-docs ready!

Name Link
🔨 Latest commit 0abea09
🔍 Latest deploy log https://app.netlify.com/sites/seqera-docs/deploys/6758f675480e8600084d855b
😎 Deploy Preview https://deploy-preview-333--seqera-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@@ -26,6 +26,10 @@ Before you get started, you need the following:
Currently, Data Studios only supports AWS Batch compute environments that **do not** have Fargate enabled.
:::

## Limitations

If you configured your AWS Batch compute environment to include an EFS file system with **EFS file system > EFS mount path**, the mount path cannot be the same as your work directory. When the work directory and EFS mount path are identical, the Data Studio snapshots fail causing a data studio instance to fail. For more information on AWS Batch configuration, see [AWS Batch][aws-batch].
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!
The only thing I would also add that the mount path cannot be empty either (because then it defaults to the work directory, leading to failure)

Enter the EFS mount path (defaults to the pipeline work directory root path if omitted). 

@@ -26,6 +26,10 @@ Before you get started, you need the following:
Currently, Data Studios only supports AWS Batch compute environments that **do not** have Fargate enabled.
:::

## Limitations

If you configured your AWS Batch compute environment to include an EFS file system with **EFS file system > EFS mount path**, the mount path cannot be the same as your work directory. When the work directory and EFS mount path are identical, the Data Studio snapshots fail causing a data studio instance to fail. For more information on AWS Batch configuration, see [AWS Batch][aws-batch].
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you configured your AWS Batch compute environment to include an EFS file system with **EFS file system > EFS mount path**, the mount path cannot be the same as your work directory. When the work directory and EFS mount path are identical, the Data Studio snapshots fail causing a data studio instance to fail. For more information on AWS Batch configuration, see [AWS Batch][aws-batch].
If you configured your AWS Batch compute environment to include an EFS file system with **EFS file system > EFS mount path**, the mount path cannot be the same as your work directory as this will cause the Data Studio snapshots, and session, to fail. The mount path cannot be left empty, as this will default to the work directory and also result in snapshot and session failure.
For more information on AWS Batch configuration, see [AWS Batch][aws-batch].

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@endre-seqera, does the suggested wording work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you configured your AWS Batch compute environment to include an EFS file system with EFS file system > EFS mount path, the mount path cannot be the same as your work directory as this causes the Data Studio snapshots, and session, to fail. You must specify a mount path for the EFS file system or the mount path is set to your work directory.

maybe

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last sentence is highly confusing (for me, at least)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about stating this in only one sentence:

If you configured your AWS Batch compute environment to include an EFS file system with EFS file system > EFS mount path, the mount path has to be explicitly specified and it cannot be the same as your CE work directory, as this causes the Data Studio snapshots, and session, to fail.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@endre-seqera, I'm trying to avoid too much of a run-on sentence.

What does it mean for the session to fail exactly? It crashes? There aren't any checkpoints saved and user work vanishes after stopping a session?

Copy link
Member

@endre-seqera endre-seqera Dec 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case failure means that the data studios session will fail to start, crashes. As part of the startup process the Compute Environment's work directory is used to set up snapshot creation. However, since EFS is not supported for this, the startup process fails. And there will be no snapshot.

We don't support a way to start and use DataStudios without the snapshoting functionality, it cannot be turned-off, so in case of EFS workdir, it will crash, fail to start up.

@justinegeffen justinegeffen added 1. Editor review Needs a language review 2. Dev/PM review Needs a review by a Dev or PM labels Dec 6, 2024
@jason-seqera jason-seqera force-pushed the PLAT-997-efs-volume-mounting branch from 59f2e28 to 0abea09 Compare December 11, 2024 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1. Editor review Needs a language review 2. Dev/PM review Needs a review by a Dev or PM
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants