From 6a167001244eb92cd96f83fa2598213990a91359 Mon Sep 17 00:00:00 2001 From: Michael Gutteridge Date: Fri, 15 Nov 2024 12:58:59 -0800 Subject: [PATCH 1/3] Add callouts to dataprotection --- _scicomputing/hutchgo_overview.md | 3 +++ _scicomputing/store_collaboration.md | 3 +++ 2 files changed, 6 insertions(+) diff --git a/_scicomputing/hutchgo_overview.md b/_scicomputing/hutchgo_overview.md index efc00c5e9..2c64ac7c1 100644 --- a/_scicomputing/hutchgo_overview.md +++ b/_scicomputing/hutchgo_overview.md @@ -8,6 +8,9 @@ HutchGO- Hutch Globus Online- is a deployment of the Globus data transfer servic Globus is a non-profit service of the University of Chicago. It is a combination of software (the Globus server and client tools) and services (the Globus control services) providing a number of different functions useful in research- data transfer, compute, and automation. +We have a number of agreements in place with the Globus team that make it the preferred way to transfer sensitive data but it is essential that you are confident that transferring data with Globus won't violate any terms of use- contact `dataprotection@fredhutch.org` for advice regarding your data transfer. +{: .notice--warning} + ## Key Concepts ### Account and Identity diff --git a/_scicomputing/store_collaboration.md b/_scicomputing/store_collaboration.md index 44fa7f806..157ecfd4c 100644 --- a/_scicomputing/store_collaboration.md +++ b/_scicomputing/store_collaboration.md @@ -5,6 +5,9 @@ primary_reviewers: vortexing ## Data Transfer Tools +Before transferring data, please make sure you have appropriate protections in place- make sure you are allowed to share data via your desired mechanism, that agreements are in place or won't be violated by transferring data. Contact `dataprotection@fredhutch.org` if you have questions or need assistance evaluating your transfer needs. +{: .notice--warning} + In collaborative settings, often data transfer and sharing can pose significant challenges. For specific guidance about data transfer, access or copying for your particular collaboration, please file a ticket including a basic description of the data access sitaution by emailing `scicomp`. There are often suitable strategies for collaborative project support that do not require data duplication (and thus increased data storage costs), or use of expensive data storage or transfer tools. Two useful data transfer tools at use at the Hutch include `Motuz` and `Aspera`. ### HutchGO From 8cd812c9dbb652c2f2db192f5692ded81901bb4b Mon Sep 17 00:00:00 2001 From: Michael Gutteridge Date: Fri, 15 Nov 2024 13:09:00 -0800 Subject: [PATCH 2/3] Update storage transfer tools, remove motuz as it isn't used for collaboration --- _scicomputing/store_collaboration.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/_scicomputing/store_collaboration.md b/_scicomputing/store_collaboration.md index 157ecfd4c..1f828729b 100644 --- a/_scicomputing/store_collaboration.md +++ b/_scicomputing/store_collaboration.md @@ -3,35 +3,36 @@ title: Data Transfer and Storage Tools for Collaborating primary_reviewers: vortexing --- -## Data Transfer Tools - Before transferring data, please make sure you have appropriate protections in place- make sure you are allowed to share data via your desired mechanism, that agreements are in place or won't be violated by transferring data. Contact `dataprotection@fredhutch.org` if you have questions or need assistance evaluating your transfer needs. {: .notice--warning} -In collaborative settings, often data transfer and sharing can pose significant challenges. For specific guidance about data transfer, access or copying for your particular collaboration, please file a ticket including a basic description of the data access sitaution by emailing `scicomp`. There are often suitable strategies for collaborative project support that do not require data duplication (and thus increased data storage costs), or use of expensive data storage or transfer tools. Two useful data transfer tools at use at the Hutch include `Motuz` and `Aspera`. +In collaborative settings, often data transfer and sharing can pose significant challenges. Access management, authentication, and data-use agreements all affect how you share data both within the Hutch and with external collaborators. + +## Data Transfer Tools ### HutchGO [HutchGO (Hutch Globus Online)](/scicomputing/hutchgo_overview) is an instance of Globus maintained by SciComp for sharing and moving data. HutchGO enables secure and high performance transfers of data internally and externally. -### Motuz - -[Motuz](https://motuz.fredhutch.org) is a tool that facilitates the transfer of small or large data between Fred Hutch storage locations (such as `Scratch` and `Fast`) and cloud storage locations such as AWS S3 buckets among others. You can find some basic how-to guidance to get started with Motuz in our [Resource Library](/compdemos/motuz/). +HutchGO allows transfers of data in-place from common Hutch storage locations (fast, working, temp). It isn't necessary to copy data into another location prior to sharing. ### Aspera Aspera is a data transfer tool that enables fast transfer of large data between it and a host using the Aspera client. The primary method of operation is to upload the data to the Aspera server, then use the web interface to create an email with a link you would then send to those outside the Hutch network. -> Note: space is limited. Because of this, data stored here is deleted after a short period of time and not backed up making the Aspera inappropriate for primary storage. Moving data to their final, secure, backed up locations as soon as possible is important when using Aspera. +Storage space in Aspera is limited. Because of this, data stored here is deleted after a short period of time and not backed up making the Aspera inappropriate for primary storage. Moving data to their final, secure, backed up locations as soon as possible is important when using Aspera. +{: .notice--info} Visit [the Aspera information page](/scicomputing/store_aspera/) for more details and information on using this storage service. ## Data Storage for Collaboration ### AWS S3 + AWS S3 storage has a variety of useful features that make it an ideal way to securely share data with collaborators, or leverage a resource as a common storage location for users at and outside of the Hutch. You can read more about how S3 can help support your collaborations using especially large data sets [in our AWS S3 storage documentation](/scicomputing/store_objectstore/). ### OneDrive + OneDrive is a cloud service that securely stores your files and folders in one place, share them with others, and update your files from any device. OneDrive is a benefit available individual users at the Fred Hutch that allows for private storage of files with the ability to share those files with others for collaboration. With OneDrive you can: - Create documents on your computer and edit on your laptop, phone, or tablet From 0fa26838a205f9bfb0c2522a9a9f437676b4403d Mon Sep 17 00:00:00 2001 From: Michael Gutteridge Date: Thu, 21 Nov 2024 05:28:13 -0800 Subject: [PATCH 3/3] Update _scicomputing/hutchgo_overview.md Co-authored-by: Taylor Firman <37599954+tefirman@users.noreply.github.com> --- _scicomputing/hutchgo_overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_scicomputing/hutchgo_overview.md b/_scicomputing/hutchgo_overview.md index 2c64ac7c1..5f4bc102a 100644 --- a/_scicomputing/hutchgo_overview.md +++ b/_scicomputing/hutchgo_overview.md @@ -8,7 +8,7 @@ HutchGO- Hutch Globus Online- is a deployment of the Globus data transfer servic Globus is a non-profit service of the University of Chicago. It is a combination of software (the Globus server and client tools) and services (the Globus control services) providing a number of different functions useful in research- data transfer, compute, and automation. -We have a number of agreements in place with the Globus team that make it the preferred way to transfer sensitive data but it is essential that you are confident that transferring data with Globus won't violate any terms of use- contact `dataprotection@fredhutch.org` for advice regarding your data transfer. +Globus is the preferred way to transfer sensitive data, but it is essential that you are confident that transferring data with Globus won't violate any terms of use - contact `dataprotection@fredhutch.org` for advice regarding your data transfer. {: .notice--warning} ## Key Concepts