Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dataset copy and move functionality #28

Merged
merged 22 commits into from
Aug 8, 2022
Merged

Add dataset copy and move functionality #28

merged 22 commits into from
Aug 8, 2022

Conversation

savindi-wijenayaka
Copy link
Collaborator

Description:

Add functionality to copy or move data from a source directory to a primary directory of a SDS framework-based directory structure.

Related issue(s):

#20

Test Environment:

Ubuntu 20.04
Python 3.9.12

@savindi-wijenayaka
Copy link
Collaborator Author

savindi-wijenayaka commented Aug 7, 2022

Future Work:

  1. Change print to log (Repository-wide enhancements #29)
  2. Pandas DataFrame.append future warning on depricating (DataFrame.append to be replaced with DataFrame.concat #30 )
  3. Change Docstring style from spinx to Google or PEP257. Can use auto generators like AutoDocstring (Repository-wide enhancements #29)

@chinchien-lin
Copy link
Collaborator

chinchien-lin commented Aug 7, 2022

Hi @savindi-wijenayaka ,

I just uploaded some test daata in the "./test_data" folder, would you be able to use it and update your example so that we can all test it?

@PrasadBabarendaGamage
Copy link
Collaborator

Hi @savindi-wijenayaka Chinchien and I were looking through your example and noticed that:

  1. the data was not being moved to the dataset folder (primary)
  2. it seemed that a folder is requested from the user, just clarifying whether all the contents in that folder will be moved to the appropriate subject/sample folder in the primary folder? If so, please update the test data folder/files to help with demonstrating that (I think at the moment both samples are in a subject folder - we would just need to put the test data in something like: input_data_path = test_data/primary/subject1/sample1/ and a subject1/sample2/dummy.txt folder, and then provide

@chinchien-lin chinchien-lin merged commit 80f3970 into SPARC-FAIR-Codeathon:main Aug 8, 2022
chinchien-lin pushed a commit that referenced this pull request Jul 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants