Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NSFS : S3 rm with --recursive option does not delete all the objects #8421

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

naveenpaul1
Copy link
Contributor

Explain the changes

  1. Issue is with list_object api, list_object is failing with specific dir strucute and not returning complete items in that bucket.
  2. dir_key and marker_dir comparison will fail when there are dir with structure slimier to "-/migrateDir.popFSDir.3803140/" and "/migrateDir.popFSDir.3803140.OldSet/" because "/" considered greater than "." to fix this all the backslash is replaced with space. This updated marker and dir_key used only for comparison.

Dir structure like this is will fail

/populatefs_dir
--/migrateDir.popFSDir.3803140
----/sparseDir
-----lnk.0.qqqqqqqqqqqqqq
-----lnk.1.qqqqqqqqqqqqqq
-----lnk.2.qqqqqqqqqqqqqq
--/migrateDir.popFSDir.3803140.OldSet
----/sparseDir
-----lnk.0.qqqqqqqqqqqqqq
-----lnk.1.qqqqqqqqqqqqqq
-----lnk.2.qqqqqqqqqqqqqq

Issues: Fixed #xxx / Gap #xxx

  1. S3 upload is successful for a data set (mix of all type of objects, sparse files, files with long name, compressed files etc)
  2. When download via "aws-alias s3 rm s3://bucket-10909 --recursive" , this still displays the left over directories and objects in the bucket. I checked from the FS that the dataset still exists there.

#8197

closed : PR #8317

Testing Instructions:

  1. run NC_CORETEST=true node ./node_modules/.bin/mocha ./src/test/unit_tests/test_namespace_fs.js
  • Doc added/updated
  • Tests added

@naveenpaul1 naveenpaul1 force-pushed the object_delete_issues branch from 9f7cc2a to f91acba Compare October 1, 2024 06:35
@pull-request-size pull-request-size bot added size/L and removed size/M labels Oct 1, 2024
@naveenpaul1 naveenpaul1 force-pushed the object_delete_issues branch 3 times, most recently from fb2fe0c to 13e093c Compare November 5, 2024 11:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant