Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hadoop FileSystem delete method returns false even though file is actually deleted #909

Open
selimelawwa opened this issue Nov 22, 2022 · 0 comments

Comments

@selimelawwa
Copy link

In my Java application I have an implementation for a file-system, where my file class is a wrapper for Hadoop filesystem methods. I am upgrading the from hadoop3-1.9.17 to hadoop3-2.2.8 and I am using the shaded jar of the new version.

My delete method implementation is as follows:

@Override
public boolean delete(boolean recursively) {
    return fs.delete(path, recursively);
}

Where fs is an instance of org.apache.hadoop.fs.FileSystem so this method calls org.apache.hadoop.fs.FileSystem.delete(Path, boolean)

Frequently when calling the above delete method on a directory which contains files it returns false 'meaning dir/file NOT deleted' however when checking the directory is actually deleted at Google Cloud Storage.

Is there a change in the upgraded GCS connector causing this (caching of file-status i.e) or is there a configuration to fix this issue?
I also created this [stackoverflow question](https://stackoverflow.com/questions/74445091/gcs-connector-hadoop3-hadoop3-2-2-8-filesystem-delete-returns-false-but-file-dir but got no answers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant