Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VL][GLUTEN-8020] Remove the libhdfs3 installation script required for static linking #8013

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

JkSelf
Copy link
Contributor

@JkSelf JkSelf commented Nov 21, 2024

What changes were proposed in this pull request?

We will only retain the dynamic libraries libhdfs.so or libhdfs3.so at runtime based on this benchmark here. So there is no need to keep the libhdfs3 installation script required for static linking.

How was this patch tested?

Existing compile scripts.

Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

@PHILO-HE
Copy link
Contributor

@JkSelf, can we also remove ENABLE_HDFS option from some build scripts?

@zhouyuan
Copy link
Contributor

@JkSelf could you please help to link with the right Issue?

@JkSelf JkSelf changed the title Clean up the libhdfs3 install [GLUTEN-8020] Clean up the libhdfs3 install Nov 22, 2024
@JkSelf JkSelf changed the title [GLUTEN-8020] Clean up the libhdfs3 install [GLUTEN-8020] Remove the libhdfs3 installation script required for static linking Nov 22, 2024
Copy link

#8020

@PHILO-HE
Copy link
Contributor

@JkSelf, can we also remove ENABLE_HDFS option from some build scripts?

Seems we still need it for deciding whether to register hdfs filesystem.

@JkSelf
Copy link
Contributor Author

JkSelf commented Nov 22, 2024

@JkSelf, can we also remove ENABLE_HDFS option from some build scripts?

Seems we still need it for deciding whether to register hdfs filesystem.

Yes. We also need to pass the enable_hdfs to velox compile scripts.

@FelixYBW
Copy link
Contributor

@JkSelf is document updated?

@FelixYBW FelixYBW changed the title [GLUTEN-8020] Remove the libhdfs3 installation script required for static linking [VL][GLUTEN-8020] Remove the libhdfs3 installation script required for static linking Nov 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants