Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for parquet and arrow #6

Closed
kdpsingh opened this issue Apr 12, 2024 · 2 comments
Closed

Add support for parquet and arrow #6

kdpsingh opened this issue Apr 12, 2024 · 2 comments

Comments

@kdpsingh
Copy link
Member

No description provided.

@kdpsingh
Copy link
Member Author

kdpsingh commented Apr 12, 2024

The main use case for working with arrow is to be able to work on Arrow frames that have been instantiated into shared memory. So I eventually need to add Arrow support to TidierData for working with arrow in a way that does not produce a copy of the entire data frame.

For now, the idea would be that we will instantiate both types of files as a data frame in memory (just like we do with CSV files).

@drizk1
Copy link
Member

drizk1 commented Apr 12, 2024

Sounds good. Reading them in as dfs will be pretty straightforward.

I'm curious to see what making tidierdata work directly with arrow frames will look like

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants