-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dataset chapter #328
Dataset chapter #328
Conversation
discuss trade-offs between: - local databases like SQLite & DuckDB - data processing libraries like Pandas, Vaex, & Polars Co-authored-by: Flavio Hafner <[email protected]>
Nice! I have also used DuckDB in combination with |
Hi @maltelueken that would be amazing! This also addresses the last point in the DuckDB part about combining with other tools. We were also lacking R experience, so couldn't comment on R libraries. |
@Morrizzzzz Would you be interested and have time to review this? |
The chapter could be more about data engineering i.e. how to use these tools or best practices for ETL pipelines. |
@recap do you have some resources to link to on data engineering and/or ETL pipelines? Sounds like a nice addition (for a new PR). We should try to restrict it to techniques/concepts we actually (can) use in projects. I think you have done some of that, no? |
Co-authored-by: Patrick Bos <[email protected]>
Co-authored-by: Bouwe Andela <[email protected]>
Co-authored-by: recap <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome addition, thanks all!
Remove trivial database section from Python guide
Add a chapter on datasets
I followed the CONTRIBUTING guidelines.
Below, describe what this Pull Request adds:
This PR removes the database section from the Python guide (as
discussed in #316), and introduces a new chapter on handling datasets.
It discusses using local databases, and other data processing
libraries, and respective trade-offs.