Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Python] Make NumPy an optional runtime dependency #25118

Closed
asfimport opened this issue Jun 1, 2020 · 2 comments
Closed

[Python] Make NumPy an optional runtime dependency #25118

asfimport opened this issue Jun 1, 2020 · 2 comments
Assignees
@raulcd
Labels
Component: Python Type: enhancement
Milestone
18.0.0

Comments

@asfimport
Copy link
Collaborator

asfimport commented Jun 1, 2020 •
edited
Loading

Since in the relatively near future, one will be able to do non-trivial analytical operations and query processing natively on Arrow data structures through pyarrow, it does not make sense to require users to always install NumPy when they install pyarrow. I propose to split the NumPy-depending parts of libarrow_python into a libarrow_numpy (which also must be bundled) and moving this part of the codebase into a separate Cython module.

This refactoring should be relatively painless though there may be a number of packaging details to chase up since this would introduce a new shared library to be installed in various packaging targets.

Reporter: Wes McKinney / @wesm

Related issues:

Note: This issue was originally created as ARROW-8998. Please see the migration documentation for further details.
@asfimport asfimport mentioned this issue Jan 11, 2023
Open
@raulcd raulcd self-assigned this May 27, 2024
@raulcd
Copy link
Member

raulcd commented May 27, 2024

I am currently investigating this. I also found this ML thread from 2021: https://lists.apache.org/thread/gwrhwv4p55ccrgrqontpodkwzgzr88w1

@github-actions github-actions bot mentioned this issue May 31, 2024
Merged
@raulcd raulcd mentioned this issue Aug 27, 2024
Closed
pitrou added a commit that referenced this issue Sep 2, 2024
@raulcd @jorisvandenbossche @pitrou
GH-25118: [Python] Make NumPy an optional runtime dependency (#41904)
9ab9532 
### Rationale for this change

Being able to run pyarrow without requiring numpy.

### What changes are included in this PR?

If numpy is not present we are able to import pyarrow and run functionality.
A new CI job has been created to run some basic tests without numpy.

### Are these changes tested?

Yes via CI.

### Are there any user-facing changes?

Yes, NumPy can be removed from the user installation and pyarrow functionality still works

* GitHub Issue: #25118

Lead-authored-by: Raúl Cumplido <[email protected]>
Co-authored-by: Joris Van den Bossche <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
@pitrou pitrou added this to the 18.0.0 milestone Sep 2, 2024
@pitrou
Copy link
Member

pitrou commented Sep 2, 2024

Issue resolved by pull request 41904
#41904

@pitrou pitrou closed this as completed Sep 2, 2024
mapleFU pushed a commit to mapleFU/arrow that referenced this issue Sep 3, 2024
@raulcd @jorisvandenbossche
@pitrou @mapleFU
apacheGH-25118: [Python] Make NumPy an optional runtime dependency (a…
2709e05 
…pache#41904)

### Rationale for this change

Being able to run pyarrow without requiring numpy.

### What changes are included in this PR?

If numpy is not present we are able to import pyarrow and run functionality.
A new CI job has been created to run some basic tests without numpy.

### Are these changes tested?

Yes via CI.

### Are there any user-facing changes?

Yes, NumPy can be removed from the user installation and pyarrow functionality still works

* GitHub Issue: apache#25118

Lead-authored-by: Raúl Cumplido <[email protected]>
Co-authored-by: Joris Van den Bossche <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this issue Sep 6, 2024
@raulcd @jorisvandenbossche
@pitrou @zanmato1984
apacheGH-25118: [Python] Make NumPy an optional runtime dependency (a…
89ac0ed 
…pache#41904)

### Rationale for this change

Being able to run pyarrow without requiring numpy.

### What changes are included in this PR?

If numpy is not present we are able to import pyarrow and run functionality.
A new CI job has been created to run some basic tests without numpy.

### Are these changes tested?

Yes via CI.

### Are there any user-facing changes?

Yes, NumPy can be removed from the user installation and pyarrow functionality still works

* GitHub Issue: apache#25118

Lead-authored-by: Raúl Cumplido <[email protected]>
Co-authored-by: Joris Van den Bossche <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
@jameswu1991 jameswu1991 mentioned this issue Sep 13, 2024
Open
khwilson pushed a commit to khwilson/arrow that referenced this issue Sep 14, 2024
@raulcd @jorisvandenbossche
@pitrou @khwilson
apacheGH-25118: [Python] Make NumPy an optional runtime dependency (a…
732b104 
…pache#41904)

### Rationale for this change

Being able to run pyarrow without requiring numpy.

### What changes are included in this PR?

If numpy is not present we are able to import pyarrow and run functionality.
A new CI job has been created to run some basic tests without numpy.

### Are these changes tested?

Yes via CI.

### Are there any user-facing changes?

Yes, NumPy can be removed from the user installation and pyarrow functionality still works

* GitHub Issue: apache#25118

Lead-authored-by: Raúl Cumplido <[email protected]>
Co-authored-by: Joris Van den Bossche <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
@raulcd raulcd mentioned this issue Sep 17, 2024
Merged
raulcd added a commit that referenced this issue Oct 10, 2024
@raulcd
GH-43846: [Python][Packaging] Remove numpy dependency from pyarrow pa…
8be5f9c 
…ckaging (#44148)

## WIP

Creating the PR to validate if / what are the CI failures 

### Rationale for this change
Once the issue has been merged:
- #25118

We do not require numpy as a pyarrow dependency.

### What changes are included in this PR?

Remove numpy as a required dependency

### Are these changes tested?

Via CI

### Are there any user-facing changes?

Yes, pyarrow won't install numpy as a required dependency.
* GitHub Issue: #43846

Authored-by: Raúl Cumplido <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
raulcd added a commit that referenced this issue Oct 10, 2024
@raulcd
GH-43846: [Python][Packaging] Remove numpy dependency from pyarrow pa…
e15bbe6 
…ckaging (#44148)

## WIP

Creating the PR to validate if / what are the CI failures 

### Rationale for this change
Once the issue has been merged:
- #25118

We do not require numpy as a pyarrow dependency.

### What changes are included in this PR?

Remove numpy as a required dependency

### Are these changes tested?

Via CI

### Are there any user-facing changes?

Yes, pyarrow won't install numpy as a required dependency.
* GitHub Issue: #43846

Authored-by: Raúl Cumplido <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
@AdrianoKF AdrianoKF mentioned this issue Oct 29, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@raulcd raulcd

Labels
Component: Python Type: enhancement
Projects
None yet
Milestone
18.0.0
Development

No branches or pull requests
3 participants
@raulcd @pitrou @asfimport