Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hopefully fixes builds on 3.12. #70

Merged
merged 56 commits into from
Nov 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
352fe5f
Hopefully fixes builds on 3.12.
enzbus Oct 7, 2023
752a8d6
Update Makefile
enzbus Oct 7, 2023
7225748
Update Makefile
enzbus Oct 7, 2023
ae09b24
trying to switch to meson
enzbus Oct 8, 2023
a8dc9ee
Merge branch 'master' of github.com:enzbus/scs-python
enzbus Oct 8, 2023
73946fb
trying to get a meson build
enzbus Oct 8, 2023
951ce78
Makefile superflous
enzbus Oct 8, 2023
a7d0868
moved vendored scs folder to avoid naming conflict
enzbus Oct 8, 2023
5895b15
basic build working
enzbus Oct 8, 2023
6672832
@rgommers comments, macros, and trying blas link
enzbus Oct 8, 2023
b5a3641
added indirect
enzbus Oct 8, 2023
e88be4f
indirect compiles (was missing flags)
enzbus Oct 8, 2023
c2f36be
pip installation works, including __init__.py
enzbus Oct 8, 2023
ae00033
pip installatin works, only blas/lapack linking missing (probably typ…
enzbus Oct 8, 2023
d29c051
Error out with a clear message if the SCS git submodule is missing
rgommers Oct 8, 2023
7601bc6
Remove -O3; that's the default in meson-python, and isn't portable.
rgommers Oct 8, 2023
3fdab57
Merge pull request #1 from rgommers/meson-fixes
enzbus Oct 8, 2023
321755c
Silence -Wunused-result warnings
rgommers Oct 8, 2023
16f1b02
blas linking works, thanks @rgommers
enzbus Oct 8, 2023
fc83dcc
Merge branch 'meson-fixes' into master
enzbus Oct 8, 2023
97f26c9
Use Accelerate as BLAS library on macOS
rgommers Oct 8, 2023
85f1079
Merge pull request #2 from rgommers/use-accelerate
enzbus Oct 8, 2023
5ab81ab
fixed OLD_setup.py to match new directories name
enzbus Oct 8, 2023
c2685c4
disabled 3.12 on github yamml jobs that are not recoverable
enzbus Oct 8, 2023
fc28677
edited github yamml with OLD_setup.py and new build
enzbus Oct 8, 2023
88264c3
black gh checker on oldsetup.py
Oct 25, 2023
a7690a1
added pyproject.toml target to bumpversion
Oct 25, 2023
f90a6a3
added ddlong flags at @bodono request
Oct 25, 2023
6cec24a
moved from https://github.com/rickstaa/action-black to official psf/b…
Oct 25, 2023
d7099ff
fixing github yaml fails
Oct 25, 2023
3fd0561
last failure of stupid black I want to see
Oct 25, 2023
fb8c217
was missing conda install on 3.12 builds
Oct 25, 2023
f07a83c
made non-darwin blas linked statically so linux wheels dont require b…
enzbus Nov 1, 2023
33b3145
made static linkage for non-darwin a meson option
enzbus Nov 2, 2023
577cfa2
added blas link build opts to github yaml
enzbus Nov 2, 2023
82644d1
debugging blas linking on windows via github ci
enzbus Nov 2, 2023
fb5dcc8
spacing on github yaml
enzbus Nov 2, 2023
41481de
trying to get windows build
Nov 2, 2023
5c3d16c
numpy.show_config changed api
enzbus Nov 2, 2023
72140e5
typo
enzbus Nov 2, 2023
580477d
one more try
enzbus Nov 2, 2023
ef655a2
miniconda lib location for windows on github ci
enzbus Nov 2, 2023
ee9017b
typo
enzbus Nov 2, 2023
538b995
maybe inverted slash?
enzbus Nov 2, 2023
a859327
maybe inverted slash?
enzbus Nov 2, 2023
e59e496
with same mixed spelling returned by np.show_config
enzbus Nov 2, 2023
bcacd76
some debugging information
enzbus Nov 2, 2023
f23e144
it was openblas.lib!
enzbus Nov 2, 2023
6d70a6b
fixed @rgommers comments on meson.build
enzbus Nov 3, 2023
4b081e6
switched from macos-10.15 to macos-latest in github ci
enzbus Nov 3, 2023
b893196
renamed OLD_setup.py to legacy_setup.py
enzbus Nov 3, 2023
12d45d4
if clause wasnt right
enzbus Nov 3, 2023
f7e2e98
removed bunch of conda installs that should not be needed (revert if …
enzbus Nov 3, 2023
a0fb593
typo
enzbus Nov 3, 2023
6036646
we needed numpy
enzbus Nov 3, 2023
9b7bbc1
problems with conda env and pip, better to restore same conda install…
enzbus Nov 3, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
[bumpversion]
current_version = 3.2.3

[bumpversion:file:setup.py]
[bumpversion:file:legacy_setup.py]
[bumpversion:file:pyproject.toml]

41 changes: 22 additions & 19 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,6 @@ jobs:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
if: "!startsWith(github.ref, 'refs/tags/') && github.ref != 'refs/heads/master'"

lint:
name: black formatter
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: rickstaa/action-black@v1
with:
black_args: ". --line-length=80 --check"

build_mkl:
runs-on: ${{ matrix.os }}
defaults:
Expand All @@ -34,7 +25,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ ubuntu-latest, macos-10.15 ] # windows-latest not finding MKL
os: [ ubuntu-latest, macos-latest ] # windows-latest not finding MKL
python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11" ]

env:
Expand All @@ -54,7 +45,7 @@ jobs:
conda install mkl scipy numpy pytest
- name: Test
run: |
python setup.py install --scs --mkl
python legacy_setup.py install --scs --mkl
pytest
rm -rf build/

Expand Down Expand Up @@ -86,7 +77,7 @@ jobs:
conda install scipy numpy pytest
- name: Test
run: |
python setup.py install --scs --openmp
python legacy_setup.py install --scs --openmp
pytest
rm -rf build/

Expand All @@ -98,8 +89,8 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ ubuntu-latest, macos-10.15, windows-latest ]
python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11" ]
os: [ ubuntu-latest, macos-latest, windows-latest ]
python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11", "3.12"]

env:
PYTHON_VERSION: ${{ matrix.python-version }}
Expand All @@ -125,18 +116,29 @@ jobs:
conda install scipy=1.7 numpy=1.21 pytest mkl openblas lapack
elif [[ "$PYTHON_VERSION" == "3.11" ]]; then
conda install scipy=1.9.3 numpy=1.23.4 pytest mkl openblas lapack
elif [[ "$PYTHON_VERSION" == "3.12" ]]; then
conda install scipy numpy pytest mkl openblas lapack
fi
- name: Build
run: |
if [[ "$PYTHON_VERSION" == "3.7" ]] || [[ "$PYTHON_VERSION" == "3.8" ]]; then
python legacy_setup.py install
elif [[ "$PYTHON_VERSION" != "3.7" ]] && [[ "$PYTHON_VERSION" != "3.8" ]]; then
python -c "import numpy as np; print('NUMPY BLAS INFOS'); print(np.show_config())"
python -m pip install --verbose .
fi
enzbus marked this conversation as resolved.
Show resolved Hide resolved
- name: Test
run: |
python setup.py install
pytest
rm -rf build/
- name: Build and test windows wheels
if: ${{env.DEPLOY == 'True' && startsWith(matrix.os, 'windows')}}
run: |
python setup.py bdist_wheel
python -m pip install build
python -m build -Csetup-args="-Dlink_blas_statically=True"
python -m pip install delvewheel
delvewheel repair dist/*whl
conda remove openblas # to check static linkage
pip install wheelhouse/*whl --force-reinstall
pytest
- name: Upload artifacts to github
Expand All @@ -154,8 +156,8 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ ubuntu-latest, macos-10.15 ]
python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11" ]
os: [ ubuntu-latest, macos-latest ]
python-version: [ 3.7, 3.8, 3.9, "3.10", "3.11", "3.12"]

env:
RUNNER_OS: ${{ matrix.os }}
Expand Down Expand Up @@ -196,7 +198,8 @@ jobs:
- name: Build source
if: ${{env.DEPLOY == 'True' && env.SINGLE_ACTION_CONFIG == 'True'}}
run: |
python setup.py sdist --dist-dir=wheelhouse
python -m pip install build
python -m build -Csetup-args="-Dlink_blas_statically=True" --outdir=wheelhouse

- name: Upload artifacts to github
if: ${{env.DEPLOY == 'True'}}
Expand Down
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[submodule "scs"]
path = scs
path = scs_source
url = https://github.com/cvxgrp/scs.git
44 changes: 24 additions & 20 deletions setup.py → legacy_setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -176,8 +176,12 @@ def install_scs(**kwargs):
extra_compile_args = ["-O3"]
extra_link_args = []
libraries = []
sources = ["src/scspy.c"] + glob("scs/src/*.c") + glob("scs/linsys/*.c")
include_dirs = ["scs/include", "scs/linsys"]
sources = (
["scs/scspy.c"]
+ glob("scs_source/src/*.c")
+ glob("scs_source/linsys/*.c")
)
include_dirs = ["scs_source/include", "scs_source/linsys"]
define_macros = [("PYTHON", None), ("CTRLC", 1)]
if args.openmp:
extra_compile_args += ["-fopenmp"]
Expand All @@ -198,16 +202,16 @@ def install_scs(**kwargs):
_scs_direct = Extension(
name="_scs_direct",
sources=sources
+ glob("scs/linsys/cpu/direct/*.c")
+ glob("scs/linsys/external/amd/*.c")
+ glob("scs/linsys/external/qdldl/*.c"),
depends=glob("src/*.h"),
+ glob("scs_source/linsys/cpu/direct/*.c")
+ glob("scs_source/linsys/external/amd/*.c")
+ glob("scs_source/linsys/external/qdldl/*.c"),
depends=glob("scs/*.h"),
define_macros=list(define_macros),
include_dirs=include_dirs
+ [
"scs/linsys/cpu/direct/",
"scs/linsys/external/amd",
"scs/linsys/external/dqlql",
"scs_source/linsys/cpu/direct/",
"scs_source/linsys/external/amd",
"scs_source/linsys/external/dqlql",
],
libraries=list(libraries),
extra_compile_args=list(extra_compile_args),
Expand All @@ -216,11 +220,11 @@ def install_scs(**kwargs):

_scs_indirect = Extension(
name="_scs_indirect",
sources=sources + glob("scs/linsys/cpu/indirect/*.c"),
depends=glob("src/*.h"),
sources=sources + glob("scs_source/linsys/cpu/indirect/*.c"),
depends=glob("scs/*.h"),
define_macros=list(define_macros)
+ [("PY_INDIRECT", None), ("INDIRECT", 1)],
include_dirs=include_dirs + ["scs/linsys/cpu/indirect/"],
include_dirs=include_dirs + ["scs_source/linsys/cpu/indirect/"],
libraries=list(libraries),
extra_compile_args=list(extra_compile_args),
extra_link_args=list(extra_link_args),
Expand All @@ -241,13 +245,13 @@ def install_scs(**kwargs):
_scs_gpu = Extension(
name="_scs_gpu",
sources=sources
+ glob("scs/linsys/gpu/*.c")
+ glob("scs/linsys/gpu/indirect/*.c"),
depends=glob("src/*.h"),
+ glob("scs_source/linsys/gpu/*.c")
+ glob("scs_source/linsys/gpu/indirect/*.c"),
depends=glob("scs/*.h"),
define_macros=list(define_macros)
+ [("PY_GPU", None), ("INDIRECT", 1)],
include_dirs=include_dirs
+ ["scs/linsys/gpu/", "scs/linsys/gpu/indirect"],
+ ["scs_source/linsys/gpu/", "scs_source/linsys/gpu/indirect"],
library_dirs=library_dirs,
libraries=libraries + ["cudart", "cublas", "cusparse"],
extra_compile_args=list(extra_compile_args),
Expand All @@ -272,10 +276,10 @@ def install_scs(**kwargs):
# MKL should be included in the libraries already:
_scs_mkl = Extension(
name="_scs_mkl",
sources=sources + glob("scs/linsys/mkl/direct/*.c"),
depends=glob("src/*.h"),
sources=sources + glob("scs_source/linsys/mkl/direct/*.c"),
depends=glob("scs/*.h"),
define_macros=list(define_macros) + [("PY_MKL", None)],
include_dirs=include_dirs + ["scs/linsys/mkl/direct/"],
include_dirs=include_dirs + ["scs_source/linsys/mkl/direct/"],
libraries=list(libraries),
extra_compile_args=list(extra_compile_args),
extra_link_args=list(extra_link_args),
Expand All @@ -289,7 +293,7 @@ def install_scs(**kwargs):
author_email="[email protected]",
url="http://github.com/cvxgrp/scs",
description="scs: splitting conic solver",
package_dir={"scs": "src"},
package_dir={"scs": "scs"},
packages=["scs"],
ext_modules=ext_modules,
cmdclass={"build_ext": build_ext_scs},
Expand Down
143 changes: 143 additions & 0 deletions meson.build
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
project('scs', 'c')

py = import('python').find_installation(pure: false)

cc = meson.get_compiler('c')
blas_deps = []
if host_machine.system() == 'darwin'
blas_deps = [dependency('Accelerate')]
else
blas_deps = [cc.find_library('openblas', static: get_option('link_blas_statically'), required : false)]
endif

# try to find blas/cblas (e.g., Linux)
if not blas_deps[0].found()
blas_deps = [cc.find_library('blas', static: get_option('link_blas_statically'), required : false)]
cblas_dep = cc.find_library('cblas', static: get_option('link_blas_statically'), required : false)
if cblas_dep.found()
blas_deps += cblas_dep
endif
endif

# We find anaconda blas on windows for github CI
if host_machine.system() == 'windows' and not blas_deps[0].found()
message(run_command(py,['-c','''import os; print('CONDA LIB FOLDER'); print(os.listdir('C:/Miniconda/envs/test\\Library\\lib'))''']).stdout())
blas_deps = [cc.find_library('openblas',
dirs: ['C:/Miniconda/envs/test\\Library\\lib'], # this is the location on github CI
static: get_option('link_blas_statically'),
required : false)]
endif

# Since nothing above was required, we stop here if failed
if not blas_deps[0].found()
error('OpenBLAS or Netlib BLAS/CBLAS is required on all platforms except Windows, and was not found.')
endif

fs = import('fs')
if not fs.exists('scs_source/README.md')
error('Missing the `scs_source` submodule! Run `git submodule update --init` to fix this.')
endif

incdir_numpy = run_command(py,['-c',
'''import os
import numpy as np
try:
incdir = os.path.relpath(np.get_include())
except Exception:
incdir = np.get_include()
print(incdir)
'''], check: true).stdout().strip()

# rw.c emits a lot of -Wunused-result warnings, silence them for now:
c_args = cc.get_supported_arguments('-Wno-unused-result')


py.extension_module(
'_scs_direct',

'scs/scspy.c',
'scs_source/linsys/cpu/direct/private.c',

# scs_source/src:
'scs_source/src/aa.c',
'scs_source/src/cones.c',
'scs_source/src/ctrlc.c',
'scs_source/src/exp_cone.c',
'scs_source/src/linalg.c',
'scs_source/src/normalize.c',
'scs_source/src/rw.c',
'scs_source/src/scs_version.c',
'scs_source/src/scs.c',
'scs_source/src/util.c',

# scs_source/linsys:
'scs_source/linsys/scs_matrix.c',
'scs_source/linsys/csparse.c',

# scs_source/linsys/external/qdldl:
'scs_source/linsys/external/qdldl/qdldl.c',

# scs_source/linsys/external/amd:
'scs_source/linsys/external/amd/amd_1.c',
'scs_source/linsys/external/amd/amd_2.c',
'scs_source/linsys/external/amd/amd_aat.c',
'scs_source/linsys/external/amd/amd_control.c',
'scs_source/linsys/external/amd/amd_defaults.c',
'scs_source/linsys/external/amd/amd_dump.c',
'scs_source/linsys/external/amd/amd_global.c',
'scs_source/linsys/external/amd/amd_info.c',
'scs_source/linsys/external/amd/amd_order.c',
'scs_source/linsys/external/amd/amd_post_tree.c',
'scs_source/linsys/external/amd/amd_postorder.c',
'scs_source/linsys/external/amd/amd_preprocess.c',
'scs_source/linsys/external/amd/amd_valid.c',
'scs_source/linsys/external/amd/SuiteSparse_config.c',

include_directories : [
'scs',
'scs_source/include',
'scs_source/linsys',
'scs_source/linsys/cpu/direct',
'scs_source/linsys/external/qdldl',
'scs_source/linsys/external/amd',
incdir_numpy],
install: true,
c_args: c_args + ['-DPYTHON', '-DCTRLC=1', '-DUSE_LAPACK=1', '-DDLONG=1'],
dependencies: blas_deps,
)

py.extension_module(
'_scs_indirect',

'scs/scspy.c',
'scs_source/linsys/cpu/indirect/private.c',

# scs_source/src:
'scs_source/src/aa.c',
'scs_source/src/cones.c',
'scs_source/src/ctrlc.c',
'scs_source/src/exp_cone.c',
'scs_source/src/linalg.c',
'scs_source/src/normalize.c',
'scs_source/src/rw.c',
'scs_source/src/scs_version.c',
'scs_source/src/scs.c',
'scs_source/src/util.c',

# scs_source/linsys:
'scs_source/linsys/scs_matrix.c',
'scs_source/linsys/csparse.c',

include_directories : [
'scs',
'scs_source/include',
'scs_source/linsys',
'scs_source/linsys/cpu/indirect',
incdir_numpy],
install: true,
c_args: c_args + ['-DPYTHON', '-DCTRLC=1', '-DPY_INDIRECT', '-DINDIRECT=1',
'-DUSE_LAPACK=1', '-DDLONG=1'],
dependencies: blas_deps,
)

py.install_sources('scs/__init__.py', subdir: 'scs')
5 changes: 5 additions & 0 deletions meson.options
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# we use this to build pre-compiled wheels on github CI
# copying anaconda openblas into SCS wheels for ease of
# installation
option('link_blas_statically', type: 'boolean',
value: false, description: 'copy BLAS compiled object into SCS module(s)')
22 changes: 16 additions & 6 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,17 @@
[build-system]
requires = [
"setuptools<65.6.0",
"wheel",
"oldest-supported-numpy",
]
build-backend = "setuptools.build_meta"
build-backend = 'mesonpy'
requires = ['meson-python', 'numpy']

[project]
name = 'scs'
version = "3.2.3"
enzbus marked this conversation as resolved.
Show resolved Hide resolved
description = 'Splitting conic solver'
readme = 'README.md'
requires-python = '>=3.8'
license = {file = 'LICENSE'}
authors = [
{name = "Brendan O'Donoghue", email = "[email protected]"}]
dependencies = [
'numpy',
'scipy',
]
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Loading