Publish to conda

[benoit-cty]

• Technical improvement.
• Details:
  • Publish to conda in CI

---

These changes change non-functional parts of this repository : CI.

[benjello]

No objection but not competent enough to review

[sandcha]

Thank you @benoit-cty 🙌
The requested changes are mainly typos and a question on an error raised by local testing (but that might not arise on CI).

[sandcha]

To test the procedure (but stop before publishing ^^) and seeing that some default value for PYPI_VERSION, PYPI_URL and PYPI_SHA256 already existed in meta.yaml file, I ran the build command.

Running this on macOS with miniconda produced the following error 🤔 :
Error: HTTP 404 CONNECTION FAILED for url <file:///Users/.../country-template/.conda/PYPI_URL>

@benoit-cty Do you see why this might happen and, more precisely if it means that some step is missing from the documentation or if it's a local issue?
As the process was already tested on openfisca-core and openfisca-france, my guess is that's it's a local issue and that we will be able to check this on the next CI publish-to-conda job execution.

📃 Full output

Knowing that miniconda was installed with HomeBrew that comes with Caskroom directory.

$ conda build .conda
No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.16
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.16

Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults

Attempting to finalize metadata for openfisca-country-template
INFO:conda_build.metadata:Attempting to finalize metadata for openfisca-country-template
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/osx-64/repodata.json HTTP/1.1" 304 0
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/noarch/repodata.json HTTP/1.1" 304 0
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/osx-64/repodata.json HTTP/1.1" 304 0
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/noarch/repodata.json HTTP/1.1" 304 0
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/channeldata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/channeldata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/channeldata.json HTTP/1.1" 200 None
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done


Attempting to finalize metadata for openfisca-country-template-dev
INFO:conda_build.metadata:Attempting to finalize metadata for openfisca-country-template-dev
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
BUILD START: ['openfisca-country-template-PYPI_VERSION-py_0.tar.bz2', 'openfisca-country-template-dev-PYPI_VERSION-py_0.tar.bz2']
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

 environment location: /usr/local/Caskroom/miniconda/base/conda-bld/openfisca-country-template_1652367881762/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl


The following NEW packages will be INSTALLED:

    bzip2:           1.0.8-h0d85af4_4          conda-forge
    ca-certificates: 2021.10.8-h033912b_0      conda-forge
    libffi:          3.4.2-h0d85af4_5          conda-forge
    libzlib:         1.2.11-h6c3fc93_1014      conda-forge
    ncurses:         6.3-h96cf925_1            conda-forge
    openssl:         3.0.3-hfe4f2af_0          conda-forge
    pip:             22.1-pyhd8ed1ab_0         conda-forge
    python:          3.9.12-h1cc4136_1_cpython conda-forge
    python_abi:      3.9-2_cp39                conda-forge
    readline:        8.1-h05e3726_0            conda-forge
    setuptools:      62.2.0-py39h6e9494a_0     conda-forge
    sqlite:          3.38.5-hd9f0692_0         conda-forge
    tk:              8.6.12-h5dbffcc_0         conda-forge
    tzdata:          2022a-h191b570_0          conda-forge
    wheel:           0.37.1-pyhd8ed1ab_0       conda-forge
    xz:              5.2.5-haf1e3a3_1          conda-forge
    zlib:            1.2.11-h6c3fc93_1014      conda-forge

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done
Source cache directory is: /usr/local/Caskroom/miniconda/base/conda-bld/src_cache
INFO:conda_build.source:Source cache directory is: /usr/local/Caskroom/miniconda/base/conda-bld/src_cache
Downloading source to cache: PYPI_URL
INFO:conda_build.source:Downloading source to cache: PYPI_URL
Downloading file:///Users/.../country-template/.conda/PYPI_URL
INFO:conda_build.source:Downloading file:///Users/.../country-template/.conda/PYPI_URL
Error: HTTP 404 CONNECTION FAILED for url <file:///Users/.../country-template/.conda/PYPI_URL>
Elapsed: 00:00.000115

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
WARNING:conda_build.source:Error: HTTP 404 CONNECTION FAILED for url <file:///Users/sch/dev/gitruc/country-template/.conda/PYPI_URL>
Elapsed: 00:00.000115

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
Could not download file:///Users/.../country-template/.conda/PYPI_URL

[sandcha]

Add python3 .github/get_pypi_info.py -p OpenFisca-Country-Template to the documentation as a first step to test the building process (conda build .conda without publishing)?

Re-tested: Needs requests module.

[benjello]

@MattiSG @sandcha @benoit-cty : where are we on this PR ?

[sandcha]

Applying rebase on this old PR, with @benoit-cty agreement. 🙂

[bonjourmauko]

I'm curious to see how this works for testing, I'm having troubles in #157

[benoit-cty]

I'm curious to see how this works for testing, I'm having troubles in #157

Before a rebase the build was working, after a rebase (badly made I'm afraid seeing the changes now) it fail:

openfisca-core 43.2.0 has requirement numpy<2.0,>=1.24.2, but you have numpy 2.1.2.
WARNING: Tests failed for openfisca-country-template-7.1.6-py_0.tar.bz2 - moving package to /opt/conda/conda-bld/broken
TESTS FAILED: openfisca-country-template-7.1.6-py_0.tar.bz2

Maybe it's because OpenFisca-Core has no pinned dependencies of Numpy in https://github.com/openfisca/openfisca-core/blob/5a131cf39f7f6166d6dd7addaae11e631d2b6422/.conda/openfisca-core/meta.yaml#L55

But there is https://github.com/openfisca/openfisca-core/blob/5a131cf39f7f6166d6dd7addaae11e631d2b6422/.conda/openfisca-core/conda_build_config.yaml#L4 so it may works...

EDIT:

• In ci: publish to conda #157 You are using rattler-build and not conda-build, the resulting meta.yaml could be found in the package https://anaconda.org/openfisca/openfisca-core/43.2.0/download/noarch/openfisca-core-43.2.0-pyh8e68882_0.conda
  where we see:

requirements:
  host:
[...]
    - numpy 1.26.4 py310hb13e2d6_0
[...]
  run:
    - PyYAML >=6.0, <7.0
    - StrEnum >=0.4.8, <0.5.0
    - dpath >=2.1.4, <3.0
    - numexpr >=2.10.1, <3.0
    - numpy