Fix CI run on ubuntu-latest-py3.12

[glatterf42]

At the moment, this PR mainly runs the CI checks to debug the mysterious error message observed even after multiple re-runs and cache deletions on ubuntu-latest-py3.12:

 Error in IRkernel::installspec() : 
  jupyter-client has to be installed but “jupyter kernelspec --version” exited with code 1.

This error is not flaky and does not seem to originate from outdated cached tool versions (at least after removing the ubuntu-latest-R-4.4.1 cache, the error persisted). It might be related to ubuntu-latest slowly changing to 24.04 over the course of this month, but the ...-py3.11 test uses the exact same runner image and workflow file without problems.

How to review

Required: describe specific things that reviewer(s) must do, in order to ensure that the PR achieves its goal.
If no review is required, write “No review: …” and describe why.

PR checklist

• Continuous integration checks all ✅
• Add or expand tests; coverage checks both ✅
• Add, expand, or update documentation.
• Update release notes.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.8%. Comparing base (c49a115) to head (8bebe09).

❗ Current head 8bebe09 differs from pull request most recent head d156586

Please upload reports for the commit d156586 to get more accurate results.

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #547     +/-   ##
=======================================
- Coverage   98.9%   98.8%   -0.2%     
=======================================
  Files         44      44             
  Lines       4819    4819             
=======================================
- Hits        4770    4765      -5     
- Misses        49      54      +5     

see 2 files with indirect coverage changes

[khaeru]

I am not sure why this should be different suddenly for one particular job, but one thing to consider is that our pip install … step happens within the GitHub Actions runner work directory:

https://github.com/iiasa/ixmp/blob/c49a1152305b4aedd349048d2c6d4f1dcbb4e82b/.github/workflows/pytest.yaml#L104-L107

In the subsequent step:

ixmp/.github/workflows/pytest.yaml

Lines 119 to 133 in c49a115

	- name: Install R dependencies and tutorial requirements
	run: |
	install.packages(c("remotes", "Rcpp"))
	remotes::install_cran(
	c("IRkernel", "reticulate"),
	dependencies = TRUE,
	# force = TRUE,
	)
	IRkernel::installspec()
	# commented: for debugging
	# print(reticulate::py_config())
	# reticulate::py_run_string("import os; print(os.environ)")
	shell: Rscript {0}

• The GHA runner invokes RScript.
• Rscript runs that script content, including IRkernel::installspec() (or your added debug calls).
• Those invoke certain Python executables.
• These will need to determine a Python /site-packages/ from which to perform imports.

It's possible that the overall PYTHONPATH is not making it through this call stack, and thus while jupyter-kernelspec runs, it doesn't know where to get its imports.

A second thing to look into is that the install of reticulate appears to be failing. I don't believe this should cause the other error, but it's a possibility:

gcc -I"/opt/R/4.4.1/lib/R/include" -DNDEBUG   -I/usr/local/include    `libpng-config --cflags` -fpic  -g -O2  -c dummy.c -o dummy.o
/bin/bash: line 1: libpng-config: command not found
gcc -I"/opt/R/4.4.1/lib/R/include" -DNDEBUG   -I/usr/local/include    `libpng-config --cflags` -fpic  -g -O2  -c read.c -o read.o
/bin/bash: line 1: libpng-config: command not found
read.c:3:10: fatal error: png.h: No such file or directory
    3 | #include <png.h>
      |          ^~~~~~~
compilation terminated.
make: *** [/opt/R/4.4.1/lib/R/etc/Makeconf:195: read.o] Error 1
ERROR: compilation failed for package ‘png’
* removing ‘/home/runner/work/_temp/Library/png’
ERROR: dependency ‘png’ is not available for package ‘reticulate’
* removing ‘/home/runner/work/_temp/Library/reticulate’

The downloaded source packages are in
	‘/tmp/Rtmpl4Ujah/downloaded_packages’
Warning messages:
1: In i.p(...) : installation of package ‘png’ had non-zero exit status
2: In i.p(...) :
  installation of package ‘reticulate’ had non-zero exit status

[glatterf42]

It's not about reticulate, because that fails even on the successful ubuntu-latest-py3.11 runs. That also seems odd, I agree, especially because I thought that was needed to run the R tutorials or so, but the test suite at large doesn't seem to mind.

[glatterf42]

It's not specifically about PYTHONPATH, either. That variable is an empty string both within Rscript as well as outside, both on Python 3.11 and 3.12.
However, the new debug output shows that within Rscript, even calling pip list doesn't work because no module named pip can be found.

[khaeru]

It may help to try something like python -c "import sys; print(sys.executable, sys.path)". PYTHONPATH is only used to initialize sys.path, which also gets (or may fail to get) entries from other locations.

[glatterf42]

Indeed: '/opt/hostedtoolcache/Python/3.12.7/x64/lib/python3.12/site-packages' is present (printed from your suggested query) outside of Rscript, but missing inside of it.

[glatterf42]

GitHub rolled back their shift from Ubuntu 22.04 to 24.04 for ubuntu-latest, so this PR is no longer needed -- for now. However, since I don't trust the eventual migration will go more smoothly, I'm going to keep the branch around.