Prepare Release 10.2.3 #442
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Update khiops-env to display error on Linux On Linux, the khiops-env script doesn't print anything in the stdout/err anymore (it is already the case on windows). It only populates environment variables if possible. In case of errors (java or mpi) it populates 2 new variables KHIOPS_MPI_ERROR and KHIOPS_JAVA_ERROR The khiops script use notify-send to diplay errors to users. It implies a new dependency in the khiops package: libnotify on rocky and libnotify-bin on debian. Errors are also logs into /var/log/syslog by using the 'logger' command.
Note: As a side effect the update-copyright.py script removed the DOS carriage returns.
150 extend the copyright script for python files (backport dev-v10)
This code was useful only for the khiops-package-version which we do not use anymore since commit a92a8ef.
100 implement tagversion coherence in ci (backport dev-v10)
Fix slice memory (backport dev-v10)
With 3 new environment variables KHIOPS_PATH, KHIOPS_COCLUSTERING_PATH, KHIOPS_ENV_PATH, the scripts khiops, khiops_coclustering and khiops-env are generic, we can adapt them to all plateforms. These variables are set in the 3 scripts by cmake at the configuration step. The khiops-core packages contains 3 more files: khiops_env, khiops and khiops-coclustering (these 2 lasts are not mandatory but it facilitates the debugging and tests) The MODL binary is used to get proc number in khiops_env On UNIXes we have khiops-env, on windows khiops_env.cmd. Renaming of khiops-env to khiops_env to uniformize names between Windows and UNIXes. Khiops_env for UNIXes - Add function get_script_dir funtion in khiops-env. in conda environment, the PREFIX variable is not always set. In order to set the binaries path, we use this new function. The khiops binaries are collocated with the khiops-env script (except on fedora-like os) NSIS - Add replaceInfile function in NSIS, used to set the KHIOPS_HOME variable during the installation. - NSIS code simplification Improve khiops_env for windows - get KHIOPS_PROC_NUMBER with command line - add a new variable GUI which is set by cmake or nsis. This variable conditions the setting of the java related variables. - add KHIOPS_MPI_ERROR, KHIOPS_JAVA_ERROR and KHIOPS_COCLUSTERING_PATH to fit to the UNIXes script - We use %~dp0 to set KHIOPS_HOME, it is the directory of the script. It is no longer necessary to set KHIOPS_HOME with NSIS (or cmake) or export it. - check mpi implementation of mpiexec Improve khiops.cmd - use %~dp0 to locate khiop_env.cmd (no need for KHIOPS_HOME anymore) - use KHIOPS_MPI_ERROR and KHIOPS_JAVA_ERROR install.cmake - produce khiops_env.cmd - minor fixes in UNIXes scripts - Improve KNI package for conda: put the .dll in the bin directory and the .lib in the lib directory (as it is for KNI.zip). - Call khiops_env -env in action test-khiops-on-iris conda.yml - use cmd shell for testing conda package on windows - force process number to test in parallel Modify khiops_env.cmd API to be the same on Windows and UNIXes - When using the '--env' flag, variables will only be displayed without changing the environment. It is useful in debugging. Improve the action test-khiops-install - test khiops-env playing with KHIOPS_PROC_NUMBER
Add a new tiny binary to show the physical core number. It is only used by khiops_env. This binary is packaged with khiops, so there is an impact on conda, deb, rpm and NSIS.
Add KHIOPS_MPI_HOST_FILE to khiops_env - Allows to specify a hostfile to mpiexec - if this variable is defined, khiops_env will export environment variables required by khiops to run correctly to MPI processes. These variables relate to GCS/S3 credentials and to khiops itself (KHIOPS_* and Khiops*) khiops_env is built on cmake configure: the variable export is done only for openmpi and the hostfile keyword is different for openmpi and mpich Remove process affinity for MS-MPI - As discussed in #350, the affinity slows down khiops. - remove the prioryty (below normal) too Improve UNIXes khiops_env - java configuration and modules loading are not in khiops_env on conda environment - To improve the install.cmake readability, 3 files are added to fill variables: - export_env_variables.sh - java_settings.sh - use_environment_module.sh fix GUI error message in khiops and khiops coclustering - The message saying to install "khiops" on conda environment is wrong. - when a non-existing option is passed, e.g. khiops --foo, the GUI message is displayed fix message in help about KHIOPS_API_MODE: false <=> true KHIOPS_API_MODE message
bug fix: when running tests without using kht_test, we have to set the variable KHIOPS_API_MODE to true
Update of the khiops_env script on Linux/Mac and Windows to be compatible with conda environments (even without CONDA_PREFIX). Conda packages are updated with khiops_env (on all OS). It has an impact on packaging (cmake, NSIS) and github actions. Add a new variable KHIOPS_MPI_HOST_FILE (in Linux only) to set up the MPI command line on multi-machines Remove process affinity on windows backport #343 from dev
This achieved by using Miniforge3 in the setup-miniconda GitHub action. Thus, only the `conda-forge` channel is used and there is no need of channel overriding.
…Hub runner Thus, Ninja / VS interaction errors are partially mitigated. See actions/runner-images#9701.
…nja on Windows This is because Ninja does not expect target platform and generator toolset. However, CMake Windows presets set these, which results in Ninja failure.
Disable "floating-point expression contraction" on clang and gcc to improve reproducibility between x86_64 and arm64. This change impacts: - Linux native builds - Conda builds for: - Linux - Mac OS: ARM and X86_64 architectures
Use only conda-forge for building the binary Khiops Conda packages
The environment modules systematically: - undloads current mpi module - loads the desired mpi module In that way we are sure to launch the appropriate mpiexec.
- Add the variable KHIOPS_MPI_VERBOSE to disable the flag "--quiet" in the mpirun command. In khiops_env, its default value is true and the mpirun is run without --quiet. if its value is different from 'true', '--quiet' is added to the command line. - On the contrary, KHIOPS_MPI_VERBOSE default value is false in the script 'khiops': we don't want to see MPI messages when running khiops in standalone. - some variables are mandatory to run MPI correctly (like OMPI_MCA_btl_vader_single_copy_mechanism), these variables are displayed when running `khiops-env --env`. In that way, all variables needed to run Khiops are avalaible for khiops' wrappers. - Add KHIOPS_MPI_VERBOSE=true to the tests on CI: test-khiops-on-iris and test-khiops-install in order to check that there is no MPI error
…rror-1-without-any-message 383 bis khiops command return error 1 without any message
Use Python 3.8-compatible string suffix removal idiom
We let CMake choose the good compiler with the variables provided by the compiler packages. By setting 'gcc' in the preset, the Conda build used the system-wide GCC
When we configure the project on fedora. A fatal error is triggered if the variable MPI_BIN is not defined. On conda, this variable is not required.
The conda version doesn't contain dashes. We handle this directly in the conda.yml with a search and replace. Rather than doing it upstream, befor calling conda build. It simplifies both local and CI builds.
From cmake version 3.4 WINDOWS_EXPORT_ALL_SYMBOLS is needed to produce lib file alongside the dll
407 backport v10
Prepare release 10.2.3
Thus, OpenMPI is instructed to allow to be run as root in a portable way. The `OMPI_ALLOW_RUN_AS_ROOT` and `OMPI_ALLOW_RUN_AS_ROOT_CONFIRM` environment variables are not taken into account by `mpiexec` on Debian 10, because they were not supported at the time Debian 10 was released.
…ebian10 429 learningtesttool fails on debian10
…ps_env` Also fix `KHIOPS_DRIVERS_PATH` environment variable display on Windows
…indows related_to #431
Thus, the instantiated `khiops_env` template is packaged into the native Windows installer. In order to make the tracing of the `khiops_env` script from template to instantiation, add `.in` extension to the Windows `khiops_env` script template. closes #431
…et_khiops_drivers_path-is-not-substituted-correctly-in-khiops_envcmd 431 on windows cmake variable set khiops drivers path is not substituted correctly in khiops envcmd
Update Khiops version to 10.2.3
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR for tagging the 10.2.3 release version.