Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Updates to CI build-test-release pipeline #27

Closed
wants to merge 60 commits into from

Conversation

a-larkins99
Copy link
Collaborator

@a-larkins99 a-larkins99 commented Jan 8, 2024

In progress pr for Jenkinsfile changes.

Jenkinsfile changes:

  • Update to use PACE-shared-lib for setting of github status following pull request;
  • Parameterize the pipeline to allow adjusting of python and Matlab version when manually running;
  • Update to use rocky8 and icdpacewin;
  • Update to use conda environments throughout to keep consistent between Windows and Linux;
  • Remove usage of podman due to this no longer working;
  • Introduce separate build installer stage;
  • Remove unused external PowerShell and bash scripts;

Github actions workflow changes:

  • upload_pypi.yml updated to remove unnecessary stage as release.py is able to upload to pypi.

Misc file changes:

  • Add MCR_license.txt
  • Update 'ApplicationName' of installer in make_package.m to result in a better named installer dir (Pace_Python_Installerinstaller -> pace_python_installer);
  • Adjust pace_neutrons\__init__.py to account for varying dir structure when determining matlab version from ctf file. This changes depending on if done during use or during release stage of Jenkins pipeline. Paths are updated to account for this where used, including in release.py;
  • Regex in release.py changed to properly detect description from changelog when the tag includes letters e.g. 0.3.0a0.

Notes:

@a-larkins99 a-larkins99 self-assigned this Jan 8, 2024
@a-larkins99 a-larkins99 force-pushed the jenkins_test branch 2 times, most recently from 0d587ae to 0c9cb69 Compare January 11, 2024 10:31
@a-larkins99 a-larkins99 force-pushed the jenkins_test branch 2 times, most recently from f26cb15 to b680d2b Compare February 22, 2024 13:55
a-larkins99 and others added 9 commits February 22, 2024 15:14
- Add load of required modules

- Add conda env creation

- Fix incorrect comment syntax

- Correct dir of .whl file during archiveArtifacts on Linux
Include conda in Jenkinsfile test step

Add MATLAB availability to test stage

Remove environment setup from test stage
Adjust powershell conda env activation commands

Adjust powershell commands for building and testing
Add verbose output to test stage
- Remove manual exit()

- Add timeout to linux testing

- Remove ||true from linux testing

- Change to signal 15 on timeout

- Try gcc module in testing

- Add numpy manually
- Explicitly set numpy version to install

- Change from conda to pip install for linux test stage

- Fix pip install options
- Attempt fix of env naming

- Add script block

- Move script into pipeline

- Add script to an environment block

- Create fn for conda env naming

- Move environment block to stages

- Fix variable access syntax
- Adjust path to be fullpath and general pathlib improvement

- Account for varying location of __init__.py

- Fix path to be relative to file
- Add typing module import

- Shorten line lengths
- Remove temp. debug stage

- Fix path syntax when setting MATLAB_REG

- Update stages to run twine as part of release.py

- Fix bug with path when assigning MATLAB_REG

- Apply path fix to MATLAB_EXE assignment

- Fix issue newline syntax

- Adjust Get-ItemProperty syntax

- Add file check to installer stage

- Update installer dir

- Make powershell file testing throw error when file not found

- Fix file check for windows

- Update file checks for installer

- Adjust linux commands to bring inline with powershell

- Remove unused ps1 and sh scripts
@a-larkins99 a-larkins99 changed the title [WIP] Jenkins test [WIP] Updates to CI build-test-release pipeline Feb 26, 2024
@a-larkins99 a-larkins99 added enhancement New feature or request and removed enhancement New feature or request labels Feb 26, 2024
a-larkins99 and others added 7 commits February 27, 2024 15:46
- Update version list creation to use list comprehension

- Update mlPath assignment to use walrus

- Remove unecessary list creation during RuntimeError missing version reporting

- Remove unecessary \ at line end and unused import

- Update list comprehension

- Update subprocess.run command to use text mode

- Add whitespace

- Disable Brille test

- Update regex search
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants