Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Build] Update, clean-up and use job for Maven publication #2718

Merged

Conversation

HannesWell
Copy link
Member

@HannesWell HannesWell commented Jan 2, 2025

Currently the publication of the Maven artifacts of Eclipse happens in five jobs manually/directly defined in the 'Releng' Jenkins instance:

  1. CBIaggregator
  2. RepositoryValidator
  3. PublishPlatformToMaven, PublishJDTtoMaven and PublishPDEToMaven

Corresponding job-definitions for these jobs already in this SCM (except for the repository-validation), but the jobs generated from them are not used.

This change replaces the existing (unused) multiple job-definitions by a single Jenkins pipeline that contains all steps and performs the final publication for all covered projects in a single matrix. The new pipeline is updated to incorporate all the changes made in the meantime to the manually maintained publication jobs.

The I-build job is changed to use the new publication job.

With respect to the existing, manually maintained publication job, the new job is adjusted in the following aspects:

  • Rename job to just 'PublishToMaven'. The CBI part by itself is irrelevant and a tool in the publication process
  • Remove documentation about the jobs involved and scripts called from the job description as it's an implementation detail that can be looked up in the pipeline if of interest.
  • Don't require the 'projects-storage.eclipse.org-bot-ssh' SSH agent for the aggregation step.
  • Only run publication on explicit invocation (from I-builds). Remove the timer-trigger: Without changes a (re-)publication is not necessary.

With this change following Jenkins jobs can be deleted:

And one new jobs will be added:

@HannesWell HannesWell requested a review from merks January 2, 2025 12:14
Copy link
Contributor

@merks merks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks okay. I can't judge all the details.

@merks
Copy link
Contributor

merks commented Jan 2, 2025

/request-license-review

Currently the publication of the Maven artifacts of Eclipse happens in
five jobs manually/directly defined in the 'Releng' Jenkins instance:
1. CBIaggregator
2. RepositoryValidator
3. PublishPlatformToMaven, PublishJDTtoMaven and PublishPDEToMaven

Corresponding job-definitions for these jobs already exist in this SCM
(except for the repository-validation), but the jobs generated from them
(in the 'Releng' folder instead of the root) are not used.

This change replaces the existing (unused) multiple job-definitions by a
single Jenkins pipeline that contains all steps and performs the final
publication for all covered projects in a single matrix.
The new pipeline is updated to incorporate all the changes made in the
meantime to the manually maintained publication jobs.

The I-build job is changed to use the new publication job.

With respect to the existing, manually maintained publication job, the
new job is adjusted in the following aspects:

- Rename job to just 'PublishToMaven'. The CBI part by itself is
irrelevant and a tool in the publication process
- Remove documentation about the jobs involved and scripts called from
the job description as it's an implementation detail that can be looked
up in the pipeline if of interest.
- Don't require the 'projects-storage.eclipse.org-bot-ssh' SSH agent for
the aggregation step.
- Only run publication on explicit invocation (from I-builds). Remove
the timer-trigger: Without changes a (re-)publication is not necessary.
- Make '-snapshot' the default value of the 'snapshotOrRelease'
parameter instead of '-release'.
@HannesWell HannesWell force-pushed the cleanup-maven-publication branch from b467438 to f7fd82a Compare January 2, 2025 12:35
@HannesWell
Copy link
Member Author

Looks okay. I can't judge all the details.

I already did a lot of testing in a separate job and there everything was fine. I mainly wanted to make you aware of the intended changes in the logic, i.e. the removed timer-trigger.

And while talking about the trigger:
In the current (to be deleted) CBIaggregator job the -release argument is the default, which has the effect that since the cron-trigger was reactivated (#2698 (comment)) we had a 'release' publication each night.
Luckily the staging maven-repo for a release is not published directly, but we probably have to discard a number of staging repos for eclipse-platform, JDT and PDE.
@merks, @akurtakov or @MohananRahul do you have access to the OSSRH account for the Eclipse-SDK and can check if this assumption is true and if yes, discard/drop:
https://central.sonatype.org/publish/release/#locate-and-examine-your-staging-repository

@merks
Copy link
Contributor

merks commented Jan 2, 2025

Sorry, I've never had access to the sonatype repositories for this so I can't comment on that.

I'm glad to see things more organized and cleaner!

Copy link
Contributor

github-actions bot commented Jan 2, 2025

/request-license-review

License review requests:

After all reviews have concluded, re-run the license-vetting check from the Github Actions web-interface to update its status.

Workflow run (with attached summary files):
https://github.com/eclipse-platform/eclipse.platform.releng.aggregator/actions/runs/12582463991

@HannesWell
Copy link
Member Author

Sorry, I've never had access to the sonatype repositories for this so I can't comment on that.

No problem, cleaning up stale/undesired staging repos (if they even really exist) can be done later at any time.

I'm glad to see things more organized and cleaner!

Then lets proceed here and apply this. There is still potential for optimization, e.g. parallelizing checks to hopefully speed them up, but that's something that can be done in a follow-up.

@HannesWell HannesWell merged commit 4555fa3 into eclipse-platform:master Jan 2, 2025
3 of 4 checks passed
@HannesWell HannesWell deleted the cleanup-maven-publication branch January 2, 2025 13:26
@HannesWell
Copy link
Member Author

The new PublishToMaven job has been created in the RelEng JIPP and added to the Publish to Maven view.
The first snapshot deployment worked as expected as one can verify by checking the time-stamps in
https://repo.eclipse.org/content/repositories/eclipse-snapshots/org/eclipse/platform

These jobs are now disabled (i.e. not triggered anymore) and I added a note that they are obsolete.
I'll delete them in a day or two when we are confident that the new job is working as desired.

These unused jobs are now already deleted.

HannesWell added a commit to HannesWell/eclipse.platform.releng.aggregator that referenced this pull request Jan 2, 2025
Restore the previous definition of the 'snapshotOrRelease' parameter.

Follow-up on
- eclipse-platform#2718
HannesWell added a commit to HannesWell/eclipse.platform.releng.aggregator that referenced this pull request Jan 2, 2025
Restore the previous (correct) definition of the 'snapshotOrRelease'
parameter.

Follow-up on
- eclipse-platform#2718
HannesWell added a commit that referenced this pull request Jan 2, 2025
Restore the previous (correct) definition of the 'snapshotOrRelease'
parameter.

Follow-up on
- #2718
HannesWell added a commit to HannesWell/eclipse.platform.releng.aggregator that referenced this pull request Jan 2, 2025
HannesWell added a commit that referenced this pull request Jan 2, 2025
@MohananRahul
Copy link
Contributor

Looks okay. I can't judge all the details.

I already did a lot of testing in a separate job and there everything was fine. I mainly wanted to make you aware of the intended changes in the logic, i.e. the removed timer-trigger.

And while talking about the trigger: In the current (to be deleted) CBIaggregator job the -release argument is the default, which has the effect that since the cron-trigger was reactivated (#2698 (comment)) we had a 'release' publication each night. Luckily the staging maven-repo for a release is not published directly, but we probably have to discard a number of staging repos for eclipse-platform, JDT and PDE. @merks, @akurtakov or @MohananRahul do you have access to the OSSRH account for the Eclipse-SDK and can check if this assumption is true and if yes, discard/drop: https://central.sonatype.org/publish/release/#locate-and-examine-your-staging-repository

I think @elsazac have it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants