ant -f eclipse.platform.swt/bundles/org.eclipse.swt/buildSWT.xml tag_projects fails with:

[the-snowwhite]

BUILD FAILED
/home/jenkins/agent/workspace/eclipse.platform.swt_PR-867/eclipse.platform.swt/bundles/org.eclipse.swt/buildSWT.xml:309: Unable to determine new tag

Describe the bug
Releng builder jobs currently fail so that none of recent merged PR's get built.

To Reproduce
Trigger a build.

Expected behavior
Build sicceds

Screenshots / build log:
https://ci.eclipse.org/releng/job/eclipse.platform.swt/view/change-requests/job/PR-863/lastFailedBuild/console

Environment:
Build system

[akurtakov]

For the record:

ant -f buildSWT.xml tag_projects
Buildfile: /home/akurtakov/git/eclipse.platform.swt/bundles/org.eclipse.swt/buildSWT.xml

get_version:
Java 15 has removed Nashorn, you must provide an engine for running JavaScript yourself. GraalVM JavaScript currently is the preferred option.

BUILD FAILED
/home/akurtakov/git/eclipse.platform.swt/bundles/org.eclipse.swt/buildSWT.xml:322: Unable to create javax script engine for javascript

Total time: 0 seconds

This makes it even harder to test on local machine unless one goes extra hoops to supply rhino to the build.

[the-snowwhite]

#867 (comment)

[the-snowwhite]

eclipse-platform/eclipse.platform.releng.aggregator#1528

[the-snowwhite]

@akurtakov
Seems like your suspicions are right on the spot:
The culprit must be right here:
https://github.com/eclipse-platform/eclipse.platform.swt/blob/master/bundles/org.eclipse.swt/buildSWT.xml#L299
ascii 97 is a and ascii 122 is z
So perhaps the solution would be to create a wrap around from z to a ?

[the-snowwhite]

Or perhaps better from z to za .. etc

[mickaelistria]

Thi failure seems introduced by automated commits to update natives (eg 6987505 ) and to disappear magically at some point (I suspect when next I-Build is published).
@iloveeclipse the place where you can see SWT often failing is https://github.com/eclipse-platform/eclipse.platform.swt/commits/master with many red crosses.

[iloveeclipse]

the place where you can see SWT often failing is https://github.com/eclipse-platform/eclipse.platform.swt/commits/master with many red crosses.

They all are unrelated.

[the-snowwhite]

chatgpt4 agrees on all this and suggests this code as a remedy:

    tags = project.getProperty("tags");
    swt_version = project.getProperty("swt_version");
    if (!tags.match(swt_version)) {
        project.setProperty("swt_new_tag", swt_version);
    } else {
        var suffix = 'a';
        while (true) {
            var t = swt_version + suffix;
            if (!tags.match(t)) {
                project.setProperty("swt_new_tag", t);
                break;
            }
            // Increment the suffix alphabetically, e.g., 'a' to 'b', ..., 'z' to 'aa', 'ab', etc.
            var i = suffix.length - 1;
            while (i >= 0) {
                if (suffix[i] === 'z') {
                    suffix = suffix.substring(0, i) + 'a' + (suffix.length > i + 1 ? suffix.substring(i + 1) : '');
                    if (i === 0) {
                        suffix = 'a' + suffix;
                    }
                    i--;
                } else {
                    suffix = suffix.substring(0, i) + String.fromCharCode(suffix.charCodeAt(i) + 1) + (suffix.length > i + 1 ? suffix.substring(i + 1) : '');
                    break;
                }
            }
            // Break out of the loop if the suffix has reached a predefined limit to prevent an infinite loop.
            if (suffix.length > some_predefined_limit) {
                break;
            }
        }
    }

However I'm very far from being able to test, debug and correct this suggestion if needed.
This should wrap around the tags to az azz etc.

[MohananRahul]

Next Build is scheduled at 6am EST.
@akurtakov @iloveeclipse @mickaelistria , FYI using I20231108-0600 build for M3 promotion or to use new IBuild

[the-snowwhite]

In the part:

    if (suffix.length > some_predefined_limit) {
    break;
}

some_predefined_limit needs to be replaced with an int limiting the number of characters azzzzz etc can grow to before
some sort of external reset of the suffix happens any suggestions ?

[the-snowwhite]

BTW. the only sensible way I can test my suggestion anytime soon would be if someone suggests a sensible value for:
some_predefined_limit and I then create a PR.....

[akurtakov]

I wonder why we need these tags at all now that we have automated builds the normal I20231108-1800 like tags should be enough.

[the-snowwhite]

For the record
Searching on swt_new_tag
Gives these results:

9 matches (9 checked) found in current file
/home/mib/git/eclipse.platform.swt/bundles/org.eclipse.swt/buildSWT.xml: 9
287: 11: 	<!-- Set swt_new_tag  -->
288: 37: 	<target name="get_new_tag" unless="swt_new_tag" depends="get_version">
297: 27: 					project.setProperty("swt_new_tag", swt_version);
302: 29: 							project.setProperty("swt_new_tag", t);
312: 23: 					<isset property="swt_new_tag"/>
316: 19: 		<echo>New tag=${swt_new_tag}.</echo>
417: 42: 	<!-- depends on 'get_new_tag' to define swt_new_tag, swt_version, maj_ver and min_ver -->
424: 22: 			<arg line="tag v${swt_new_tag}"/>
427: 22: 			<arg line="tag v${swt_new_tag}"/>

[the-snowwhite]

Seems like the argument for keeping the tags could be faster checking ?

<target name="tag_projects" depends="get_new_tag">
	<!-- These two lines perform the same check. check_libraries is slower because it checks each fragment separetely  -->
	<!--antcall target="check_libraries"/-->
	<antcall target="check_fragment_libraries"/>

	<exec dir="${repo.src}" executable="git" failonerror="true">
		<arg line="tag v${swt_new_tag}"/>
	</exec>
	<exec dir="${repo.bin}" executable="git" failonerror="true">
		<arg line="tag v${swt_new_tag}"/>
	</exec>
</target>

[the-snowwhite]

Sorry I could'nt help myself it was to tempting not to try:
#869

[akurtakov]

What we should be aiming at is moving from 3z to 4a

[mickaelistria]

A quick short fix can be to add an extra loop form 'A' to 'Z' (upper-case)

[the-snowwhite]

Im not quite sure if github support tag suffixes with more than 1 letter or uppercase.
I wote right now for testing incrementing from z to a before ANY merges related to this fix

[mickaelistria]

Im not quite sure if github support tag suffixes with more than 1 letter or uppercase.

Git and GitHub don't care about that

mistria@localhost:~/git/eclipse.platform.swt$ git tag taga
mistria@localhost:~/git/eclipse.platform.swt$ git tag tagaa
mistria@localhost:~/git/eclipse.platform.swt$ git tag tagAA
mistria@localhost:~/git/eclipse.platform.swt$ git tag tagaA
mistria@localhost:~/git/eclipse.platform.swt$ git tag tagAa
mistria@localhost:~/git/eclipse.platform.swt$ git push me tagA taga tagaa tagaA tagAa tagAA
Enumerating objects: 2663, done.
Counting objects: 100% (2663/2663), done.
Delta compression using up to 8 threads
Compressing objects: 100% (709/709), done.
Writing objects: 100% (2377/2377), 641.43 KiB | 19.44 MiB/s, done.
Total 2377 (delta 1484), reused 2097 (delta 1207), pack-reused 0
remote: Resolving deltas: 100% (1484/1484), completed with 147 local objects.
To github.com:mickaelistria/eclipse.platform.swt.git
 * [new tag]               tagA -> tagA
 * [new tag]               taga -> taga
 * [new tag]               tagaa -> tagaa
 * [new tag]               tagaA -> tagaA
 * [new tag]               tagAa -> tagAa
 * [new tag]               tagAA -> tagAA

You can see those tags on my fork of this repo.

[the-snowwhite]

I have created a new PR with a single letter suffix wrap around from z to a.
#870

[the-snowwhite]

Next Build is scheduled at 6am EST. @akurtakov @iloveeclipse @mickaelistria , FYI using I20231108-0600 build for M3 promotion or to use new IBuild

Well seems like we have a viable option to meet this deadline in:
#869
I think removing the tags is a more far reaching effort that can give unforeseen consequences unlucky to deal with in a freeze period.
The tags and removing the extra java code can always be dealt wit later on ?
I have removed the draft status ftom the z -> az suffix commit if someone is trigger happy ?

[akurtakov]

I have pushed in the wrong branch and PR and merge and etc. - Hopefully the fix/workaround is 9a71c28 and https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/master/275/ will show whether this is the case or not, let's give it sometime.

[the-snowwhite]

Well
Seems like:
https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/master/276/
got over the critical point

[akurtakov]

This is definetely not a long term fix but rather a bandaid. Anyway, I can't spend more time on this one as there is a need for total overhaul of the scripts.
I'm starting an I-build to prove things should be fine.

[the-snowwhite]

@mickaelistria
Currently from here it seems like your mishap was a push directly to master of your bandaid pr resulting in.
That the last closed issue track-able merged build is
https://ci.eclipse.org/releng/job/eclipse.platform.swt/view/change-requests/
#869
I know your time is limited but what is now needed to get the ci build into the right flow ?

[mickaelistria]

Build looks OK now: https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/master/277/ . Please make sure that other patches are rebased on top of master.

[the-snowwhite]

Hmmm
I know and am able to rebase my own patch on top of master(by re basing and force pushing that pr).
looking at:
https://github.com/eclipse-platform/eclipse.platform.swt/commits/master
Can this be done in a single pr containing in a new branch made from:
c7549d9
the rebased on top of master ?

[the-snowwhite]

A bit of a hassle as I had deleted the branch:
But this should do the job:
#874

[mickaelistria]

See https://github.com/eclipse-platform/eclipse.platform.swt/commits/master , your patches were merged yesterday and there has been a successful build since then thanks to . You can just update to latest I-Build or download the particular jars from https://download.eclipse.org/eclipse/downloads/drops4/I20231109-0710/

[the-snowwhite]

@mickaelistria
@akurtakov
@iloveeclipse
@MohananRahul
I truly do not understand everybody who suddenly think that it is OK to close all related matters to this issue with
references to the commits in master branch.

The problems were related to merged commits to master branch not being built and binaries therefore not distributed properly towards the M3 and 2023-13 releases.
Furtherly aggregated by an accidental direct commit to master and merge(which is easy to do as the setup docs recommend setting origin to upstream and not fork so that that is the default push unless something else is selected).

Therefore A reference to build logs in https://ci.eclipse.org/releng/job/eclipse.platform.swt/view/change-requests/
Or
https://ci.eclipse.org/releng/job/eclipse.platform.swt/view/default/job/master/

Would seem like a more appropriate place to be able to refer to and verify that builds now are being run correctly ?

However (perhaps maybe because Im a newcomer and do not know what an I-build means ?)
I'm not yet able to find anything in the 2 ci.eclipse.org links that can confirm that the build pipeline now is fixed. for the most recent PR's

[akurtakov]

Unfortunately the build system is truly convoluted. Let me try to explain a bit
So the build that worked for SWT is https://ci.eclipse.org/releng/job/eclipse.platform.swt/view/default/job/master/277/ .
If you look at following builds at https://ci.eclipse.org/releng/job/eclipse.platform.swt/view/default/job/master/buildTimeTrend you'll see that they are marked at "aborted". This is because the scripts there are no changes detected thus aborted.
If you look at a PR created after the fix has been merged in master like https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/PR-873/1/ you'll notice that the build works just fine for it.
To complicate things even more the build that publishes Eclipse SDK with SWT and so on to downloads.eclipse.org is https://ci.eclipse.org/releng/job/Builds/job/I-build-4.30/ which itself is a git submodules aggregator build https://github.com/eclipse-platform/eclipse.platform.releng.aggregator .
So the fix is swt master and aggregator build published to downloads.
I hope this explains why I'm hesitant to see even more complication in that jungle of releng scripts we have.

[the-snowwhite]

Unfortunately the build system is truly convoluted. Let me try to explain a bit So the build that worked for SWT is https://ci.eclipse.org/releng/job/eclipse.platform.swt/view/default/job/master/277/ . If you look at following builds at https://ci.eclipse.org/releng/job/eclipse.platform.swt/view/default/job/master/buildTimeTrend you'll see that they are marked at "aborted". This is because the scripts there are no changes detected thus aborted. If you look at a PR created after the fix has been merged in master like https://ci.eclipse.org/releng/job/eclipse.platform.swt/job/PR-873/1/ you'll notice that the build works just fine for it. To complicate things even more the build that publishes Eclipse SDK with SWT and so on to downloads.eclipse.org is https://ci.eclipse.org/releng/job/Builds/job/I-build-4.30/ which itself is a git submodules aggregator build https://github.com/eclipse-platform/eclipse.platform.releng.aggregator . So the fix is swt master and aggregator build published to downloads. I hope this explains why I'm hesitant to see even more complication in that jungle of releng scripts we have.

@akurtakov
Thank you for the info enlightenment.
It makes very much sense that the keyword is simplification.
Hope I was not too much a distraction for you getting the build jobs up and running again alt least for some time so far.... :-)