Bug: 1052 Adding integration test for Mirror can't find product bundle dependencies. #1085

fipro78 · 2022-06-28T14:28:19Z

No description provided.

github-actions · 2022-08-10T06:30:49Z

Test Results

582 files +3 582 suites +3 4h 1m 43s ⏱️ + 36m 28s
385 tests +1 375 ✅ - 2 7 💤 ±0 2 ❌ +2 1 🔥 +1
1 155 runs +3 1 128 ✅ - 2 22 💤 ±0 2 ❌ +2 3 🔥 +3

For more details on these failures and errors, see this check.

Results for commit 6a3e4ac. ± Comparison against base commit 6b9cefe.

♻️ This comment has been updated with latest results.

laeubi · 2022-09-14T08:10:05Z

@fipro78 can you try to minimize the testcase as indicated in the bug report? I think a smaller test-case can help us to identify the root cause more quickly.

fipro78 · 2022-09-14T08:20:27Z

@laeubi well I can try to if I remember what I have done back then. But IIRC you also asked for a testcase about plugin based products. Shouldn't then there be plugins that are integrated? IIUC you want me to drop everything else but the product and the target right?

laeubi · 2022-09-14T08:24:28Z

Yes I think you can drop all the reactor projects (beside target+product of course), and even reduce the product to only contain the org.osgi.service.jaxrs bundle as a reproducer. Then one even can reduce the target further (smaller target = faster execution times) so we get the smallest possible reproducing test-case :-)

This should then make debugging a lot easier as we have only a handful of bundles in the debug in contrast to now a few hundreds.

fipro78 · 2022-09-14T08:55:02Z

I removed everything and tried to build the project alone locally, but it fails with this error:

[WARNING] The POM for org.eclipse.tycho:tycho-build:jar:3.0.0-SNAPSHOT is missing, no dependency information available
[WARNING] Failed to read extensions descriptor C:\Development\Sources\tycho\tycho-its\projects\product.pluginBased.mavenArtefacts\.mvn\extensions.xml: Plugin org.eclipse.tycho:tycho-build:3.0.0-SNAPSHOT or one of its dependencies could not be resolved: Could not find artifact org.eclipse.tycho:tycho-build:jar:3.0.0-SNAPSHOT
[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[ERROR] Child module C:\Development\Sources\tycho\tycho-its\projects\product.pluginBased.mavenArtefacts\tycho-its-project.issue1052.target\pom.xml of C:\Development\Sources\tycho\tycho-its\projects\product.pluginBased.mavenArtefacts\pom.xml does not exist @
[ERROR] Child module C:\Development\Sources\tycho\tycho-its\projects\product.pluginBased.mavenArtefacts\tycho-its-project.issue1052.jaxrs.product\pom.xml of C:\Development\Sources\tycho\tycho-its\projects\product.pluginBased.mavenArtefacts\pom.xml does not exist @
[ERROR] Unresolveable build extension: Plugin org.eclipse.tycho:tycho-maven-plugin:3.0.0-SNAPSHOT or one of its dependencies could not be resolved: Failed to collect dependencies at org.eclipse.tycho:tycho-maven-plugin:jar:3.0.0-SNAPSHOT -> org.eclipse.tycho:tycho-core:jar:3.0.0-SNAPSHOT -> org.eclipse.platform:org.eclipse.core.runtime:jar:3.25.0 -> org.eclipse.platform:org.eclipse.equinox.preferences:jar:[3.10.0,4.0.0) @

No idea what is wrong. The pom.xml contains the plugins-repository and I trigger the build via

mvn -Dtycho-version=3.0.0-SNAPSHOT clean verify

Any ideas?

laeubi · 2022-09-14T09:03:42Z

Are you using maven 3.8.6?

laeubi · 2022-09-14T09:05:07Z

Sorry for confusion, I think this is because the snapshot repository needs to be configured in the settings.xml

https://github.com/eclipse-tycho/tycho/wiki#configure-snapshots-repository-globally

fipro78 · 2022-09-14T10:00:54Z

@laeubi
I have updated the test case. It shows several errors related to some signing topics, but at the end it fails with the reported error.

laeubi

This looks good now!

laeubi · 2022-09-25T08:13:21Z

@fipro78 I have now debugged this and the problem is that there are Projector warnings that prevent a solution here:

Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.client 0.0.0.
Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.core 0.0.0.
Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.sse 0.0.0.
Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; org.osgi.dto [1.0.0,2.0.0).
Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; org.osgi.framework.dto [1.8.0,2.0.0).
Status WARNING: org.eclipse.equinox.p2.director code=0 Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to osgi.contract; (&(osgi.contract=JavaJAXRS)(version=2.1.0)).

This then leads to the situation (as all requirements of a product are considered "non greedy" and optional) that the final result of the target only contains [a.jre.javase 17.0.0, tycho-its-project.issue1052.jaxrs.product 1.0.0.qualifier] what of course is not enough. It then (later on) fails to publish the content to the repository that is more later used to perform the mirror operation.

laeubi · 2022-09-25T12:41:54Z

....mavenArtefacts/tycho-its-project.issue1052.target/tycho-its-project.issue1052.target.target

+<?pde version="3.8"?>
+<target name="tycho-its-project.issue1052.target">
+	<locations>
+		<location includeAllPlatforms="false" includeConfigurePhase="true" includeMode="planner" includeSource="true" type="InstallableUnit">


This location can be removed.

laeubi · 2022-09-25T12:55:13Z

@fipro78 can you tell if you would have expsected that the reported requirements are already fulfilled? If yes by what bundle(s) from the target platform?

fipro78 · 2022-09-25T13:56:23Z

@laeubi
Well initially the product was working. The target and the product contained everything that was needed and it was possible to start the product via PDE inside Eclipse. You then asked me to drop everything to make the example simpler for debugging. So I really can't tell as with this modification the example got broken. Probably I removed to much.

laeubi · 2022-09-25T13:59:17Z

What I suspect here is that the product would actually works, but P2 actually has a problem here because it can't understand some of the requirements. At least the actual error seems misleading here.

fipro78 · 2022-09-25T14:11:30Z

IIRC the initial product configuration did not show theses warnings, e.g. org.osgi.dto is available if Equinox is included. And I think I also added the necessary xml libraries. I would need to restore the initial setting to see. IMHO the error you see now is different than the initial one.

laeubi · 2022-09-25T14:13:11Z

IIRC the initial product configuration did not show theses warnings,

Well that is because until now these warnings are never printed ;-)

I added some output with this PR:

Make Warnings in the Projector visible #1416

laeubi · 2022-09-25T14:14:56Z

By the way that are the units that are part of the current target platform:
tycho-its-project.issue1052.jaxrs.product 1.0.0.qualifier
osgi.annotation.source 8.0.1.202109301733
org.osgi.namespace.service.source 1.0.0.201505202024
org.osgi.util.promise.source 1.1.1.201810101357
org.osgi.namespace.implementation.source 1.0.0.201505202024
osgi.annotation 8.0.1.202109301733
org.osgi.service.jaxrs 1.0.1.202109301733
org.osgi.service.jaxrs.source 1.0.1.202109301733
org.osgi.util.promise 1.1.1.201810101357
org.osgi.namespace.service 1.0.0.201505202024
org.osgi.util.function 1.1.0.201802012106
org.osgi.util.function.source 1.1.0.201802012106
org.osgi.namespace.implementation 1.0.0.201505202024
a.jre.javase 17.0.0

fipro78 · 2022-09-25T15:07:12Z

The initial target definition contained all necessary dependencies. You can see this in the first commit. While the service api and impl bundles are not necessary for the test, it seems the rest is necessary to show the real error. That said, the initial commit should have been only reduced by removing the api and impl, not the rest.

laeubi · 2022-09-25T16:39:19Z

it seems the rest is necessary to show the real error. That said, the initial commit should have been only reduced by removing the api and impl, not the rest.

I then get the following warnings from the projector:

Unable to satisfy dependency from org.apache.aries.jax.rs.whiteboard 1.0.1 to osgi.contract; (&(osgi.contract=JavaJAXRS)(version=2.1.0)).
Unable to satisfy dependency from org.apache.aries.jax.rs.whiteboard 1.0.1 to osgi.contract; (&(osgi.contract=JavaAnnotation)(version=1.3.0)).
Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.0.201802012106 to osgi.contract; (&(osgi.contract=JavaJAXRS)(version=2.1.0)).
Unable to satisfy dependency from slf4j.api 1.7.2 to java.package; org.slf4j.impl 1.6.0.

The list of resolved units is:

org.osgi.service.jaxrs 1.0.0.201802012106
jakarta.xml.bind-api 2.3.3
jakarta.annotation-api 1.3.5
org.apache.felix.gogo.runtime 1.1.4.v20210111-1007
org.apache.servicemix.specs.annotation-api-1.3 1.3.0.1
org.osgi.util.promise 1.1.0.201802012106
com.sun.activation.javax.activation 1.2.0
org.eclipse.equinox.executable_root.win32.win32.x86_64 3.8.1700.v20220509-0833
org.eclipse.pde.ds.lib 1.1.500.v20210209-1250
org.slf4j.api 1.7.30.v20200204-2150
javax.annotation 1.3.5.v20200909-1856
org.eclipse.equinox.executable.feature.jar 3.8.1700.v20220509-0833
org.eclipse.osgi 3.18.0.v20220516-2155
a.jre.javase 17.0.0
org.eclipse.equinox.http.servlet 1.7.200.v20211021-1418
org.osgi.util.function 1.1.0.201802012106
org.osgi.util.function 1.2.0.202109301733
org.osgi.service.http.whiteboard 1.0.0.201505202024
slf4j.api 1.7.2
org.eclipse.equinox.launcher.win32.win32.x86_64 1.2.500.v20220509-0833
org.eclipse.osgi.services 3.10.200.v20210723-0643
org.apache.servicemix.specs.jaxws-api-2.2 2.9.0
org.eclipse.equinox.launcher 1.6.400.v20210924-0641
org.apache.felix.gogo.command 1.1.2.v20210111-1007
org.apache.felix.scr 2.1.24.v20200924-1939
org.osgi.util.promise 1.2.0.202109301733
org.apache.felix.gogo.shell 1.1.4.v20210111-1007
org.eclipse.equinox.executable.feature.group 3.8.1700.v20220509-0833
org.apache.aries.jax.rs.whiteboard 1.0.1
org.eclipse.equinox.console 1.4.500.v20211021-1418
org.eclipse.equinox.supplement 1.10.500.v20220516-2155
jakarta.servlet-api 4.0.0
tycho-its-project.issue1052.jaxrs.product 1.0.0.qualifier
org.apache.felix.http.servlet-api 1.1.2
org.apache.aries.javax.jax.rs-api 1.0.1
org.apache.servicemix.specs.saaj-api-1.3 2.9.0
tycho-its-project.issue1052.impl 1.0.0.202209251633
org.apache.felix.http.jetty 4.1.14
tycho-its-project.issue1052.api 1.0.0.202209251633

fipro78 · 2022-09-25T17:07:47Z

I would think that the contracts should be resolved via

org.apache.aries.javax.jax.rs-api
org.apache.servicemix.specs.annotation-api

I wonder why the annotation api is resolved via three different jars. Also SLF4J api is available twice. The resolution error does not make sense to me.

laeubi · 2022-09-25T17:38:26Z

@merks are you aware of any problems regarding P2 and osgi.contract handling?

laeubi · 2022-09-25T17:46:38Z

I would think that the contracts should be resolved via

org.apache.aries.javax.jax.rs-api org.apache.servicemix.specs.annotation-api

If I look at this I see

Provide-Capability: osgi.contract;osgi.contract=JavaJAXRS;uses:="javax
 .ws.rs,javax.ws.rs.client,javax.ws.rs.container,javax.ws.rs.core,java
 x.ws.rs.ext,javax.ws.rs.sse";version:List<Version>="2.1,2,1.1"

as there is a version LIST @HannesWell I remember you have investigated some missing support in P2 regarding list support?

Could it be:

Fix creation of generic-capabilities with multiple versions eclipse-equinox/p2#64 ?

HannesWell · 2022-09-25T19:31:33Z

I would think that the contracts should be resolved via
org.apache.aries.javax.jax.rs-api org.apache.servicemix.specs.annotation-api

If I look at this I see

Provide-Capability: osgi.contract;osgi.contract=JavaJAXRS;uses:="javax
 .ws.rs,javax.ws.rs.client,ja

as there is a version **LIST** @HannesWell I remember you have investigated some missing support in P2 regarding list support?

Could it be:

* [Fix creation of generic-capabilities with multiple versions eclipse-equinox/p2#64](https://github.com/eclipse-equinox/p2/pull/64) ?

Yes that's right and it could definitivly be connected.
But a question that I asked myself a few times already but never answered is, does Tycho/P2 even consider Require-/Provide-Capability Headers in general?

laeubi · 2022-09-26T04:23:23Z

But a question that I asked myself a few times already but never answered is, does Tycho/P2 even consider Require-/Provide-Capability Headers in general?

Obviously it does or we wont see this error here, but maybe I don't fully understand the question?

merks · 2022-09-26T09:14:40Z

Here's a simple way to see what the p2 publisher does. Using `platform:/resource/ in the Repository Explorer shows the p2 published results of all workspace projects. Turning on Expert mode lists all the IUs. You can filter to find one and you can double click the version to see the serialized-to-XML p2 metadata of the IU.

There should be nothing special about how osgi.contract is handled...

tycho-its/projects/product.pluginBased.mavenArtefacts/.mvn/extensions.xml

laeubi · 2022-12-13T13:10:44Z

@fipro78 Tycho is updated to new eclipse release so with some small adjustments this should finally work now!

laeubi · 2024-01-19T13:04:59Z

@fipro78 I know this is a very long outstanding issue so sorry for the delay, but I think the root cause might be this here:

Always include the projects resolved target platform as a source #3389

lets see if it not breaks anything and after the merge I hope this will be finally resolved. If not I'm a bit out of clue as it seems no one else is really using plugin based products or facing the issue ...

…ensions.xml

laeubi · 2024-01-20T17:23:41Z

@fipro78 I now have analyzed the problem and actually it works, but the failure we see here is introduced by some mismatch between maven metadata and OSGi metadata:

https://mvnrepository.com/artifact/org.osgi/org.osgi.service.jaxrs/1.0.1 declares the org.apache.aries.javax.jax.rs-api as a provided dependency, that means it is not included in the target.
but org.osgi.service.jaxrs require package javax.ws.rs.client
If one adds this, dependency it reveals that org.apache.aries.javax.jax.rs-api does not declare any maven dependencies but it requires package javax.xml.bind.annotation
If one now adds for example https://mvnrepository.com/artifact/javax.xml.bind/jaxb-api/2.3.1 and what requires https://mvnrepository.com/artifact/javax.activation/javax.activation-api/1.2.0 then the build passes.

Another issue is that Tycho does not fail when resolve the product, that's a bit strange....

See eclipse-tycho#1085

laeubi · 2024-01-20T19:22:25Z

I now replicated the behavior with two minimal test cases:

Fail on missing plugin product requirement #3393

this shows:

feature based product fails to resolve with the error that package javax.ws.rs.client is missing (what is correct)
plugin based product does not fail to resolve (whats wrong)

both test cases have a a requirement to the org.osgi.service.jaxrs bundle that requires package javax.ws.rs.client.

laeubi · 2024-01-20T19:27:01Z

@fipro78 unless the bug in Tycho is fixed, you must watch out for messages like this:

[INFO] Resolving dependencies of MavenProject: ....
[WARNING] Problems resolving provisioning plan.:
   Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.client 0.0.0.
   Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.core 0.0.0.
   Unable to satisfy dependency from org.osgi.service.jaxrs 1.0.1.202109301733 to java.package; javax.ws.rs.sse 0.0.0.

these will give you the hints what is missing and needs to be added to the target for your project to package the product. If sou still see issues with none of these warnings please let us know so I can reopen this PR as further work will be performed in #3393.

fipro78 mentioned this pull request Jun 28, 2022

[WARNING] Mirror tool: Problems resolving provisioning plan #1052

Closed

This comment was marked as outdated.

Sign in to view

laeubi force-pushed the issue_1052_reproducer branch from dfca2f3 to 65b426c Compare August 10, 2022 04:56

laeubi force-pushed the issue_1052_reproducer branch from 65b426c to 43363bd Compare September 7, 2022 13:35

laeubi approved these changes Sep 14, 2022

View reviewed changes

laeubi force-pushed the issue_1052_reproducer branch from 2b52c19 to 5ba7575 Compare September 22, 2022 16:25

laeubi changed the title ~~Bug: 1052 Adding integration test~~ Bug: 1052 Adding integration test for Mirror can't find product bundle dependencies. Sep 25, 2022

laeubi reviewed Sep 25, 2022

View reviewed changes

laeubi mentioned this pull request Sep 25, 2022

Fix creation of generic-capabilities with multiple versions eclipse-equinox/p2#64

Merged

laeubi force-pushed the issue_1052_reproducer branch from 5ba7575 to f18764a Compare December 12, 2022 15:38

laeubi reviewed Dec 13, 2022

View reviewed changes

tycho-its/projects/product.pluginBased.mavenArtefacts/.mvn/extensions.xml Outdated Show resolved Hide resolved

laeubi force-pushed the issue_1052_reproducer branch from 23c2b07 to 3788a65 Compare January 25, 2023 15:59

laeubi force-pushed the issue_1052_reproducer branch from 3788a65 to e20e3d5 Compare March 26, 2023 14:52

laeubi force-pushed the issue_1052_reproducer branch from e20e3d5 to adc26fb Compare May 18, 2023 04:31

laeubi force-pushed the issue_1052_reproducer branch from adc26fb to 7aea981 Compare July 20, 2023 05:34

laeubi force-pushed the issue_1052_reproducer branch from 7aea981 to 82afece Compare December 26, 2023 06:33

akurtakov force-pushed the issue_1052_reproducer branch from 82afece to aa7898f Compare January 6, 2024 07:29

fipro78 and others added 3 commits January 20, 2024 06:07

Bug: 1052 Adding integration test

fe7d117

Bug: 1052 Simplify integration test

addd71c

Update tycho-its/projects/product.pluginBased.mavenArtefacts/.mvn/ext…

6a3e4ac

…ensions.xml

laeubi force-pushed the issue_1052_reproducer branch from aa7898f to 6a3e4ac Compare January 20, 2024 05:07

laeubi added a commit to laeubi/tycho that referenced this pull request Jan 20, 2024

Analysis / Experiments

782d4e5

See eclipse-tycho#1085

laeubi closed this Jan 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug: 1052 Adding integration test for Mirror can't find product bundle dependencies. #1085

Bug: 1052 Adding integration test for Mirror can't find product bundle dependencies. #1085

fipro78 commented Jun 28, 2022

This comment was marked as outdated.

github-actions bot commented Aug 10, 2022 •

edited

Loading

laeubi commented Sep 14, 2022

fipro78 commented Sep 14, 2022

laeubi commented Sep 14, 2022

fipro78 commented Sep 14, 2022

laeubi commented Sep 14, 2022

laeubi commented Sep 14, 2022

fipro78 commented Sep 14, 2022

laeubi left a comment

laeubi commented Sep 25, 2022

laeubi Sep 25, 2022

laeubi commented Sep 25, 2022

fipro78 commented Sep 25, 2022

laeubi commented Sep 25, 2022

fipro78 commented Sep 25, 2022

laeubi commented Sep 25, 2022

laeubi commented Sep 25, 2022

fipro78 commented Sep 25, 2022

laeubi commented Sep 25, 2022

fipro78 commented Sep 25, 2022

laeubi commented Sep 25, 2022

laeubi commented Sep 25, 2022

HannesWell commented Sep 25, 2022

laeubi commented Sep 26, 2022

merks commented Sep 26, 2022

laeubi commented Dec 13, 2022

laeubi commented Jan 19, 2024

laeubi commented Jan 20, 2024

laeubi commented Jan 20, 2024

laeubi commented Jan 20, 2024

Bug: 1052 Adding integration test for Mirror can't find product bundle dependencies. #1085

Bug: 1052 Adding integration test for Mirror can't find product bundle dependencies. #1085

Conversation

fipro78 commented Jun 28, 2022

This comment was marked as outdated.

github-actions bot commented Aug 10, 2022 • edited Loading

Test Results

laeubi commented Sep 14, 2022

fipro78 commented Sep 14, 2022

laeubi commented Sep 14, 2022

fipro78 commented Sep 14, 2022

laeubi commented Sep 14, 2022

laeubi commented Sep 14, 2022

fipro78 commented Sep 14, 2022

laeubi left a comment

Choose a reason for hiding this comment

laeubi commented Sep 25, 2022

laeubi Sep 25, 2022

Choose a reason for hiding this comment

laeubi commented Sep 25, 2022

fipro78 commented Sep 25, 2022

laeubi commented Sep 25, 2022

fipro78 commented Sep 25, 2022

laeubi commented Sep 25, 2022

laeubi commented Sep 25, 2022

fipro78 commented Sep 25, 2022

laeubi commented Sep 25, 2022

fipro78 commented Sep 25, 2022

laeubi commented Sep 25, 2022

laeubi commented Sep 25, 2022

HannesWell commented Sep 25, 2022

laeubi commented Sep 26, 2022

merks commented Sep 26, 2022

laeubi commented Dec 13, 2022

laeubi commented Jan 19, 2024

laeubi commented Jan 20, 2024

laeubi commented Jan 20, 2024

laeubi commented Jan 20, 2024

github-actions bot commented Aug 10, 2022 •

edited

Loading