Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reopen java.net.MalformedURLException in executable jar with OSGi Connect (Atomos) #6245

Open
novos40 opened this issue Aug 28, 2024 · 12 comments
Assignees

Comments

@novos40
Copy link

novos40 commented Aug 28, 2024

Found an old issue (java.net.MalformedURLException in executable jar with OSGi Connect (Atomos)) which was closed without positive solution, I guess. Now I have the same problem in the same situation:

! Unexpected error in the run body: Unable to cache bundle: jar/com.atomiton.sff.api.jar
org.osgi.framework.BundleException: Unable to cache bundle: jar/com.atomiton.sff.api.jar
        at org.apache.felix.framework.Felix.installBundle(Felix.java:3258)
        at org.apache.felix.framework.BundleContextImpl.installBundle(BundleContextImpl.java:147)
        at org.apache.felix.framework.BundleContextImpl.installBundle(BundleContextImpl.java:120)
        at aQute.launcher.Launcher.installEmbedded(Launcher.java:913)
        at aQute.launcher.Launcher.update(Launcher.java:630)
        at aQute.launcher.Launcher.activate(Launcher.java:575)
        at aQute.launcher.Launcher.launch(Launcher.java:403)
        at aQute.launcher.Launcher.run(Launcher.java:185)
        at aQute.launcher.Launcher.main(Launcher.java:161)
        at aQute.launcher.pre.EmbeddedLauncher.executeWithRunPath(EmbeddedLauncher.java:170)
        at aQute.launcher.pre.EmbeddedLauncher.findAndExecute(EmbeddedLauncher.java:119)
        at aQute.launcher.pre.EmbeddedLauncher.main(EmbeddedLauncher.java:52)
Caused by: java.net.MalformedURLException: no protocol: jar/com.atomiton.sff.api.jar
        at java.base/java.net.URL.<init>(URL.java:772)
        at org.apache.felix.framework.util.SecureAction.createURL(SecureAction.java:310)
        at org.apache.felix.framework.cache.JarRevision.initialize(JarRevision.java:148)
        at org.apache.felix.framework.cache.JarRevision.<init>(JarRevision.java:76)
        at org.apache.felix.framework.cache.BundleArchive.createRevisionFromLocation(BundleArchive.java:819)
        at org.apache.felix.framework.cache.BundleArchive.reviseInternal(BundleArchive.java:491)
        at org.apache.felix.framework.cache.BundleArchive.<init>(BundleArchive.java:155)
        at org.apache.felix.framework.cache.BundleCache.create(BundleCache.java:457)
        at org.apache.felix.framework.Felix.installBundle(Felix.java:3254)
        ... 11 more

Everything works from eclipse. Executable jar fails with the above exception. Here is the content of .bndrun file:

-runfw: org.apache.felix.framework;version='[7.0.5,7.0.5]'
-runee: JavaSE-21
-runpath: \
        biz.aQute.launcher-full,\
        org.apache.felix.atomos

-runrequires: \
	bnd.identity;id='org.apache.felix.gogo.shell',\
	bnd.identity;id='org.apache.felix.gogo.command',\
	bnd.identity;id='org.apache.felix.log',\
	bnd.identity;id='org.jboss.netty',\
	bnd.identity;id='org.hsqldb.hsqldb',\
	bnd.identity;id='org.ops4j.pax.logging.pax-logging-api',\
	bnd.identity;id='org.ops4j.pax.logging.pax-logging-log4j2',\
	bnd.identity;id='org.apache.commons.jexl',\
	bnd.identity;id='org.apache.commons.lang3',\
	bnd.identity;id='osgi.enterprise',\
	bnd.identity;id='oda.lib3',\
	bnd.identity;id='com.atomiton.sff.api',\
	bnd.identity;id='com.atomiton.sff.imp.base',\
	bnd.identity;id='com.atomiton.sff.imp.facet',\
	bnd.identity;id='com.atomiton.sff.imp.netty',\
	bnd.identity;id='com.atomiton.sff.dataflow'

-runbundles: \
	com.atomiton.sff.api;version=snapshot,\
	com.atomiton.sff.imp.base;version=snapshot,\
	com.atomiton.sff.imp.netty;version=snapshot,\
	com.atomiton.sff.dataflow;version=snapshot,\
	com.atomiton.sff.imp.facet;version=snapshot,\
	oda.lib3;version=snapshot,\
	org.apache.commons.jexl;version='[2.1.1,2.1.2)',\
	org.apache.commons.lang3;version='[3.5.0,3.5.1)',\
	org.jboss.netty;version='[3.10.6,3.10.7)',\
	org.apache.felix.gogo.command;version='[1.1.2,1.1.3)',\
	org.apache.felix.gogo.shell;version='[1.1.4,1.1.5)',\
	org.apache.felix.scr;version='[2.1.28,2.1.29)',\
	org.osgi.util.function;version='[1.1.0,1.1.1)',\
        org.osgi.util.promise;version='[1.1.1,1.1.2)',\
	osgi.enterprise;version='[5.0.0,5.0.1)',\
	org.lucee.activation;version='[1.1.1,1.1.2)',\
	org.ops4j.pax.logging.pax-logging-api;version='[2.1.0,2.1.1)',\
	org.ops4j.pax.logging.pax-logging-log4j2;version='[2.1.0,2.1.1)',\
	org.apache.felix.configadmin;version='[1.9.26,1.9.27)',\
	org.apache.felix.gogo.runtime;version='[1.1.6,1.1.7)',\
	org.apache.felix.log;version='[1.3.0,1.3.1)',\
	org.hsqldb.hsqldb;version='[2.7.3,2.7.4)'

-resolve.effective: active
-runvm: @../run/java-options.txt,\n\
	-ea,\n\
	-Xmx8192m,\n\

As you can see I have

-runpath: \
        biz.aQute.launcher-full,\
        org.apache.felix.atomos

so launcher can start framework with connector. I had to repackage biz.aQute.launcher into biz.aQute.launcher-full to include aQute.bnd.build package which seem to be required to build the connector. With default launcher I'm getting the following error at startup:

ERROR: Bundle biz.aQute.launcher [4] Error starting atomos:boot:C:\Users\o_dan\.m2\repository\biz\aQute\bnd\biz.aQute.launcher\7.0.0\biz.aQute.launcher-7.0.0.jar (org.osgi.framework.BundleException: Unable to resolve biz.aQute.launcher [4](R 4.0): missing requirement [biz.aQute.launcher [4](R 4.0)] osgi.wiring.package; (&(osgi.wiring.package=aQute.bnd.build)(version>=4.5.0)(!(version>=5.0.0))) Unresolved requirements: [[biz.aQute.launcher [4](R 4.0)] osgi.wiring.package; (&(osgi.wiring.package=aQute.bnd.build)(version>=4.5.0)(!(version>=5.0.0)))])
org.osgi.framework.BundleException: Unable to resolve biz.aQute.launcher [4](R 4.0): missing requirement [biz.aQute.launcher [4](R 4.0)] osgi.wiring.package; (&(osgi.wiring.package=aQute.bnd.build)(version>=4.5.0)(!(version>=5.0.0))) Unresolved requirements: [[biz.aQute.launcher [4](R 4.0)] osgi.wiring.package; (&(osgi.wiring.package=aQute.bnd.build)(version>=4.5.0)(!(version>=5.0.0)))]
! org.osgi.framework.FrameworkEvent[source=biz.aQute.launcher [4]]
	at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:4398)
	at org.apache.felix.framework.Felix.startBundle(Felix.java:2308)
	at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1566)
	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:297)
	at java.base/java.lang.Thread.run(Thread.java:1583)

It does work eventially even with this error due to framework refresh at some point (so bundles get rewired), I guess, but I can't stand having exception as a first thing in the log.

The whole excersise is to allow running python code in OSGi environment via graalvm polyglot. With graal v23.1.1 I used to just build polyglot bundle and disable classpath isolation, but with the latest v24.0.2 polyglot only works from module path hence the necessity to use atomos. It works nicely when run from eclipse.

Now how do I make it work from executable jar? What am I doing wrong here?

@novos40
Copy link
Author

novos40 commented Aug 30, 2024

Found the problem and a fix. In aQuite.launcher.Launcher.java in lines 909..911 there is a check for JRT protocol

//				if (parms.embedded && JRT_PROTOCOL.equals(resource.getProtocol())) {
					path = resource.toString();
//				}

This check needs to be commented out like above or also allow "jar:" protocol (simple comment seems to work for both embedded and normal eclipse launch). The problem is that in embedded case the internal bundle location is just a path within the executable jar like /jar/bundle.jar without any protocol hence java.net.MalformedURLException: no protocol exception. Correct URL, however, is already generated in lines 905..906 as a resource URL

			URL resource = getClass().getClassLoader()
				.getResource(path);

in the form jar:file:/executable.jar!/jar/bundle.jar, i.e. with correct protocol "jar:file:/". If you pass this as a location into context.installBundle(path) bundle jar is read from executable jar correctly.

Now the problem is how to build biz.aQute.launcher-full,jar with modified file. Tried to add modified sources to my maven project, but got a whole bunch of missing dependencies and weird compilation errors (e.g. aQute.lib.collections.SortedList implements both List and SortedSet interfaces which have incompatible reversed() methods). I did not spend much time figuring this out and simply collected all dependencies, compiled modified Launcher.java and patched the existing jar. Everything worked after that.

It would be nice if somebody explained how to correctly build biz.aQute.launcher.jar with correct Launcher and including aQute.bnd.build package without resorting to hacks and patches.

Thanks in advance

@juergen-albert
Copy link
Contributor

If I understood everything correctly, you have a customized Launcher and want to know how to use this launcher in your exported jar right?

In that case you can simply copy the whole bnd launcher project, rename it a bit, do your fix and let bnd build it. Then you have to put the bundle with your launcher in it on the -runpath if I'm not wrong. bnd will look for any bundle with the Header Launcher-Plugin in the Manifest and use this, instead of its build in launcher. Some Details can be found here: https://bnd.bndtools.org/chapters/300-launching.html under Launcher Architecture.

@juergen-albert
Copy link
Contributor

I will close the issue, given my comment above. If you have the feeling, that my answer is not what you wanted feel free to reopen this issue.

@novos40
Copy link
Author

novos40 commented Aug 30, 2024

OK, Obviously I was not clear of what I want.

  1. I don't want to use custom launcher. I want standard launcher to work in all cases.
  2. The cases are:

a) Support org.apache.felix.atomos in -runpath so framework can work with --module-path parameter in command line. Current problem with that is the following exception at startup

ERROR: Bundle biz.aQute.launcher [4] Error starting atomos:boot:C:\Users\o_dan\.m2\repository\biz\aQute\bnd\biz.aQute.launcher\7.0.0\biz.aQute.launcher-7.0.0.jar (org.osgi.framework.BundleException: Unable to resolve biz.aQute.launcher [4](R 4.0): missing requirement [biz.aQute.launcher [4](R 4.0)] osgi.wiring.package; (&(osgi.wiring.package=aQute.bnd.build)(version>=4.5.0)(!(version>=5.0.0))) Unresolved requirements: [[biz.aQute.launcher [4](R 4.0)] osgi.wiring.package; (&(osgi.wiring.package=aQute.bnd.build)(version>=4.5.0)(!(version>=5.0.0)))])
org.osgi.framework.BundleException: Unable to resolve biz.aQute.launcher [4](R 4.0): missing requirement [biz.aQute.launcher [4](R 4.0)] osgi.wiring.package; (&(osgi.wiring.package=aQute.bnd.build)(version>=4.5.0)(!(version>=5.0.0))) Unresolved requirements: [[biz.aQute.launcher [4](R 4.0)] osgi.wiring.package; (&(osgi.wiring.package=aQute.bnd.build)(version>=4.5.0)(!(version>=5.0.0)))]
! org.osgi.framework.FrameworkEvent[source=biz.aQute.launcher [4]]
	at org.apache.felix.framework.Felix.resolveBundleRevision(Felix.java:4398)
	at org.apache.felix.framework.Felix.startBundle(Felix.java:2308)
	at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1566)
	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:297)
	at java.base/java.lang.Thread.run(Thread.java:1583)

Apparently aQute.bnd.build package must be included in the launcher jar as, apparently, it must be able to build connector before any other bundles are loaded.

b) Standard launcher does not work from executable jar. It fails with the following exception:

Caused by: java.net.MalformedURLException: no protocol: jar/com.atomiton.sff.api.jar
        at java.base/java.net.URL.<init>(URL.java:772)
        at org.apache.felix.framework.util.SecureAction.createURL(SecureAction.java:310)
  1. The fix for the first problem is to include aQute.bnd.build package (and its dependencies) into launcher jar
  2. The fix for the second problem is to use correct URL for embedded jar access as described above.
  3. I am not at all familiar with inner workings of launcher so I have no idea if my fix is correct in all situations. I'd like you to build correct launcher jar with both problems fixed (e.g. v 7.0.1) and make it available from maven repo for everyone to use

In a meanwhile I'd also like you to explain to me how to build launcher jar from sources.
First I've tried to copy biz.aQuite.launcher project and run java -jar biz.aQute.bnd-7.0.0.jar build command in it. This results in the following exception

Errors
000: Exception: java.lang.NullPointerException: Cannot invoke "aQute.bnd.build.Workspace.getProject(String)" because "this.ws" is null
        at aQute.bnd.main.bnd.getProject(bnd.java:2429)
        at aQute.bnd.main.bnd.handleOptions(bnd.java:4038)
        at aQute.bnd.main.bnd.getFilteredProjects(bnd.java:950)
        at aQute.bnd.main.bnd.perProject(bnd.java:920)
        at aQute.bnd.main.bnd.perProject(bnd.java:916)
        at aQute.bnd.main.bnd._build(bnd.java:986)
        at aQute.lib.getopt.CommandLine.execute(CommandLine.java:159)
        at aQute.bnd.main.bnd._bnd(bnd.java:445)
        at aQute.lib.getopt.CommandLine.execute(CommandLine.java:159)
        at aQute.bnd.main.bnd.start(bnd.java:293)
        at aQute.bnd.main.bnd.main(bnd.java:270)

001: Cannot invoke "aQute.bnd.build.Workspace.getProject(String)" because "this.ws" is null

Second I've tried to copy the whole repo and run the same command from within biz.aQute.launcher folder. I've got the following output:

Warnings
000: biz.aQute.launcher: Unused -privatepackage instructions , no such package(s) on the class path: [aQute.launcher, aQute.launcher.constants, aQute.launcher.minifw, aQute.launcher.plugin]
001: biz.aQute.launcher: pre.bnd: Unused -privatepackage instructions , no such package(s) on the class path: [aQute.launcher.agent, aQute.launcher.pre]
002: biz.aQute.launcher: pre.bnd: System command git describe --dirty --always --abbrev=9 failed with exit code 128 (allowed)
003: biz.aQute.launcher: pre.bnd: System command git rev-list -1 --no-abbrev-commit HEAD failed with exit code 128 (allowed)
004: biz.aQute.launcher: System command git describe --dirty --always --abbrev=9 failed with exit code 128 (allowed)
005: biz.aQute.launcher: System command git rev-list -1 --no-abbrev-commit HEAD failed with exit code 128 (allowed)

and generated biz.aQute.launcher.jar is empty.
What am I doing wrong here?

@novos40
Copy link
Author

novos40 commented Aug 30, 2024

I will close the issue, given my comment above. If you have the feeling, that my answer is not what you wanted feel free to reopen this issue.

How do I reopen the issue? It looks like I am not allowed.

@stbischof
Copy link
Contributor

Did you tried co checkout this project, fix as you expect and test if then still everything works, and then do a PR?

@novos40
Copy link
Author

novos40 commented Sep 5, 2024

As I've mentioned earlier, I don't know how to build the project. After the build the resulting jar is empty. I can try if you provide instructions on how to build.

@chrisrueger
Copy link
Contributor

chrisrueger commented Sep 5, 2024

As I've mentioned earlier, I don't know how to build the project. After the build the resulting jar is empty. I can try if you provide instructions on how to build.

Have you tried the build instructions here? @novos40
https://github.com/bndtools/bnd/blob/master/CONTRIBUTING.md

@pkriens
Copy link
Member

pkriens commented Nov 4, 2024

@juergen-albert since we're talking about releasing, will you fix this or should we close this?

@novos40
Copy link
Author

novos40 commented Nov 5, 2024

Sorry, I did not have time to fix this. For my purposes the problem is actually bigger as, apparently, Atomos can't create new module layers unless it was itself loaded via module path. This means it must be specified in command line. This, in turn, makes it impossible to have a single config file where all bundles and modules are specified and discovered/loaded dynamically after JVM is already started. So I need a way to execute my custom config code to add discovered modules before connector (i.e. Atomos instance) is created/initialized. I don't see how it is possible with this implementation (unless you can educate me). I will likely modify EmbeddedLauncher to look for something like "Launcher-Config" to be able to specify a class with custom config logic to be executed before anything else. (Custom logic would include setting/modifying system properties and placing discovered modules into new module layers so they can be processed by Atomos)
Since it looks like I'm the only one who needs this level of customization you can close this. Currently this is not a show-stopper so I will deal with this sometime later.

Thanks!

@pkriens
Copy link
Member

pkriens commented Nov 5, 2024

You are aware of the experimental code that runs a connect framework? this requires -runframework services in your bndrun. If this is set to services, the launcher will load a ModuleConnector from the META-INF/services. You should be able to do what you want then?

private Framework createConnect(ClassLoader loader, Map<String, String> configuration) {

This is eons ago for me and and have no time to figure it out. Just wanted to be sure you knew that this experiment is still in the code.

@novos40
Copy link
Author

novos40 commented Nov 5, 2024

No, this is not the problem. Connector gets created and works just fine as long as launcher bundle exports all required packages (this was one of the problems: missing packages for connector creation in launcher jar). The problem is that I need to run custom code BEFORE connector is created and framework starts. After connector is initialized (obviously before framework start) there is little you can do to add new modules to JVM module path UNLESS connector (i.e. Atomos) was loaded via module path. At least this is the error it gives when you try to add module layers via Atomos API. Using --module-path or related option in command line immediately breaks compatibility with old environments still running java 8. They are not going to be updated any time soon as they are closed systems. I would like to have a fully automated system which can start via standard java -jar <my.jar> command line and then discover and properly initialize its own environment whether it is java 8 or 21+ and whether modules are used or not. It is a developing situation because the main reason to use modules is to be able to run python code in java via graalvm. GraalVm team also makes progress, but currently they seem to only support running guest languages via module system. I think they mark it as a bug after my request, but it's not high priority and I'd like to have this capability anyway as I can't really expect anyone to support both module-bases and legacy-based JVMs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants
@chrisrueger @pkriens @juergen-albert @novos40 @stbischof and others