Will qtjambi support GraalVM native image in the future?

[CierConnor]

Hello, author！
Javafx can be run in aot mode, which is cool.
Will qtjambi support GraalVM native image in the future?

[omix]

I am not familiar with this framework. Up to now, there is nothing planned to connect these technologies. However, I am happy about more details.

[omix]

Using QtJambi with GraalVM is possible.
Follow the instructions to create a "hello World" example (class: Test).

The next step is to run the program in normal VM to acknowledge all JNI, resource and reflection usages. Therefore call:

graalvm\bin\java -agentlib:native-image-agent=config-output-dir=native-image -cp QtJambi\6.4.3\deployment\qtjambi-6.4.3.jar;. -Djava.library.path=C:\Qt\6.4.2\msvc2019_64\bin;QtJambi\6.4.3\build\windows-x64\qtjambi\bin -Dio.qt.acknowledge-resources=true Test

(io.qt.acknowledge-resources will be available in next release.
Without it, QtJambi will not be able to find resources like new QIcon(":/images/file.png") in the executable.)

GraalVM creates a number of json files in directory native-images.
Now, create a native image from Java classes and meta data:

graalvm\bin\native-image -cp QtJambi\6.4.3\deployment\qtjambi-6.4.3.jar;. --no-fallback -H:ConfigurationFileDirectories=native-image Test

Finally, call the program:

set PATH=C:\Qt\6.4.2\msvc2019_64\bin;QtJambi\6.4.3\build\windows-x64\qtjambi\bin;%PATH%
test.exe

[ITSweets]

Finally, run the executable's command to set environment variables, are you going to provide dll(so) no longer as a jar, but independently like Qt? I think that's good

[omix]

The easiest way to ship QtJambi is Maven. There, you can get all Java modules and native components as JAR. However, it is not necessary to use QtJambi's DLLs from JAR bundle. You can unzip them and place them next to your Qt libraries.

In normal execution QtJambi is searching for libraries in PATH/LD_LIBRARY_PATH, java.library.path and io.qt.library-path-override (1). If there is no DLL it analyzes classpath for native bundles (2). DLLs are always extracted before loaded. Usually, they are extracted to process-specific temporary directory and deleted during/after process termination. There are system properties allowing you to change this behavior (see here...).

You could run Java with -Dio.qt.keep-temp-deployment=true -Dio.qt.deploymentdir=bin to extract the DLLs to directory of your choice. Then, modify your Java main method and add System.setProperty("io.qt.no-native-deployment", "true"); right before ever touching any QtJambi class or object. This property skips library search at startup by only expecting DLLs in library path. From no on, QtJambi will no longer use native bundles. Now, you can create Graal native image as described above. The executable will need all required Qt and QtJambi DLLs in a directory to be found as stated above (see 1). Native image binary never links against Qt or QtJambi DLLs. They are loaded by Qtjambi's library management at startup.

[ITSweets]

Wow! This is so cool. HOPEFULLY THE SAME WILL BE AVAILABLE IN QTJAMBI 5.X. Since exposure to QtJambi, the framework is the most powerful Gui in Java. Any GUI will have more or less problems later when it comes to binary distribution, either too slow to start or too high in memory. Obviously, if you can package QtJambi in GraalVM, this will be a revolutionary moment for QtJambi.

[omix]

As far as I see, even older versions of QtJambi can be used with Graal and it does not matter which Qt version is used. Up to now, the only modification of QtJambi related to Graal is io.qt.acknowledge-resources and it is only needed to let Graal's Java agent automatically detect when classpath resources are used. That's actually something you also can specify manually as described in GraalVM documentaion.

[ITSweets]

During your testing, after the executable is generated, does the executable need to run with a dependency on the jar? I see you use the --no-fallback compile option in the command. Use this to compile out the executable, I tried this. However, this kind of compilation failure is displayed after the compilation is completed, which means that the compilation with this option may be the product of failure. According to my tests, JavaFX compiled can be run without jar with only one executable. I don't know if the compilation on your side with this option runs independently of the jar?

[omix]

As far as I understood, the native image (executable) incorporates all Java code from all jars the program is using. There are no more jars required. Also, I thought, native image by default falls back in VM mode, i.e. it executes Java code from jars in interpreted manner as JVM does. By using --no-fallback you force native image to be Java-independent. That's how I understood it. Maybe, I'm wrong. Nevertheless, without --no-fallback, my executable did not have issues with JNI calls. I think that's because a JVM is taking over.

[ITSweets]

Will the upcoming io.qt.acknowledge-resources only be used to analyze QtJambi-related reflection agents? If you use QtJambi as the GUI, you may use IQ and other third F libraries for some logic processing, such as Quartz task scheduling, Spring..... Can the tool also analyze QtJambi dependencies?

[omix]

io.qt.acknowledge-resources is only used when running your program in Java with -agentlib:native-image-agent. Native image agent recognizes all uses of Java reflection, Java method handles, JNI access to Java code, loaded resources and serialization. The information is stored to several json files as meta data needed by native image generator. By io.qt.acknowledge-resources you make sure, whenever Qt is using a resource from classpath e.g. by QFile(":/resource/from/classpath") or by QUrl("qrc:/resource/from/classpath") this is acknowledged by native-image-agent. That's all. If you don't use classpath resources, skip this. If you intend to write resource information json file manually, skip this.

If you are accessing classpath resources not listed in the meta data, the resource will simply not be available.
If your program is making a JNI access to Java code not listed in the meta data, the program will freeze. Thus, make sure, you do every possible execution path of your program with native-image-agent prior to creating native image.

[omix]

I'm happy to make you happy with QtJambi.
Do you want to support this project?

[ITSweets]

Hi author! GraalVM compiles QtJambi without a GUI. I followed your operation to compile the local image with GraalVM on Windows 10, everything went smoothly, the compiled executable has 20.8 MB, but no GUI form shows up, it becomes a console app. Why is that? No other libraries are used, and the program is simple.

public class Main {
    public static void main(String[] args) {
        QApplication.initialize(args);

        var app = new QWidget();
        app.resize(600, 400);
        app.setWindowTitle("QtJambi 5.15.13");
        app.show();
        QApplication.exec();
        QApplication.shutdown();
    }
}

The command to generate a reflection-related JSON file is as follows:
D:\Environment\GraalVM-CE-Java-17-22.3.0\bin\java -agentlib:native-image-agent=config-output-dir=native-image -cp D:\Environment\Maven-Repository\io\qtjambi\qtjambi\5.15.13\qtjambi-5.15.13.jar;. -Djava.library.path=D:\Environment\Qt-5.15.5-MSVC-2019-64Bit\bin;C:\Users\Administrator\Desktop\New\QtJambi\bin -Dio.qt.acknowledge-resources=true Main.java

But executing the command yields a warning: Unsupported reflection method: methodTypeDescriptor

My pragma is as follows:
D:\Environment\GraalVM-CE-Java-17-22.3.0\bin\native-image -cp qtjambi-5.15.13.jar --no-fallback -H:ConfigurationFileDirectories=native-image vip.itsweets.Main

At this point, the executable is generated. But there is no GUI. What's going on?
Without a GUI, it becomes a console app with no output

Qt 5.15.15 (I compiled it myself)
QtJambi 5.15.13
Windows 10
MSVC 2019
GraalVM-CE-Java-17-22.3.0

[omix]

I think this is something to ignore: Unsupported reflection method: methodTypeDescriptor
I don't know where this method is used. Nevertheless, native-image worked for me.

[omix]

The command to generate a reflection-related JSON file seems to be wrong. It should end with vip.itsweets.Main because that's your main class. Running your program with java -agentlib:native-image-agent should show the widget as well. When building native-image of an extensive UI application you will have to make sure that every code path is visited by JRE here.

[ITSweets]

Or did I miss something? When you tested before, did the GUI program display normally after compiling?

[omix]

Up to now, I started my generated native image from command line. When I start it by double-click in explorer, a terminal appears showing the outprints. Nevertheless, the Qt window also appears. After closing the window, also the terminal disappears as the program ends here.
I guess ther might be some command line flag for Graal native-image to produce window-only programs for Windows. JRE for Windows provides an additional program javaw.exe starting without terminal window.

[omix]

If the native program freezes there is a JNI call or Java reflection not listed in the configuration JSON files.

[ITSweets]

Hi, just tried again. QtJambi with GraalVM is doable, and as you said, Gui works as expected. But there will be a console window under Windows, and I think I'll probably have to go to the GraalVM community or the documentation for answers about that. I know that there is a Maven plugin on GitHub for GraalVM for JavaFX, and in the example link given to you above, they compile the Java native executable without a console, maybe I can check his compilation log.

[ITSweets]

It does seem to be caused by the package, and it works fine for me to leave Main not in the package, but the warning still appears when generating reflection-related JSON. I don't know why? It's exciting though!! At least QtJambi's development module will be deprecated in the future, and QtJambi should make a module compatible with GraalVM

[ITSweets]

Hi, I have found the answer in GraalVM on how to remove the console window under Windows system. This has nothing to do with GraalVM and QtJambi. THIS ONLY REQUIRES EXECUTING THE RELEVANT INSTRUCTIONS OF THE VC COMPILER: EDITBIN /SUBSYSTEM:WINDOWS main.exe

[ITSweets]

At this point, I think it is necessary for you to customize a module for the combination of GraalVM and QtJambi. Because the project is bound to have complex situations, multiple packages... It is also inconvenient to compile manually, you can get a Maven plugin or other tools to implement it.

[ITSweets]

Oh no! I found that I used Microsoft's VC compiler to remove the console window and click the program's close button, and the Gui disappeared. But the process is still stationed... It seems that the instructions of one VC are not enough to completely solve the problem

[ITSweets]

It's strange, I executed the command EDITBIN /SUBSYSTEM:WINDOWS main.exe on the executable file again and found that this time the process was not stationed after the program was closed. This is good. This also requires multiple tests to determine what the process is stationed after the previous exit. Overall, though, it's good news

[ITSweets]

Hi author! I found that when using GraalVM to generate QtJambi native images, some of the functions required had to be triggered once with trace reflection. In this way, all functions of the compiled program can be guaranteed to be normal. Example: There is a QPushButton on the interface, the click event is processed in another class MainHandler, which needs to trigger the click event when tracking, otherwise, the compiled native program, and then click the button, the corresponding slot function will not be executed, and the program will be stuck

[omix]

Yes! Signal-slot connections either rely on Java reflection (connect(receiver, "slot()")) or on serialization (connect(receiver::slot)). You need to run all of these at least one time in agent mode.

[ITSweets]

But every compilation has to trigger every function of the program, facing complex projects. This kind of release is too expensive, and it is likely that there may be missed triggers when there are many function points. This approach is clearly not going to work.

[ITSweets]

I found out again, hahaha. In the example program above where GraalVM compiles JavaFX, it seems that some classes are manually configured to avoid that some classes may not be automatically traced to the proxy to eventually cause some functionality of the program not to work as expected. It seems that I still have to study the documentation. There is also about removing the console window, you can pass parameters to the VC compiler's linker, and the parameter is the /SUBSYSTEM:WINDOWS above to get rid of the console window. JavaFX above does just that, because I found that this parameter is specified when the linker is linked in the log, so JavcFX's program does not have a console window. But for the time being, I haven't found which option it uses to accomplish that.