Null pointer exception when running BenchmarkRunner on benchmark columba-1.4

[enochii]

Overall Description

Hello, I am using Tai-e's BenchmarkRunner to run the benchmark columba-1.4, and encounter a null pointer exception.

Current Behavior

When running BenchmarkRunner on benchmark columba-1.4, a null pointer exception is thrown. Here is the output:

> Task :BenchmarkRunner.main()

Analyzing columba-1.4
Tai-e starts ...
Output directory: C:\code\Tai-e\output
Writing options to C:\code\Tai-e\output\options.yml
Writing log to C:\code\Tai-e\output\tai-e.log
Writing analysis plan to C:\code\Tai-e\output\tai-e-plan.yml
WorldBuilder starts ...
The world cache mode is enabled.
Loading the world cache from C:\code\Tai-e\cache\world-cache-685358273.bin
[Load the world cache] elapsed time: 6.82s
12396 classes with 115878 methods in the world
WorldBuilder finishes, elapsed time: 6.84s
pta starts ...
Using reflection log from C:\code\Tai-e\java-benchmarks\columba\1.4\refl.log
Reflective target 'byte[]' for Class.forName is not found
Reflective target 'java.lang.String[]' for Class.forName is not found
Reflective caller class 'sun.nio.cs.AbstractCharsetProvider' is absent
Reflective target 'sun.security.pkcs.SignerInfo[]' for Class.forName is not found
Reflective target '<com.sun.crypto.provider.BlowfishCipher: void <init>()>' for Constructor.newInstance is not found
Reflective target '<com.sun.crypto.provider.SunJCE: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.Big5: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.Big5_HKSCS: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.EUC_CN: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.EUC_JP: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.EUC_KR: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.EUC_TW: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.ExtendedCharsets: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.GB18030: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.GBK: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.ISO2022_JP: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.ISO2022_KR: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.ISO_8859_3: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.Johab: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.MS1258: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.MS932: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.SJIS: void <init>()>' for Constructor.newInstance is not found
Reflective target '<sun.nio.cs.ext.TIS_620: void <init>()>' for Constructor.newInstance is not found
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "Object.equals(Object)" because "superclass" is null
	at pascal.taie.language.classes.ClassHierarchyImpl.isSubclass(ClassHierarchyImpl.java:397)
	at pascal.taie.language.type.TypeSystemImpl.isSubtype(TypeSystemImpl.java:180)
	at pascal.taie.analysis.pta.core.solver.TypeFilter.isAssignable(TypeFilter.java:66)
	at pascal.taie.analysis.pta.core.solver.TypeFilter.lambda$apply$0(TypeFilter.java:58)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at pascal.taie.analysis.pta.core.solver.TypeFilter.apply(TypeFilter.java:59)
	at pascal.taie.analysis.pta.core.solver.DefaultSolver.addPFGEdge(DefaultSolver.java:787)
	at pascal.taie.analysis.pta.core.solver.Solver.addPFGEdge(Solver.java:173)
	at pascal.taie.analysis.pta.core.solver.DefaultSolver.lambda$processArrayStore$7(DefaultSolver.java:430)
	at java.base/java.lang.Iterable.forEach(Iterable.java:75)
	at pascal.taie.analysis.pta.core.solver.DefaultSolver.processArrayStore(DefaultSolver.java:425)
	at pascal.taie.analysis.pta.core.solver.DefaultSolver.analyze(DefaultSolver.java:320)
	at pascal.taie.analysis.pta.core.solver.DefaultSolver.solve(DefaultSolver.java:246)
	at pascal.taie.analysis.pta.PointerAnalysis.runAnalysis(PointerAnalysis.java:119)
	at pascal.taie.analysis.pta.PointerAnalysis.analyze(PointerAnalysis.java:107)
	at pascal.taie.analysis.pta.PointerAnalysis.analyze(PointerAnalysis.java:64)
	at pascal.taie.analysis.AnalysisManager.runProgramAnalysis(AnalysisManager.java:148)
	at pascal.taie.analysis.AnalysisManager.runAnalysis(AnalysisManager.java:135)
	at pascal.taie.analysis.AnalysisManager.lambda$execute$0(AnalysisManager.java:104)
	at pascal.taie.util.Timer.runAndCount(Timer.java:93)
	at pascal.taie.analysis.AnalysisManager.lambda$execute$1(AnalysisManager.java:103)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at pascal.taie.analysis.AnalysisManager.execute(AnalysisManager.java:102)
	at pascal.taie.Main.executePlan(Main.java:152)
	at pascal.taie.Main.lambda$main$0(Main.java:60)
	at pascal.taie.util.Timer.lambda$runAndCount$0(Timer.java:112)
	at pascal.taie.util.Timer.runAndCount(Timer.java:93)
	at pascal.taie.util.Timer.runAndCount(Timer.java:111)
	at pascal.taie.util.Timer.runAndCount(Timer.java:107)
	at pascal.taie.Main.main(Main.java:51)
	at pascal.taie.analysis.pta.BenchmarkRunner.run(BenchmarkRunner.java:72)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at pascal.taie.analysis.pta.BenchmarkRunner.runAll(BenchmarkRunner.java:67)
	at pascal.taie.analysis.pta.BenchmarkRunner.main(BenchmarkRunner.java:60)

> Task :BenchmarkRunner.main() FAILED

Execution failed for task ':BenchmarkRunner.main()'.
> Process 'command 'C:\Users\enochii\.jdks\graalvm-ce-17\bin\java.exe'' finished with non-zero exit value 1

Expected Behavior

No null pointer exception.

Tai-e Version

083f0cc

Tai-e Arguments

-cp Tai-e.test pascal.taie.analysis.pta.BenchmarkRunner columba-1.4 -cs ci

JDK Version

graalvm-ce-17

System Environment

Windows 10 x64

Additional Information

optiona.yml:

optionsFile: null
printHelp: false
classPath:
- java-benchmarks/columba/1.4/lib
- java-benchmarks/columba/1.4/lib/bsh-2.0b2.jar
- java-benchmarks/columba/1.4/lib/checkstyle-all-3.4.jar
- java-benchmarks/columba/1.4/lib/commons-cli-1.0.jar
- java-benchmarks/columba/1.4/lib/commons-logging.jar
- java-benchmarks/columba/1.4/lib/core-renderer.jar
- java-benchmarks/columba/1.4/lib/flexdock-0.4.1.jar
- java-benchmarks/columba/1.4/lib/forms-1.0.5.jar
- java-benchmarks/columba/1.4/lib/frapuccino-1.0pre1.jar
- java-benchmarks/columba/1.4/lib/groovy-1.0.jar
- java-benchmarks/columba/1.4/lib/ical4j-0.9.18.jar
- java-benchmarks/columba/1.4/lib/jarbundler-1.9.jar
- java-benchmarks/columba/1.4/lib/jcoverage.jar
- java-benchmarks/columba/1.4/lib/jdom-1.0.jar
- java-benchmarks/columba/1.4/lib/je-2.1.30.jar
- java-benchmarks/columba/1.4/lib/jhall-2.0_02.jar
- java-benchmarks/columba/1.4/lib/jpim.jar
- java-benchmarks/columba/1.4/lib/jreleaseinfo-1.2.0.jar
- java-benchmarks/columba/1.4/lib/jscf-0.3.jar
- java-benchmarks/columba/1.4/lib/junit.jar
- java-benchmarks/columba/1.4/lib/jwizz-0.1.3.jar
- java-benchmarks/columba/1.4/lib/jython-2.2a1.jar
- java-benchmarks/columba/1.4/lib/looks-2.0.2.jar
- java-benchmarks/columba/1.4/lib/lucene-1.4.3.jar
- java-benchmarks/columba/1.4/lib/macchiato-1.0pre1.jar
- java-benchmarks/columba/1.4/lib/mac_mock.jar
- java-benchmarks/columba/1.4/lib/migcalendar-5.6.jar
- java-benchmarks/columba/1.4/lib/quaqua.jar
- java-benchmarks/columba/1.4/lib/ristretto-1.2-all.jar
- java-benchmarks/columba/1.4/lib/swingx.jar
- java-benchmarks/columba/1.4/lib/tagsoup-1.0rc3.jar
- java-benchmarks/columba/1.4/lib/usermanual.jar
- java-benchmarks/columba/1.4/native/linux
appClassPath:
- java-benchmarks/columba/1.4/columba.jar
mainClass: org.columba.core.main.Main
inputClasses: []
javaVersion: 6
prependJVM: false
allowPhantom: true
worldBuilderClass: pascal.taie.frontend.soot.SootWorldBuilder
outputDir: output
preBuildIR: false
worldCacheMode: true
scope: APP
nativeModel: true
planFile: null
analyses:
  pta: advanced:null;reflection-log:java-benchmarks\columba\1.4\refl.log;cs:ci;distinguish-string-constants:null;merge-string-objects:false;reflection-inference:null
  may-fail-cast: ""
  poly-call: ""
onlyGenPlan: false
keepResult:
- $KEEP-ALL

[ayanamists]

Sorry for the delayed response. This problem seems comprehensive and relates to many different parts of Tai-e. I guess this problem may be introduced by the commit 591ecc8 to 648b3a6. Can you use the latest release or the commit 9950155?

We'll fix it as soon as possible.

[enochii]

Thanks a lot! When it is fixed, please kindly remind me here if it's not too much trouble~

[q983868425]

Have you made any progress on this issue? I'm having the same problem.

[zhangt2333]

After some basic debugging, I discovered that the type org.columba.ristretto.auth.mechanism.LoginMechanism is missing its corresponding class, indicating a bug in Soot's frontend robustness—potentially related to reference analysis.

For the current issue (i.e., running Tai-e with -cp Tai-e.test pascal.taie.analysis.pta.BenchmarkRunner columba-1.4 -cs ci), the following modification is needed:

In java-benchmarks/benchmark-info.yml:

-  apps: [ columba/1.4/columba.jar ]
+  apps: [ columba/1.4/columba.jar, columba/1.4/lib/ristretto-1.2-all.jar ]

This also serves as a reminder that until our new frontend is released, we need to manually specify the actual application classes using the -acp or --input-classes options.

[q983868425]

After some basic debugging, I discovered that the type org.columba.ristretto.auth.mechanism.LoginMechanism is missing its corresponding class, indicating a bug in Soot's frontend robustness—potentially related to reference analysis.

For the current issue (i.e., running Tai-e with -cp Tai-e.test pascal.taie.analysis.pta.BenchmarkRunner columba-1.4 -cs ci), the following modification is needed:

In java-benchmarks/benchmark-info.yml:

-  apps: [ columba/1.4/columba.jar ]
+  apps: [ columba/1.4/columba.jar, columba/1.4/lib/ristretto-1.2-all.jar ]

This also serves as a reminder that until our new frontend is released, we need to manually specify the actual application classes using the -acp or --input-classes options.

Thank you for your prompt reply. The stability problem of soot you mentioned, I also found it when testing other benchmarks. Some classes in some dependent jars cannot be parsed.