spark3.3.3 java exception thrown at native-engine/blaze/src/rt.rs:80

[SeaSand1024]

Describe the bug
A clear and concise description of what the bug is.

Caused by: java.lang.RuntimeException: called Result::unwrap() on an Err value: External("Java exception thrown at native-engine/blaze/src/rt.rs:80: org.apache.arrow.c.Data.importSchema(Lblaze/org/apache/arrow/memory/BufferAllocator;Lorg/apache/arrow/c/ArrowSchema;Lorg/apache/arrow/c/CDataDictionaryProvider;)Lblaze/org/apache/arrow/vector/types/pojo/Schema;")
To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

[SeaSand1024]

Isn't this compatible with centos7？

[richox]

could you check the executor log for detailed error message? blaze has no special dependences to operating system. centos7 should be OK.

[zdy02004]

I have the same problem ,How is this problem solved ? executor log has no other useful infomation ,Only like this :

2024-09-23 18:50:23.483 INFO - 18:50:23.479 [task-result-getter-1] ERROR org.apache.spark.scheduler.TaskSetManager - Task 0 in stage 1.0 failed 4 times; aborting job
2024-09-23 18:50:23.773 INFO - org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 19) (117-116-13 executor 2): java.lang.RuntimeException: called Result::unwrap() on an Err value: External("Java exception thrown at native-engine/blaze/src/rt.rs:80: java.lang.NoSuchMethodError: org.apache.arrow.c.Data.importSchema(Lblaze/org/apache/arrow/memory/BufferAllocator;Lorg/apache/arrow/c/ArrowSchema;Lorg/apache/arrow/c/CDataDictionaryProvider;)Lblaze/org/apache/arrow/vector/types/pojo/Schema;")
2024-09-23 18:50:23.773 INFO - at org.apache.spark.sql.blaze.JniBridge.callNative(Native Method)
2024-09-23 18:50:23.773 INFO - at org.apache.spark.sql.blaze.BlazeCallNativeWrapper.(BlazeCallNativeWrapper.scala:66)
2024-09-23 18:50:23.773 INFO - at org.apache.spark.sql.blaze.NativeHelper$.executeNativePlan(NativeHelper.scala:89)
2024-09-23 18:50:23.773 INFO - at org.apache.spark.sql.execution.blaze.shuffle.BlazeShuffleWriterBase.nativeShuffleWrite(BlazeShuffleWriterBase.scala:80)
2024-09-23 18:50:23.773 INFO - at org.apache.spark.sql.execution.blaze.plan.NativeShuffleExchangeExec$$anon$1.write(NativeShuffleExchangeExec.scala:143)
2024-09-23 18:50:23.773 INFO - at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
2024-09-23 18:50:23.773 INFO - at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
2024-09-23 18:50:23.773 INFO - at org.apache.spark.scheduler.Task.run(Task.scala:136)
2024-09-23 18:50:23.774 INFO - at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548)
2024-09-23 18:50:23.774 INFO - at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504)
2024-09-23 18:50:23.774 INFO - at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551)
2024-09-23 18:50:23.774 INFO - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
2024-09-23 18:50:23.774 INFO - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
2024-09-23 18:50:23.774 INFO - at java.lang.Thread.run(Thread.java:748)

I compiled it in aarch64 machine which system page size is 64k.