[core] Use java11 as default

[FangYongs]

Purpose

Linked issue: close #4140

Tests

no

API and Format

no

Documentation

no

[xuzifu666]

@FangYongs Hi, just curious about many user use jdk 8 currently, this update may cause users trouble to fit their env. if wrong can point out. Thanks

[LinMingQiang]

Is jdk 1.8 no longer supported? In my company 1.8 is still the main version.

[xuzifu666]

Is jdk 1.8 no longer supported? In my company 1.8 is still the main version.

+1 with @LinMingQiang，many company use jdk8.

[wwj6591812]

Why not use java-8 as default?

[FangYongs]

Hi @xuzifu666 @LinMingQiang @wwj6591812, JDK 1.8 is already a very early version. Most computing engines have supported JDK 11 and above, and even some new versions of third-party dependencies only support JDK 11. Considering these situations, after discussing with @JingsongLi, we plan to upgrade the default JDK version from 1.8 to 11 starting from the Paimon-1.0. For businesses that rely on 1.8, you can consider upgrading to JDK 11. cc @JingsongLi

[JingsongLi]

Hi @xuzifu666 @LinMingQiang @wwj6591812, JDK 1.8 is already a very early version. Most computing engines have supported JDK 11 and above, and even some new versions of third-party dependencies only support JDK 11. Considering these situations, after discussing with @JingsongLi, we plan to upgrade the default JDK version from 1.8 to 11 starting from the Paimon-1.0. For businesses that rely on 1.8, you can consider upgrading to JDK 11. cc @JingsongLi

Hi @FangYongs , Considering everyone's comments above, I would like to ask what was the original intention behind upgrading to JDK11? What problem do you want to solve?

[FangYongs]

@JingsongLi The original requirement is to use Caffeine3 because Caffeine2 has some problems when evicting data, @Aitozi can provide more context. Currently, Caffeine3 only supports Java 11 and above versions.

[Aitozi]

It may be related to ben-manes/caffeine#1757. I tried upgrading Caffeine before, but the shaded version had the prefix org.apache.paimon.shaded.caffeine3.xxx, which does not work with the current org.apache.paimon.shaded.caffeine2.xxx in the code path.

I suggest upgrading this shaded version locally and collecting performance data for the block cache. Then we can decide whether to upgrade from JDK8 to JDK11.

As for the upgrade process, here is a solution:

We can allow users to build Paimon on either JDK8 or JDK11. To do this, we need to shade paimon-caffeine-3 and paimon-caffeine-2 in the same package path maybe all called org.apache.paimon.shaded.caffeine.xxx ?.

Then choose the appropriate version in Paimon project pom during building.
Regarding lookup hash store format, if Sorted version doesn't work well without caffeine-3 on JDK8, we could disable it optionally.

WDYT? @FangYongs @JingsongLi

[JingsongLi]

@FangYongs @Aitozi Thanks for the information.

Can we have an another solution, such as cherry-pick caffeine's bugfix?

[wwj6591812]

I think we can use guava cache when jave-8, and use Caffeine3 when jave-11.

[JingsongLi]

Let's just use Guava.