[Bug] Unable to async refresh dim table in full cache async lookup join when partition of dim table has been reloaded

[Dd969696]

Search before asking

• I searched in the issues and found nothing similar.

Paimon version

0.9.0

Compute Engine

Flink

Minimal reproduce step

First async lookup join some paimon table in full cache mode with dynamic partition = max_pt(),
then create new partition in paimon table, finally the async task for refreshing local dim table will be rejected because executor thread pool has been terminated.
It will reopen lookupTable when partition has been changed (org.apache.paimon.flink.lookup.FileStoreLookupFunction#tryRefresh)

the executor thread pool for async refreshing local dim table task has been closed (org.apache.paimon.flink.lookup.FullCacheLookupTable#close)

but we won't build new thread pool, hence we will occur exception as follow:

What doesn't meet your expectations?

The executor thread pool for async refreshing local dim table task should be rebuilt and the refreshing task shouldn't be rejected.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!