Run Kotlinter in process isolation

[jeremymailen]

First in a series of changes leading to a 5.0.0 release. Starting with Kotlin 2.1.0 it's no longer acceptable to run ktlint and the embedded Kotlin compiler in the main Gradle classloader. The solution is to use Gradle worker process isolation.

Positive consequences:

• Users can configure the version of ktlint
• Kotlinter is compatible with any version of Kotlin with matching language specification

Side effects to watch:

• Performance may be slower

[jeremymailen]

Tested it on a substantial multi-module project with command gradle clean lintKotlin:

• Before: 13s
• After: 25s

So it will be slower, but this is the cost of isolation. Incremental of course is still fast and under 1s because generally not many files change between runs.

[jeremymailen]

We do experience the classloader leak bug however.
If I configure gradle.properties:

org.gradle.jvmargs=-Xmx1024m "-XX:MaxMetaspaceSize=1024m"

This helps quite a bit, but even then if you repeat clean lintKotlin several times it will run the gradle daemon out of metaspace. It may reclaim memory, but not aggressively enough.

@osipxd you seem to have some insight here. Is the workaround to use process isolation instead of classloader? It seems like we could run into different problems there.

[osipxd]

I haven't investigated possible workarounds deeply myself. But I've seen another possible workaround by @martinbonnin:

• [gradle-plugin] Manage our classloaders manually apollographql/apollo-kotlin#6305

[jeremymailen]

Tested it on a substantial multi-module project with command gradle clean lintKotlin:

• Before: 13s
• After: 25s

So it will be slower, but this is the cost of isolation. Incremental of course is still fast and under 1s because generally not many files change between runs.

Switching to process isolation yields:

• After: 19s

So it's actually better than classloader isolation, but still a small performance hit.

[jeremymailen]

I haven't investigated possible workarounds deeply myself. But I've seen another possible workaround by @martinbonnin:

• [gradle-plugin] Manage our classloaders manually apollographql/apollo-kotlin#6305

Thanks for the quick response. It actually looks like process isolation is not bad and it fixes the issue.

[martinbonnin]

FWIW, we went the process isolation route before but that created other questions like how many processes can you run in parallel? What heap size to allocate in each process, etc... It can work (and from the issues linked in gradle/gradle#18313, looks like this is definitely a solution) but it's no silver bullet either.

[martinbonnin]

FYI, there is a pull request to fix this 🎉 : gradle/gradle#31654