Performance improvement: only parse files needed by active recipes

[murdos]

Just an idea: parsing POM can be quite slow when you use a number of dependencies, but this parsing is in fact useless if you have no active recipes of Maven category.

[jkschneider]

This is a really good point. I'm kind of embarrassed to say we never really thought of it. Shouldn't be a tough fix?

[murdos]

Indeed, that was a tough fix! 😄
Should the same logic be also applied to the gradle plugin?

[wetneb]

Is this fix still working? I am trying to run OpenRewrite on a Java codebase (https://github.com/OpenRefine/OpenRefine), activating only Java recipes, and OpenRewrite still tries to parse a lot of other files.

To reproduce:

git clone --depth 1 https://github.com/OpenRefine/OpenRefine
cd OpenRefine
mvn -U org.openrewrite.maven:rewrite-maven-plugin:run \
  -Drewrite.activeRecipes=org.openrewrite.java.RemoveUnusedImports

Then, observe that the logs contain the following lines:

[WARNING] There were problems parsing some source files, run with --errors to see full stack traces
[WARNING] There were problems parsing .github/workflows/label_transfer/script.py
[INFO] Project [OpenRefine - main] Parsing source files
[INFO] Parsing as quark /tmp/OpenRefine/main/tests/data/ozone_8hr_dmax.csv as its size 14Mb exceeds size threshold 10Mb

Because the active recipe is a Java recipe, I would not expect the plugin to attempt Python or CSV files.

Alternatively it would be convenient if one could restrict the plugin to a particular list of includes (but I did not find a counterpart to <exclusions/> in the config).

[timtebeek]

Some context also provided in this follow up issue:

• Restrict the rewriting to specific file types #643 (comment)