Look into parallelisation

[tbillington]

Look into parallelisation options such as rayon, crossbeam, or even tokio for parallelisation. Much of what kondo does is able to be done in parallel.

[Noah-Kennedy]

I think the best approach here would likely be to try using tokio or async-std to get asynchronous IO. I might try spinning up a fork with this later this week when I have time.

[tbillington]

Neat :) I'd recommend looking for inspiration in ripgrep (https://github.com/BurntSushi/ripgrep/tree/master/crates/ignore). It does parallel file searching without requiring an entire async runtime and gets better performance for it. Not sure how complex it is though.

[richard-uk1]

Another reason that multithreading would be good: on wayland the compositor expects to be able to ping the UI often. If the UI thread blocks (as is the case with the clean method), then the compositor sometimes kills the app. The solution is to do all work in a separate thread and communicate using messages.

[tbillington]

Ah that's interesting, I didn't know that!

[tbillington]

@derekdreery kondo switched to using multiple threads for the CLI in 1dcc368.

It now uses 3 threads, one for running ahead of the user input that "discovers" the projects, one for presenting them to the user and receiving the users action, and one for cleaning the projects the user has chosen.

They communicate via channels and don't block each other. The only limit I've set is the discover thread will only find up to 5 projects ahead of the UI thread, mostly to avoid unnecessary work if the user exits early.

[tbillington]

I'm open to more parallelism, but since the CLI is now not blocked by discovery or cleaning it feels very snappy to use.

It's also very fast as is for my use case now, so I have less motivation for a solution based on multithreaded async work, but I'm still not opposed to the idea.

[Noah-Kennedy]

FWIW I disagree with my original statement from three years ago, as I know a lot more about the subject now.

If you want to do some sort of async thing, io_uring on linux would allow for more speed on that platform, but other than that you won't see gains from just slapping async onto things.