feat: index type hierarchy in java files #6189
Conversation
kasiaMarek
force-pushed
the
java-type-hierarchy
branch
from
bf01ddf to
31d9c29
Compare
kasiaMarek
force-pushed
the
java-type-hierarchy
branch
from
31d9c29 to
29887d7
Compare
| else | ||
| sourceTopLevels.filter(sym => !isTrivialToplevelSymbol(uri, sym)).toList | ||
| else if (isJava) { | ||
| sourceTopLevels.toList.headOption |
There was a problem hiding this comment.
Do you have any idea if the indexing is becoming much slower?
There was a problem hiding this comment.
Benchmark Mode Cnt Score Error Units
MetalsBench.javaMtagsPackage ss 10 0.168 ± 0.014 s/op
MetalsBench.toplevelJavaMtags ss 10 2.843 ± 0.112 s/op
It does get slower but it doesn't seem to be too bad. In practice for e.g. Metals there is no visible slowdown.
There was a problem hiding this comment.
Ok cool, let's merge it then 🚀
There was a problem hiding this comment.
for e.g. Metals there is no visible slowdown
Actually, I was wrong (I looked at the whole indexed workspace time, which can be very misleading on laptop doing million things in the background).
The actual slowdown:
samples before:
time: indexed library sources in 7.45s
time: indexed library sources in 9.97s
time: indexed library sources in 11s
samples after:
time: indexed library sources in 26s
time: indexed library sources in 22s
time: indexed library sources in 19s
So there is an over 200% slowdown as visible on the CI.
Indexing Java top level doesn't seem to be slower than for Scala but we didn't index Java files almost at all before.
MetalsBench.typeHierarchyIndex ss 10 0.404 ± 0.008 s/op (for Scala lines: 383135)
~ 1.054 for 1m lines
MetalsBench.toplevelJavaMtags ss 10 2.843 ± 0.112 s/op (for Java lines: 5170931)
~ 0.54 for 1m lines
There was a problem hiding this comment.
So the biggest slowdown I think is because we index JDK multiple times on one machine, which doesn't seem necessary. Should we have a separate database for JDKs then?
There was a problem hiding this comment.
It does seem reasonable to index JDK just once instead of doing it for every project.
|
Could you take a look next week if something isn't wrong, the times for some of the CI jobs jumped over 200 %
|
connected to: #5623