Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Stamps deterministic #42

Merged
merged 2 commits into from
Apr 10, 2024
Merged

Make Stamps deterministic #42

merged 2 commits into from
Apr 10, 2024

Conversation

jjudd
Copy link

@jjudd jjudd commented Mar 15, 2024

No description provided.

Copy link
Member

@JaredNeil JaredNeil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any performance numbers on this change? I know there was a different change that made things much slower at some point, but I don't remember if it was similar to this in any way.

def sortStampTypeMap(
stampTypeMap: util.Map[String, Schema.Stamps.StampType]
): TreeMap[String, Schema.Stamps.StampType] = {
TreeMap[String, Schema.Stamps.StampType]() ++ stampTypeMap.asScala
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
TreeMap[String, Schema.Stamps.StampType]() ++ stampTypeMap.asScala
TreeMap.from(stampTypeMap.asScala)

Though going from a Java type to a Scala type just to put it ordered map and then convert back to a Java type seems wasteful. Why not go directly to the ordered Java map?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good questions, I don't remember this code was written so long ago. Lemme go take another look. I bet we can go directly to the ordered Java map and that was a rough edge I forgot about last year.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated this to use a java.util.TreeMap directly.

@jjudd
Copy link
Author

jjudd commented Mar 29, 2024

The performance regression last year wasn't caused by the sorting to make things deterministic. It was caused by a change I made in the worker that worked around the Scala 2 compiler tasty classloader caching bug before I knew that bug existed. I don't remember exactly the details of that regression as it's been too long, but I know it wasn't related to the sorting changes like these.

Copy link
Member

@JaredNeil JaredNeil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure would be nice if they just used TreeMap internally from the beginning.

@jjudd jjudd merged commit 5df571d into lucid-master Apr 10, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants