Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: make Kind a lot smaller #82

Closed
wants to merge 2 commits into from
Closed

refactor: make Kind a lot smaller #82

wants to merge 2 commits into from

Conversation

Eh2406
Copy link
Member

@Eh2406 Eh2406 commented Apr 6, 2021

This is two changes that make Kind a lot smaller. That intern makes Incompatibility significantly smaller. And as the incompatibility_store is a large portion of memory usage, this reduces the total memory usage significantly.

The trick is that data that is going into the package_terms does not need to be kept in the Kind.

Using dhat-rs I am seeing the elm file go from:

dhat: Total:     352,307,668 bytes in 412,858 blocks
dhat: At t-gmax: 118,080 bytes in 58 blocks

to:

dhat: Total:     252,962,988 bytes in 412,852 blocks
dhat: At t-gmax: 79,168 bytes in 58 blocks

So a ~30% reduction.
Unfortunately that did not come with a speedup, but what can you do.

Copy link
Member

@mpizenberg mpizenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's funny cause it's almost the reverse commit of "Add more information in incompatibility::Kind " that I did back in September last year :)

My reasons were mainly that I found it made the code clearer, but I had no concern about memory at that time. I think both approaches can be justified. If you think it's worth it I trust you!

@Eh2406
Copy link
Member Author

Eh2406 commented Apr 17, 2021

That is funny! The code clarity point is a valid one.
Most of the savings come from removing the Range<V> witch are like 8 * |V| + 5 bytes, and so adds up fast. I wonder if there is a something with most of the readability and most of the mem savings.

Also, with this not helping with perf, I would be Ok with keeping this in our back pocket until we have a memory constrained user.

I'm going to think about it some more before taking any action.

@Eh2406
Copy link
Member Author

Eh2406 commented Apr 23, 2021

I think most of the confusion comes from destructuring the SmallVec. So here is a version witch only removes the Range<V> parts, and uses accessor to get things out of the SmallVec. What do you think? Is this clearer?

This, something else, or leave if for now? Whatever you think.

Copy link
Member

@mpizenberg mpizenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a big fan of keeping only parts of the attributes to be honest. I think it may make the code confusing which is worst than having more code.

Also, somewhat related to this PR, I think we don't have property tests checking that a derivation tree is a valid proof of conflict. Saying this because I didn't check that the code building the derivation tree is valid.

@@ -43,28 +43,25 @@ enum Kind<P: Package, V: Version> {
/// Initial incompatibility aiming at picking the root package for the first decision.
NotRoot(P, V),
/// There are no versions in the given range for this package.
NoVersions(P, Range<V>),
NoVersions(P),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think things like NoVersion(P) might imply that there is no version at all for package P while actually that's limited to a given range.

impl<K: PartialEq + Eq + Hash, V> Index<&K> for SmallMap<K, V> {
type Output = V;
fn index(&self, key: &K) -> &V {
&self.get(key).unwrap()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading terms[package] is nice but probably treacherous. I think it would be better if we could avoid this. Then the caller is aware of when they are calling unwrap().

@@ -71,6 +71,14 @@ impl<V: Version> Term<V> {
_ => panic!("Negative term cannot unwrap positive range"),
}
}

/// Unwrap the range contains in the term.
pub(crate) fn as_range(&self) -> &Range<V> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a big fan of this function either. as_range on a term feels like it should be negated for a Negative(range). But a term can never be converted into a range without loosing on the semantics. So I'd probably go for a name such as unwrap() or similar, meaning that some information is lost. Maybe unwrap_range or extract_range.

@Eh2406
Copy link
Member Author

Eh2406 commented Apr 25, 2021

I'm not a big fan of keeping only parts of the attributes to be honest. I think it may make the code confusing which is worst than having more code.

That is a good point.
At this point having tried a bunch of variations, I don't think any of them are worth the code complexity. If we have a real user that is complaining about our memory usage, we can reconsider then.

@Eh2406 Eh2406 closed this Apr 25, 2021
@Eh2406 Eh2406 deleted the small-kind branch April 27, 2021 21:19
@Eh2406
Copy link
Member Author

Eh2406 commented Dec 7, 2023

In #157 (comment) reported large improvements from reducing the calls to V.clone(), which suggests that at some point this optimization will be helpful to them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants