Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImageConfigParser: Don't sort merged values #170

Merged
merged 1 commit into from
Nov 25, 2024
Merged

Conversation

dbnicholson
Copy link
Member

The order of configuration values may be significant, so the parser should not sort them. To do that, use a single counter for add and del variants and emit the values that have positive counts without sorting. This relies on the Python dict stable order, which has been the case since Python 3.6.

I've done no real world testing of this. It should also be stated that the order you get is "first merged section wins", so it's not really reliable. For example:

[section]
# From product config
option_add_product = foo
# From local product config
option_add_local_product = bar

You'd get foo\nbar even though you might really prefer bar\nfoo from your local setting. With explicit sorting you'd always get the same values regardless of what order the unmerged options came in.

@wjt
Copy link
Member

wjt commented Nov 22, 2024

I'll look more closely at this on Monday.

At a first glance it looks very neat, and it solves part of the problem discussed on #169 – without this change, if we make the Kolibri channels field mergeable then it's impossible to preserve order.

The harder part is that in the derived-personality case, you might want (e.g.) en_GB.ini to add 2 more channels to the set provided by en.ini, but you may not want them to go after all the channels from en.ini. I think the way you would deal with that is to first _del all the inherited channels (removing them from the counter), then explicitly add them all back along with the 2 extra ones. Or explicitly set the field (which I think overrides the merging? or could be made to). Not ideal but it's not (substantially) worse than what we have now.

@dbnicholson
Copy link
Member Author

The harder part is that in the derived-personality case, you might want (e.g.) en_GB.ini to add 2 more channels to the set provided by en.ini, but you may not want them to go after all the channels from en.ini. I think the way you would deal with that is to first _del all the inherited channels (removing them from the counter), then explicitly add them all back along with the 2 extra ones. Or explicitly set the field (which I think overrides the merging? or could be made to). Not ideal but it's not (substantially) worse than what we have now.

Indeed, it's currently not possible to specify the desired semantics of the configuration, but it's also pretty hard to implement them. You can always fully override at any time by using the unsuffixed version, but that's probably not what you want in most cases. Fortunately, this is a part of the code that has tests, so you can throw whatever crazy stuff in there and check the behavior.

The order of configuration values may be significant, so the parser
should not sort them. To do that, use a single counter for add and del
variants and emit the values that have positive counts without sorting.
This relies on the Python dict stable order, which has been the case
since Python 3.6.
@wjt wjt force-pushed the unsorted-option-merge branch from a8cb798 to 59980bc Compare November 25, 2024 09:42
@wjt
Copy link
Member

wjt commented Nov 25, 2024

I rebased this backwards so that it can be merged onto eos6.0.

# Merge the values together with newlines like they were
# in the original configuration.
vals = add_vals - del_vals
sect[option] = '\n'.join(sorted(vals.keys()))
sect[option] = '\n'.join(k for k, v in values.items() if v > 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was curious whether Counter provides a shorter way to spell “only keys that have positive values” and it does:

In [7]: values = Counter(foo=3, bar=-1)

In [8]: +values
Out[8]: Counter({'foo': 3})

In [9]: '\n'.join(+values)
Out[9]: 'foo'

So we could write:

Suggested change
sect[option] = '\n'.join(k for k, v in values.items() if v > 0)
sect[option] = '\n'.join(+values)

But I've never seen this before and I think it's too obscure.

@wjt wjt merged commit 057c13c into master Nov 25, 2024
2 checks passed
@wjt wjt deleted the unsorted-option-merge branch November 25, 2024 11:48
wjt added a commit that referenced this pull request Nov 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants