(🎁) Automatically sync settings with config file

[KotlinIsland]

Attention

Use ruff instead of black + blackconnect, it's better in every way. (plugin)

Original issue

I think it's pretty silly to be specifying the exact same options twice, I think it would be optimal if BlackConnect supported automatic syncing from the pyproject.toml file, and that this is the default option.

• Related (🎁) Save config options to a vcs shareable place #44

[flying-sheep]

Without this and #44, this plugin is strictly less useful to me than simply adding a file watcher running pre-commit run --files $FilePath$ on save. I have too many projects and dev environments to manually fiddle with the setting each time.

With my global file watcher, I go into each project, check the checkmark, and commit watcherTasks.xml to git.

[lensvol]

I am sorry to hear that. To help me better understand your use case, please clarify the following: which specific setting do you struggle with and why do you change it so often?

[KotlinIsland]

The settings in the config file rarely change, but when they do it is very frustrating to forget to also sync the BlackConnect settings (for every single person that is working on the project). Although more of an issue is initial setup. With a file watcher solution one can easily commit the config file and never have to think of it again, but with BlackConnect every time someone clones the project they also have to deal with configuring the entirely redundant settings in BlackConnect.

[flying-sheep]

Black’s string normalization, line length, magic comma, exclude, include, … settings for any Python project are defined in pyproject.toml. That’s where black and blackd read them from. If I had coded this plugin, I wouldn’t even have thought about adding those settings to its UI unless that UI was just a graphical way to write the settings to pyproject.toml

[KotlinIsland]

Please note blackd will not use pyproject.toml configuration.

https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#where-black-looks-for-the-file

While that's not entirely true, I do agree with you about the settings UI.

[lensvol]

As mentioned above, blackd does not care about your pyproject.toml_ at all.

Moreover, the plugin itself was initially created to satisfy use cases that came from my own experience and those of my colleagues. Many of those projects we had did not use pyproject.toml at all! To accommodate more expansive range of use cases, an explicit "Settings" screen was used which gave individual users the ability to tailor things to their liking. To complement that, "Load from pyproject.toml" button was also added and it seems enough for 99% percent of users.

I see merit in the suggestion of automatically trying to load settings from pyproject.toml if it is present and show a pop-up balloon with a suggestion to sync them in case any changes are detected. Will that change solve your issues with the initial setup?

[flying-sheep]

Yeah, that sounds reasonable!

Many of those projects we had did not use pyproject.toml at all!

That’s very surprising to me. The standard way to create Python packages since like 6 years involves specifying your build system in the [build-system] table in pyproject.toml. Any Python package project without a pyproject.toml has maintainers that haven’t interacted with the wider Python community in the last 6 years.

Granted, the official packaging tutorial has only been updated recently.

[lensvol]

Well, that may seem surprising at first, but there exists a whole "terra incognita" of in-house Python projects, some of which alive for 10+ years and have their very special environment setups. Also, it is worth mentioning that "Python package" is not the default method of distributing your code in those environments.

This plugin tries to be as conservative and non-opinionated as possible for the reasons outlined above.

[flying-sheep]

Also, it is worth mentioning that "Python package" is not the default method of distributing your code in those environments.

In some cases, sure. If people like reinventing the wheel. Companies I’ve been at try to utilize existing infrastructure.

[Zeckie]

I see merit in the suggestion of automatically trying to load settings from pyproject.toml if it is present and show a pop-up balloon with a suggestion to sync them in case any changes are detected. Will that change solve your issues with the initial setup?

In projects that contain a pyproject.toml file with black settings, couldn't (and shouldn't) it just use those settings, rather than have to maintain separate sets of settings and sync them? In what situations would it make sense to have them being different?

[lensvol]

While this seems like a rad suggestion, sadly in that case you cannot escape from dealing with two separate sets of settings:

• In-memory representation of plugin's settings.
• On-disk set of black code style settings in pyproject.toml.

Both of those sets can be modified independently of each other, which presents at least a couple of problems for transparent sync:

• In-memory representation gets changed, but write to disk fails for whatever reason. What now?
• pyproject.toml gets changed (e.g. repo is updated, user updates it by hand, some other tool makes changes) and IDE fails to pick up the changes (VFS updates are async) in time for the next call to blackd. What now?

Workarounds can be implemented for all of these cases, but it will quickly get too burdensome and complicated for what I see mostly as a niche feature.

At the moment, I plan to implement the following features:

• Pop-up balloon on IDE start for syncing code style settings with pyproject.toml if there is a difference.
• File system watcher that looks for changes to pyproject.toml and asks user if those changes should also be applied to IDE.

Given that changes to code style in the project are usually infrequent this should be enough to help your use case.

Hope that helps.

[Zeckie]

While this seems like a rad suggestion, sadly in that case you cannot escape from dealing with two separate sets of settings:

• In-memory representation of plugin's settings.
• On-disk set of black code style settings in pyproject.toml.

Both of those sets can be modified independently of each other, which presents at least a couple of problems for transparent sync:

I am proposing that in projects that contain pyproject.toml with black settings ([tool.black] section), that blackconnect will use those settings instead of the formatting settings configured in the IDE settings. If the user wants to change the settings for that project, they edit pyproject.toml. If they have multiple projects, each project with black settings in pyproject.toml will use those settings, and any without them will use the configured settings.

• In-memory representation gets changed, but write to disk fails for whatever reason. What now?

I'm not clear which file you are suggesting that it is writing to here, are you able to clarify? I do not see why it would need to write the settings loaded from pyproject.toml to another file.

• pyproject.toml gets changed (e.g. repo is updated, user updates it by hand, some other tool makes changes) and IDE fails to pick up the changes (VFS updates are async) in time for the next call to blackd. What now?

If the IDE has not noticed the changes, will it also have an old version of the source code being edited? Would it have to manage the conflicts from the source code changing on disk, in the IDE, and from blackd?
If the user is editing a python file while updating to a new version of pyproject.toml, I think it would be understandable for blackconnect to use a slightly outdated cache of the settings. Most likely, whatever caused the file to change would have been triggered by the user, and as long as either blackconnect notices the change quickly, or there is some indication in the IDE that there are background tasks running, the user would understand that it did not take effect immediately.

For example, if I am working on the black project, which (at the time of writing) contains the following in pyproject.toml:

[tool.black]
line-length = 88
target-version = ['py37', 'py38']
include = '\.pyi?$'
extend-exclude = '''
/(
  # The following are specific to Black, you probably don't want those.
  | blib2to3
  | tests/data
  | profiling
)/
'''
# We use preview style for formatting Black itself. If you
# want stable formatting across releases, you should keep
# this off.
preview = true

Would it really make sense for my blackconnect settings (say I'd configured line length to be 100) to be used instead of what was in pyproject.toml?

Workarounds can be implemented for all of these cases, but it will quickly get too burdensome and complicated for what I see mostly as a niche feature.

How is this a niche feature? Wouldn't it be useful for most people working on open source python projects?

At the moment, I plan to implement the following features:

• Pop-up balloon on IDE start for syncing code style settings with pyproject.toml if there is a difference.
• File system watcher that looks for changes to pyproject.toml and asks user if those changes should also be applied to IDE.

Given that changes to code style in the project are usually infrequent this should be enough to help your use case.

Yes, those would be helpful, but I think just using the settings from pyproject.toml would be better, to avoid using a user's personal choice of formatting instead of the standard formatting across the project for all users.

It would make sense to show a note to the user in the blackconnect settings like:

Settings in a [tool.black] section of pyproject.toml will override settings here

I'd be happy with this always applying, but I could understand if some users wanted to be able to configure this, with a choice like:

• Settings from pyproject.toml override settings configured in IDE settings (my proposal)
• Prompt to sync settings from pyproject.toml
• Always use settings configured in IDE settings (current behavior)

[DetachHead]

coming from vscode where the black plugin reads the settings from pyproject.toml, it seems really inconvenient to have to manually maintain the config separately. ideally when opening a project you want to have to manually configure as little as possible, so the more config that can be loaded from committed config files the better

[eirnym]

Any news?

[KotlinIsland]

@eirnym PyCharm comes with black support built-in, or you could use ruff format, which has an excellent plugin

[eirnym]

And with builtin black plugin you basically can't create modules like json.py under any package

[KotlinIsland]

And with builtin black plugin you basically can't create modules like json.py under any package

Ah yes, https://youtrack.jetbrains.com/issue/PY-61699/Black-integration-failed-to-format-file-when-package-contains-stdlib-name-like-logging

Yeah, use ruff, it's better in every way. (plugin)

[eirnym]

Thank you, I'll use ruff