What is the best workflow for developing python packages and publishing via Conda?

[jacksonwb]

Checklist

• I added a descriptive title
• I searched open requests and couldn't find a duplicate

What is the idea?

I'm trying to understand the most ergonomic way to develop a python package for the conda ecosystem and wanted to get a better understanding of the workflows that people use.

Having done plenty of development in the pypi / pip ecosystem, the pattern I am familiar with is:

• define a pyproject.toml package definition
• install the package in development mode
• build the package

This flow allows a single place where dependencies are defined, and makes development easy as builds are not needed when only editing python files in the source tree to test in an env.

What is the equivalent flow when packaging with conda. As best I can tell there are no natural ways to do this. conda develop doesn't seem like a viable option and a number of other tools in this area seem to fragile. The only similar flow I can think of is:

• Define a pyproject.toml and follow the above flow for development
• Then create a meta.yaml conda package and manually find the conda equivalent of the deps
• Build the package

Is this how people are developing with conda? Are there tools to automate the dep mapping / duplication?
Are there better patterns I am missing?

Apologies if this is the wrong place for this discourse. Happy to post elsewhere.

Why is this needed?

I am sure that there are others with similar confusion and this discourse will help the community.

What should happen?

No response

Additional Context

No response

[travishathaway]

Hi @jacksonwb,

Thanks for asking this question. Just FYI, we also have a Discourse server that you could post such topics too, but this is also an appropriate place for it as it is related to the conda build project.

The workflow you define is more or less the correct way to develop a Python package for the conda ecosystem. We do have a tool available for automatically generating recipes from PyPI projects. It's called Grayskull. From my own personal experience, Grayskull gets you about 95% of the way there when dealing with a pure Python dependency (i.e. Python packages that have no C, C++, etc. dependencies).

If you want to see a wide variety of community contributed packages, I would recommend checking out conda-forge. Conda-forge also has a nice guide for contributing packages where you can learn more: https://conda-forge.org/docs/maintainer/adding_pkgs.html

Some of the things mentioned in the conda-forge documentation is specific to their package host architecture, but there are also many things mentioned there that apply to all conda packages.

With that being said, there are currently those in the community who are aiming to provide a workflow that only requires a single file. That is being done in the Pixi project. While support for this feature doesn't currently exist, I do know that it is on their roadmap, so keep an eye on that project too.

Hope this was helpful.

[github-actions]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   - What OS and version you reproduced the issue on
   - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!