Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build .conda artifacts by default #5527

Merged
merged 10 commits into from
Nov 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 3 additions & 29 deletions conda_build/cli/main_build.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
get_or_merge_config,
zstd_compression_level_default,
)
from ..deprecations import deprecated
from ..utils import LoggingContext
from .actions import KeyValueAction, PackageTypeNormalize
from .main_render import get_render_parser
Expand Down Expand Up @@ -488,19 +487,14 @@ def parse_args(args: Sequence[str] | None) -> tuple[ArgumentParser, Namespace]:
"Do not display value of environment variables specified in build.script_env."
),
)
# TODO: Remove in 25.1
default_pkg_format = context.conda_build.get("pkg_format")
if default_pkg_format is None:
warn_about_default_pkg_format = True
default_pkg_format = conda_pkg_format_default
else:
warn_about_default_pkg_format = False
parser.add_argument(
"--package-format",
dest="conda_pkg_format",
choices=CondaPkgFormat.acceptable(),
action=PackageTypeNormalize,
default=CondaPkgFormat.normalize(default_pkg_format),
default=CondaPkgFormat.normalize(
context.conda_build.get("pkg_format", conda_pkg_format_default)
),
help=(
"Choose which package type(s) are outputted. (Accepted inputs .tar.bz2 or 1, .conda or 2)"
),
Expand All @@ -510,26 +504,6 @@ def parse_args(args: Sequence[str] | None) -> tuple[ArgumentParser, Namespace]:
parsed = parser.parse_args(args)
check_recipe(parsed.recipe)

# TODO: Remove in 25.1
if (
all(not arg.startswith("--package-format") for arg in (args or []))
and warn_about_default_pkg_format
and "purge" not in parsed.recipe
and "purge-all" not in parsed.recipe
):
deprecated.topic(
"24.11",
"25.1",
topic="The default `pkg_format` of '.tar.bz2'",
addendum=(
"\n\n"
"The new default `pkg_format` value will be '.conda'. "
"If you want to keep using `.tar.bz2`, consider:\n"
"- Setting `conda_build.pkg_format: 'tar.bz2' in your condarc file.\n"
"- Using `--pkg-format=tar.bz2` in the CLI.\n"
),
deprecation_type=FutureWarning,
)
return parser, parsed


Expand Down
2 changes: 1 addition & 1 deletion conda_build/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def ext(self):
no_rewrite_stdout_env_default = "false"
ignore_verify_codes_default = []
exit_on_verify_error_default = False
conda_pkg_format_default = CondaPkgFormat.V1
conda_pkg_format_default = CondaPkgFormat.V2
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the main change. Everything else is to fix assumptions about tar.bz2 being always default.

zstd_compression_level_default = 19


Expand Down
7 changes: 6 additions & 1 deletion conda_build/metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -1751,7 +1751,12 @@ def dist(self):
return f"{self.name()}-{self.version()}-{self.build_id()}"

def pkg_fn(self):
return f"{self.dist()}.tar.bz2"
ext = (
CondaPkgFormat.V2.ext
if self.config.conda_pkg_format == CondaPkgFormat.V2
else CondaPkgFormat.V1.ext
)
return f"{self.dist()}{ext}"

def is_app(self):
return bool(self.get_value("app/entry"))
Expand Down
6 changes: 5 additions & 1 deletion conda_build/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -1141,10 +1141,14 @@ def package_has_file(package_path, file_path, refresh_mode="modified"):
conda_package_handling.api.extract(
package_path, dest_dir=td, components="info"
)
else:
elif package_path.endswith(".tar.bz2"):
conda_package_handling.api.extract(
package_path, dest_dir=td, components=file_path
)
else:
conda_package_handling.api.extract(
package_path, dest_dir=td, components="pkg"
)
resolved_file_path = os.path.join(td, file_path)
if os.path.exists(resolved_file_path):
# TODO :: Remove this text-mode load. Files are binary.
Expand Down
21 changes: 20 additions & 1 deletion docs/source/concepts/generating-index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,26 @@ Parts of a channel
"version": "0.1.0"
},
...
},
"packages.conda": {
"super-fun-package-0.2.0-py310_0.conda": {
"build": "py37_0",
"build_number": 0,
"depends": [
"some-depends"
],
"license": "BSD",
"md5": "a75683f8d9f5b58c19a8ec5d0b7f796e",
"name": "super-fun-package",
"sha256": "e39029d601b9f493ea05c37a2630a9fe5810f1fe3c3f4250e51886838e8e0287",
"size": 4125,
"subdir": "win-64",
"timestamp": 1530731987654,
"version": "0.2.0"
},
...
}
}
How an index is generated
-------------------------
Expand Down Expand Up @@ -218,7 +237,7 @@ already extracted can save a lot of time in fully re-creating the index, should
that be necessary.
An aside: one design goal of the ``.conda`` package format was to make indexing as
fast as possible. To achieve this, the .conda format separates metadata from the
fast as possible. To achieve this, the ``.conda`` format separates metadata from the
actual package contents. Where the old ``.tar.bz2`` container required extracting
the entire package to obtain the metadata, the new package format allows
extraction of metadata without touching the package contents. This allows
Expand Down
4 changes: 2 additions & 2 deletions docs/source/concepts/package-naming-conv.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ The following figure compares a canonical name to a filename:
|
Conda supports both ``.conda`` and ``.tar.bz2`` package extensions. The ``.conda``
format is generally smaller and more efficient than ``.tar.bz2`` packages.
Read our `blog post`_ about it to learn more.
format (default since 25.1) is generally smaller and more efficient than ``.tar.bz2``
packages. Read our `blog post`_ about it to learn more.

The build string is created as the package is built. Things that
contribute to it are the variants specified either by the command
Expand Down
4 changes: 2 additions & 2 deletions docs/source/resources/package-spec.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ There are currently two formats of archives that are supported:
- **Description**

* - .tar.bz2
- The original format of conda packages. Is the default output of conda-build.
- The original format of conda packages.
* - .conda
- 2nd Gen. This is a more compact and thus faster. Can be outputed from conda-build by setting output in ``.condarc`` file.
- 2nd Gen. This is a more compact and thus faster. Default since 25.1.

The formats are identical across platforms and operating systems.
During the install process, all files are extracted into the
Expand Down
19 changes: 19 additions & 0 deletions news/5527-dot-conda-default
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
### Enhancements

* Make `.conda` the new default package format. `.tar.bz2` files can still be generated with `--package-format=1` and/or `conda_build.pkg_format: 1` in your `.condarc` file. (#5183 via #5527)

### Bug fixes

* <news item>

### Deprecations

* <news item>

### Docs

* <news item>

### Other

* <news item>
Loading
Loading