Skip to content

Commit

Permalink
Build .conda artifacts by default (#5527)
Browse files Browse the repository at this point in the history
* Build .conda artifacts by default

* revert deprecation warnings

* amend test

* add news

* pre-commit

* autodetect correct extension for filename

* force tar bz2 here

* update docs
  • Loading branch information
jaimergp authored Nov 27, 2024
1 parent 4260236 commit 3cf75b6
Show file tree
Hide file tree
Showing 17 changed files with 116 additions and 106 deletions.
32 changes: 3 additions & 29 deletions conda_build/cli/main_build.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@
get_or_merge_config,
zstd_compression_level_default,
)
from ..deprecations import deprecated
from ..utils import LoggingContext
from .actions import KeyValueAction, PackageTypeNormalize
from .main_render import get_render_parser
Expand Down Expand Up @@ -488,19 +487,14 @@ def parse_args(args: Sequence[str] | None) -> tuple[ArgumentParser, Namespace]:
"Do not display value of environment variables specified in build.script_env."
),
)
# TODO: Remove in 25.1
default_pkg_format = context.conda_build.get("pkg_format")
if default_pkg_format is None:
warn_about_default_pkg_format = True
default_pkg_format = conda_pkg_format_default
else:
warn_about_default_pkg_format = False
parser.add_argument(
"--package-format",
dest="conda_pkg_format",
choices=CondaPkgFormat.acceptable(),
action=PackageTypeNormalize,
default=CondaPkgFormat.normalize(default_pkg_format),
default=CondaPkgFormat.normalize(
context.conda_build.get("pkg_format", conda_pkg_format_default)
),
help=(
"Choose which package type(s) are outputted. (Accepted inputs .tar.bz2 or 1, .conda or 2)"
),
Expand All @@ -510,26 +504,6 @@ def parse_args(args: Sequence[str] | None) -> tuple[ArgumentParser, Namespace]:
parsed = parser.parse_args(args)
check_recipe(parsed.recipe)

# TODO: Remove in 25.1
if (
all(not arg.startswith("--package-format") for arg in (args or []))
and warn_about_default_pkg_format
and "purge" not in parsed.recipe
and "purge-all" not in parsed.recipe
):
deprecated.topic(
"24.11",
"25.1",
topic="The default `pkg_format` of '.tar.bz2'",
addendum=(
"\n\n"
"The new default `pkg_format` value will be '.conda'. "
"If you want to keep using `.tar.bz2`, consider:\n"
"- Setting `conda_build.pkg_format: 'tar.bz2' in your condarc file.\n"
"- Using `--pkg-format=tar.bz2` in the CLI.\n"
),
deprecation_type=FutureWarning,
)
return parser, parsed


Expand Down
2 changes: 1 addition & 1 deletion conda_build/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ def ext(self):
no_rewrite_stdout_env_default = "false"
ignore_verify_codes_default = []
exit_on_verify_error_default = False
conda_pkg_format_default = CondaPkgFormat.V1
conda_pkg_format_default = CondaPkgFormat.V2
zstd_compression_level_default = 19


Expand Down
7 changes: 6 additions & 1 deletion conda_build/metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -1767,7 +1767,12 @@ def dist(self):
return f"{self.name()}-{self.version()}-{self.build_id()}"

def pkg_fn(self):
return f"{self.dist()}.tar.bz2"
ext = (
CondaPkgFormat.V2.ext
if self.config.conda_pkg_format == CondaPkgFormat.V2
else CondaPkgFormat.V1.ext
)
return f"{self.dist()}{ext}"

def is_app(self):
return bool(self.get_value("app/entry"))
Expand Down
6 changes: 5 additions & 1 deletion conda_build/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -1141,10 +1141,14 @@ def package_has_file(package_path, file_path, refresh_mode="modified"):
conda_package_handling.api.extract(
package_path, dest_dir=td, components="info"
)
else:
elif package_path.endswith(".tar.bz2"):
conda_package_handling.api.extract(
package_path, dest_dir=td, components=file_path
)
else:
conda_package_handling.api.extract(
package_path, dest_dir=td, components="pkg"
)
resolved_file_path = os.path.join(td, file_path)
if os.path.exists(resolved_file_path):
# TODO :: Remove this text-mode load. Files are binary.
Expand Down
21 changes: 20 additions & 1 deletion docs/source/concepts/generating-index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,26 @@ Parts of a channel
"version": "0.1.0"
},
...
},
"packages.conda": {
"super-fun-package-0.2.0-py310_0.conda": {
"build": "py37_0",
"build_number": 0,
"depends": [
"some-depends"
],
"license": "BSD",
"md5": "a75683f8d9f5b58c19a8ec5d0b7f796e",
"name": "super-fun-package",
"sha256": "e39029d601b9f493ea05c37a2630a9fe5810f1fe3c3f4250e51886838e8e0287",
"size": 4125,
"subdir": "win-64",
"timestamp": 1530731987654,
"version": "0.2.0"
},
...
}
}
How an index is generated
-------------------------
Expand Down Expand Up @@ -218,7 +237,7 @@ already extracted can save a lot of time in fully re-creating the index, should
that be necessary.
An aside: one design goal of the ``.conda`` package format was to make indexing as
fast as possible. To achieve this, the .conda format separates metadata from the
fast as possible. To achieve this, the ``.conda`` format separates metadata from the
actual package contents. Where the old ``.tar.bz2`` container required extracting
the entire package to obtain the metadata, the new package format allows
extraction of metadata without touching the package contents. This allows
Expand Down
4 changes: 2 additions & 2 deletions docs/source/concepts/package-naming-conv.rst
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ The following figure compares a canonical name to a filename:
|
Conda supports both ``.conda`` and ``.tar.bz2`` package extensions. The ``.conda``
format is generally smaller and more efficient than ``.tar.bz2`` packages.
Read our `blog post`_ about it to learn more.
format (default since 25.1) is generally smaller and more efficient than ``.tar.bz2``
packages. Read our `blog post`_ about it to learn more.

The build string is created as the package is built. Things that
contribute to it are the variants specified either by the command
Expand Down
4 changes: 2 additions & 2 deletions docs/source/resources/package-spec.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ There are currently two formats of archives that are supported:
- **Description**

* - .tar.bz2
- The original format of conda packages. Is the default output of conda-build.
- The original format of conda packages.
* - .conda
- 2nd Gen. This is a more compact and thus faster. Can be outputed from conda-build by setting output in ``.condarc`` file.
- 2nd Gen. This is a more compact and thus faster. Default since 25.1.

The formats are identical across platforms and operating systems.
During the install process, all files are extracted into the
Expand Down
19 changes: 19 additions & 0 deletions news/5527-dot-conda-default
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
### Enhancements

* Make `.conda` the new default package format. `.tar.bz2` files can still be generated with `--package-format=1` and/or `conda_build.pkg_format: 1` in your `.condarc` file. (#5183 via #5527)

### Bug fixes

* <news item>

### Deprecations

* <news item>

### Docs

* <news item>

### Other

* <news item>
Loading

0 comments on commit 3cf75b6

Please sign in to comment.