Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add recipe extension cep #72

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
54 changes: 54 additions & 0 deletions cep-recipe-extensions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Extensions to the recipe spec

This document describes extensions to the recipe specification as merged in cep-14.

## The `build.post_process` section

We became aware that the `conda_build_config.yaml` has ways to specify post-processing steps. These are regex replacements that are performed on any new files.
wolfv marked this conversation as resolved.
Show resolved Hide resolved

Instead of adding these instructions to the `conda_build_config.yaml`, we decided to add a new section to the recipe spec: `build.post_process`. This section is a list of dictionaries, each with the following keys:

- `files`: globs to select files to process
- `regex`: the regex to apply
- `replacement`: the replacement string

The regex specification follows Rust `regex` syntax. Most notably, the replacement string can refer to capture groups with `$1`, `$2`, etc.
This also means that replacement strings need to escape `$` with `$$`.

Internally, we use `replace_all` from the `regex` crate. This means that the regex is applied to the entire file, not line by line.

### Example

```yaml
build:
post_process:
- files:
- "*.txt"
regex: "foo"
replacement: "bar"
- files:
- "*.cmake"
regex: "/sysroot/"
replacement: "$${PREFIX}/"
```

## Globs, positive and negative
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this have to do with recipe post-processing? 🤔

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. This really feels like it should just be incorporated into the build script(s).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The positive/negative globs should be part of the recipe. Using the build scripts just to shuffle files around into separate output is a lot of core for little gain, especially because it's a well-defined operation and doesn't overload the recipe syntax IMO.

C.f. also conda/conda-build#5216

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No this doesn't really have anything to do with post-processing per se. The idea is that any list of globs can also include negative globs.

List of globs appear in multiple places in the recipe.


Following some community discussion we would like to extend the recipe format in such a way that anywhere where a list of globs is accepted, we alternatively accept a dictionary with `include` and `exclude` keys. The values of these keys are lists of globs that are included or excluded respectively.

For example:

```yaml
files:
include:
- "*.txt"
exclude:
- "foo.txt"
```

The evaluation would go as follows:

- first record all matches for the `include` globs
- then remove all matches for the `exclude` globs

In conda-build, this is discussed here: [Link to PR 5216](https://github.com/conda/conda-build/pull/5216)