+| Title | A new recipe format – part 5 - tests defined in the recipe |
+
| Status | Draft |
+| Author(s) | Wolf Vollprecht <wolf@prefix.dev> |
+| Created | Jun 12, 2024 |
+| Updated | Jun 12, 2024 |
+| Discussion | https://github.com/conda-incubator/ceps/pull/?? |
+| Implementation | https://github.com/prefix-dev/rattler-build |
+
+
+## Abstract
+
+Testing is an important part of the packaging workflow - the goal is to make
+sure that packages work correctly and package the files they are supposed to
+contain.
+
+In the CEP we want to clearly define how tests can be declared in the
+`recipe.yaml` file and how they are stored in the final package.
+
+## History
+
+Conda-build has a (single) test section in the recipes. These tests are rendered
+into the `info/test` folder of the package. The tests are rendered to a
+`run_test.sh` or `run_test.bat` (shell) script, a `run_test.py` or `run_test.pl`
+script for Python or Perl.
+
+An `import` test is added to the `run_test.py` script.
+
+Any additional test-time dependencies are added to a separata
+`test_time_dependencies.json` file in the `info/test` folder. There is only a
+single dependency list for all tests (making cross-compilation tests more
+difficult).
+
+There is also only a single test definition which means it's not easy to run
+multiple (independent) tests (e.g. with different sets of requirements).
+
+## Proposal
+
+The new test section should be more flexible. Test definitions should be well
+separated.
+
+## Tests definition
+
+With the exception of the package content tests, this is a recap of the previous
+CEPs (CEP 14).
+
+The recipes can contain a `tests` section that is a list of individual tests
+that should be executed by the package build program after the package is built.
+
+```yaml
+tests:
+ - script: # following the script definition in the CEP-14
+ - echo "Hello World"
+ requirements:
+ build: # [MatchSpec]
+ - ${{ "qemu" if target_platform != build_platform }}
+ run: # [MatchSpec]
+ - python
+ - pytest # match spec
+ files:
+ # files to be copied to the package for test execution
+ source: # GlobVec
+ - tests/**
+ # Additional files from the recipe directory
+ # recipe:
+ # - ...
+ - python:
+ imports: # [string]
+ - foo # python imports to test
+ pip_check: true # bool
+ - package_contents:
+ include: # [list of globs]
+ - foo.h
+ - foo.hpp
+ - foo/*.hpp
+ - downstream: bar # string
+```
+
+For `script`, `python` and `downstream` tests, please refer to CEP-14. The
+`package_contents` test is a new test type that runs before package creation and
+checks if the files listed are present (or not present) in the package.
+
+## Test rendering to the final package
+
+Tests should be shipped as part of the package in order to facilitate running
+tests outside of the package build execution, and as part of e.g. downstream
+tests.
+
+For this reason, tests should be rendered as part of the package. However, the
+rendered tests should still contain the original test definition including the
+un-evaluated Jinja selectors and if/else sections - in order to facilitate
+running tests in cross-compilation settings.
+
+A build tool SHOULD render the list of tests into a JSON file in the `info/`
+folder of the package. The file should be named `tests.json`. It should contain
+a list of dictionaries, where each dictionary is a test definition.
+
+- Any Jinja or conditionals should be kept intact and evaluated at runtime.
+ (Question: should the list of tests be filtered? Should we introduce a `skip`
+ key instead?)
+- The package content tests are filtered as they are not executed at test-time
+ but at build-time and we can assume that they already passed.
+- The source files for the command tests are copied into the
+ `/etc/conda/test-files/