Skip to content

Commit

Permalink
add support for multi_env installers (#509)
Browse files Browse the repository at this point in the history
* add support for multi_env installers (WIP)

* make sure conda is in base

* docs and validation

* fix api

* add for osx-pkg too

* allow per-env channels, environment and environment_file

* update docs / schema checks

* update examples

* fix api call

* fix example

* better error msg

* default is None, not ()

* fix some extra_envs assumptions

* support for extra_envs.exclude; restore global menu_packages support; clarify channels(_remap) handling

* do not remove tk on windows example

* add tests

* disable exit on error temporarily for this test

* rework example and tests; exclude is buggy

* test was wrong :D

* better tests for dav1d
  • Loading branch information
jaimergp authored Aug 12, 2022
1 parent 8426b02 commit 31e1947
Show file tree
Hide file tree
Showing 15 changed files with 525 additions and 114 deletions.
30 changes: 29 additions & 1 deletion CONSTRUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ Path to an environment file to construct from. If this option is present, the
create a temporary environment, constructor will build and installer from
that, and the temporary environment will be removed. This ensures that
constructor is using the precise local conda configuration to discover
and install the packages.
and install the packages. The created environment MUST include `python`.

## `transmute_file_type`

Expand All @@ -158,6 +158,34 @@ _type:_ string<br/>
The channel alias that would be assumed for the created installer
(only useful if it includes conda).

## `extra_envs`

_required:_ no<br/>
_type:_ dictionary<br/>
Create more environments in addition to the default `base` provided by `specs`,
`environment` or `environment_file`. This should be a map of `str` (environment
name) to a dictionary of options:
- `specs` (list of str): which packages to install in that environment
- `environment` (str): same as global option, for this env
- `environment_file` (str): same as global option, for this env
- `channels` (list of str): using these channels; if not provided, the global
value is used. To override inheritance, set it to an empty list.
- `channels_remap` (list of str): same as global option, for this env;
if not provided, the global value is used. To override inheritance, set it to
an empty list.
- `user_requested_specs` (list of str): same as the global option, but for this env;
if not provided, global value is _not_ used

Notes:
- `ignore_duplicate_files` will always be considered `True` if `extra_envs` is in use.
- `conda` needs to be present in the `base` environment (via `specs`)
- support for `menu_packages` is planned, but not possible right now. For now, all packages
in an `extra_envs` config will be allowed to create their shortcuts.
- If a global `exclude` option is used, it will have an effect on the environments created
by `extra_envs` too. For example, if the global environment excludes `tk`, none of the
extra environmentss will have it either. Unlike the global option, an error will not be
thrown if the excluded package is not found in the packages required by the extra environment.

## `installer_filename`

_required:_ no<br/>
Expand Down
58 changes: 57 additions & 1 deletion constructor/construct.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@
create a temporary environment, constructor will build and installer from
that, and the temporary environment will be removed. This ensures that
constructor is using the precise local conda configuration to discover
and install the packages.
and install the packages. The created environment MUST include `python`.
'''),

('transmute_file_type', False, str, '''
Expand All @@ -124,6 +124,32 @@
(only useful if it includes conda).
'''),

('extra_envs', False, (dict,), '''
Create more environments in addition to the default `base` provided by `specs`,
`environment` or `environment_file`. This should be a map of `str` (environment
name) to a dictionary of options:
- `specs` (list of str): which packages to install in that environment
- `environment` (str): same as global option, for this env
- `environment_file` (str): same as global option, for this env
- `channels` (list of str): using these channels; if not provided, the global
value is used. To override inheritance, set it to an empty list.
- `channels_remap` (list of str): same as global option, for this env;
if not provided, the global value is used. To override inheritance, set it to
an empty list.
- `user_requested_specs` (list of str): same as the global option, but for this env;
if not provided, global value is _not_ used
Notes:
- `ignore_duplicate_files` will always be considered `True` if `extra_envs` is in use.
- `conda` needs to be present in the `base` environment (via `specs`)
- support for `menu_packages` is planned, but not possible right now. For now, all packages
in an `extra_envs` config will be allowed to create their shortcuts.
- If a global `exclude` option is used, it will have an effect on the environments created
by `extra_envs` too. For example, if the global environment excludes `tk`, none of the
extra environmentss will have it either. Unlike the global option, an error will not be
thrown if the excluded package is not found in the packages required by the extra environment.
'''),

('installer_filename', False, str, '''
The filename of the installer being created. If not supplied, a reasonable
default will determined by the `name`, `version`, platform, and installer type.
Expand Down Expand Up @@ -425,6 +451,20 @@
]


_EXTRA_ENVS_SCHEMA = {
"specs": (list, tuple),
"environment": (str,),
"environment_file": (str,),
"channels": (list, tuple),
"channels_remap": (list, tuple),
"user_requested_specs": (list, tuple),
"exclude": (list, tuple),
# TODO: we can't support menu_packages for extra envs yet
# will implement when the PR for new menuinst lands
# "menu_packages": (list, tuple),
}


def ns_platform(platform):
p = platform
return dict(
Expand Down Expand Up @@ -557,6 +597,22 @@ def verify(info):
if not pat.match(value) or value.endswith(('.', '-')):
sys.exit("Error: invalid %s '%s'" % (key, value))

for env_name, env_data in info.get("extra_envs", {}).items():
disallowed = ('/', ' ', ':', '#')
if any(character in env_name for character in disallowed):
sys.exit(
f"Environment names (keys in 'extra_envs') cannot contain any of {disallowed}. "
f"You tried to use: {env_name}"
)
for key, value in env_data.items():
if key not in _EXTRA_ENVS_SCHEMA:
sys.exit(f"Key '{key}' not supported in 'extra_envs'.")
types = _EXTRA_ENVS_SCHEMA[key]
if not isinstance(value, types):
types_str = " or ".join([type_.__name__ for type_ in types])
sys.exit(f"Value for 'extra_envs.{env_name}.{key}' "
f"must be an instance of {types_str}")


def generate_doc():
print('generate_doc() is deprecated. Use scripts/make_docs.py instead')
Expand Down
Loading

0 comments on commit 31e1947

Please sign in to comment.