Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add extra pages to pkg installers #852

Merged
merged 36 commits into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
2a8f0e4
Add extra pages feature for pkg installers
marcoesters Aug 20, 2024
62b3cc6
Compile xcodeprojects
marcoesters Aug 20, 2024
59a02c6
Restructure signing
marcoesters Sep 4, 2024
f35df22
Use manual code signing for example
marcoesters Sep 4, 2024
44eac8f
Create CodeSign class
marcoesters Sep 4, 2024
555eccb
Add testing for signed installers
marcoesters Sep 5, 2024
769a07a
Update documentation
marcoesters Sep 5, 2024
71fcd61
Enable MacOS signing testing on CI
marcoesters Sep 5, 2024
1866452
Expand post-install pages to Windows
marcoesters Sep 6, 2024
c5971c1
Clarify in tests that codesign outputs to stderr
marcoesters Sep 6, 2024
80479ac
Add news file
marcoesters Sep 6, 2024
9b856c6
Remove debug code
marcoesters Sep 6, 2024
9d08014
Test: do not run add-trusted-cert on CI
marcoesters Sep 6, 2024
d9b13ed
Merge branch 'pkg-extra-pages' of github.com:marcoesters/constructor …
marcoesters Sep 6, 2024
ca9f855
Test: Skip trusting
marcoesters Sep 6, 2024
0a956b5
Add to system keychain
marcoesters Sep 6, 2024
3f65457
Use system keychain on CI
marcoesters Sep 6, 2024
5fc9d33
Skip keychain domains
marcoesters Sep 6, 2024
0dfc46c
Set default keychain instead of using list-keychains
marcoesters Sep 6, 2024
7fc2d23
Actually set default keychain
marcoesters Sep 6, 2024
1ad573d
Unset password for system keychain
marcoesters Sep 6, 2024
875af89
Use system keychain only for certificate trusting
marcoesters Sep 6, 2024
5f5b2e4
Fix CI logic
marcoesters Sep 6, 2024
261fbe2
Do not test for installer signing
marcoesters Sep 6, 2024
db7e736
Remove special signing keyword
marcoesters Sep 6, 2024
dd8e347
Set up debug session for macos-13
marcoesters Sep 9, 2024
fcfee0f
Move tmate action behind example runs
marcoesters Sep 9, 2024
477ad90
Start session after setup steps
marcoesters Sep 9, 2024
737d364
Disable default shell
marcoesters Sep 9, 2024
5869348
Do not set -e with default shell
marcoesters Sep 9, 2024
79aade5
Copy full environment for self-signed certificates
marcoesters Sep 9, 2024
f11f283
Apply suggestions from code review
marcoesters Sep 10, 2024
188c9b4
Update docs
marcoesters Sep 10, 2024
1bda971
Factor out xcodebuild loop into function
marcoesters Sep 10, 2024
0c211da
Add plug-in examples
marcoesters Sep 10, 2024
c6858a1
Fix typing for python 3.8
marcoesters Sep 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions CONSTRUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -777,6 +777,20 @@ If this key is missing, it defaults to a message about Anaconda Cloud.
You can disable it altogether if you set this key to `""` (empty string).
(MacOS only).

### `post_install_pages`

_required:_ no<br/>
_types:_ list, string<br/>

Adds extra pages to the installers to be shown after installation.

For PKG installers, these can be compiled `installer` plug-ins or
directories containing an Xcode project. In the latter case,
constructor will try and compile the project file using `xcodebuild`.

For Windows, the extra pages must be `.nsi` files.
They will be inserted as-is before the conclusion page.

### `conclusion_file`

_required:_ no<br/>
Expand All @@ -788,9 +802,7 @@ plain text (.txt), rich text (.rtf) or HTML (.html). If both
`conclusion_file` and `conclusion_text` are provided,
`conclusion_file` takes precedence. (MacOS only).

If the installer is for windows and conclusion file type is nsi,
it will use the nsi script to add in extra pages and the conclusion file
at the end of the installer.
If the installer is for Windows, the file type must be nsi.

### `conclusion_text`

Expand Down
15 changes: 12 additions & 3 deletions constructor/construct.py
Original file line number Diff line number Diff line change
Expand Up @@ -569,6 +569,17 @@
If this key is missing, it defaults to a message about Anaconda Cloud.
You can disable it altogether if you set this key to `""` (empty string).
(MacOS only).
'''),

('post_install_pages', False, (list, str), '''
Adds extra pages to the installers to be shown after installation.

For PKG installers, these can be compiled `installer` plug-ins or
directories containing an Xcode project. In the latter case,
constructor will try and compile the project file using `xcodebuild`.

For Windows, the extra pages must be `.nsi` files.
They will be inserted as-is before the conclusion page.
'''),

('conclusion_file', False, str, '''
Expand All @@ -578,9 +589,7 @@
`conclusion_file` and `conclusion_text` are provided,
`conclusion_file` takes precedence. (MacOS only).

If the installer is for windows and conclusion file type is nsi,
it will use the nsi script to add in extra pages and the conclusion file
at the end of the installer.
If the installer is for Windows, the file type must be nsi.
'''),

('conclusion_text', False, str, '''
Expand Down
9 changes: 6 additions & 3 deletions constructor/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,12 @@ def main_build(dir_path, output_dir='.', platform=cc_platform,
for key in ('license_file', 'welcome_image', 'header_image', 'icon_image',
'pre_install', 'post_install', 'pre_uninstall', 'environment_file',
'nsis_template', 'welcome_file', 'readme_file', 'conclusion_file',
'signing_certificate'):
if info.get(key): # only join if there's a truthy value set
info[key] = abspath(join(dir_path, info[key]))
'signing_certificate', 'post_install_pages'):
if value := info.get(key): # only join if there's a truthy value set
if isinstance(value, str):
info[key] = abspath(join(dir_path, info[key]))
elif isinstance(value, list):
info[key] = [abspath(join(dir_path, val)) for val in value]

# Normalize name and set default value
if info.get("windows_signing_tool"):
Expand Down
5 changes: 5 additions & 0 deletions constructor/nsis/main.nsi.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,11 @@ Page Custom InstModePage_Create InstModePage_Leave
# Custom options now differ depending on installation mode.
Page Custom mui_AnaCustomOptions_Show
!insertmacro MUI_PAGE_INSTFILES

#if post_install_pages is True
@POST_INSTALL_PAGES@
#endif

#if with_conclusion_text is True
!define MUI_FINISHPAGE_TITLE __CONCLUSION_TITLE__
!define MUI_FINISHPAGE_TITLE_3LINES
Expand Down
121 changes: 97 additions & 24 deletions constructor/osxpkg.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,20 @@
import os
import shlex
import shutil
import subprocess
import sys
import xml.etree.ElementTree as ET
from os.path import abspath, dirname, exists, isdir, join
from pathlib import Path
from plistlib import dump as plist_dump
from tempfile import NamedTemporaryFile
from typing import List

from . import preconda
from .conda_interface import conda_context
from .construct import ns_platform, parse
from .imaging import write_images
from .signing import CodeSign
from .utils import (
add_condarc,
approx_size_kb,
Expand Down Expand Up @@ -423,6 +426,80 @@ def pkgbuild_prepare_installation(info):
shutil.rmtree(f"{pkg}.expanded")


def create_plugins(pages: list = None, codesigner: CodeSign = None):
def _build_xcode_projects(xcodeporj_dirs: List[Path]):
xcodebuild = shutil.which("xcodebuild")
if not xcodebuild:
raise RuntimeError(
"Plugin directory contains an uncompiled project,"
" but xcodebuild is not available."
)
try:
subprocess.run([xcodebuild, "--help"], check=True, capture_output=True)
except subprocess.CalledSubprocessError:
raise RuntimeError(
"Plugin directory contains an uncompiled project, "
"but xcodebuild requires XCode to compile plugins."
)
for xcodeproj in xcodeproj_dirs:
build_cmd = [
xcodebuild,
"-project",
str(xcodeproj),
f"CONFIGURATION_BUILD_DIR={PLUGINS_DIR}",
# do not create dSYM debug symbols directory
"DEBUG_INFORMATION_FORMAT=",
]
explained_check_call(build_cmd)

if not pages:
return
elif isinstance(pages, str):
pages = [pages]

fresh_dir(PLUGINS_DIR)

for page in pages:
xcodeproj_dirs = [
file.resolve()
for file in Path(page).iterdir()
if file.suffix == ".xcodeproj"
]
if xcodeproj_dirs:
_build_xcode_projects(xcodeproj_dirs)
else:
plugin_name = os.path.basename(page)
page_in_plugins = join(PLUGINS_DIR, plugin_name)
shutil.copytree(page, page_in_plugins)

if codesigner:
with NamedTemporaryFile(suffix=".plist", delete=False) as entitlements:
plist = {
"com.apple.security.cs.allow-unsigned-executable-memory": True,
"com.apple.security.cs.disable-library-validation": True,
}
plist_dump(plist, entitlements)
jaimergp marked this conversation as resolved.
Show resolved Hide resolved

for path in Path(PLUGINS_DIR).iterdir():
codesigner.sign_bundle(path, entitlements=entitlements.name)
os.unlink(entitlements.name)

plugins = [file.name for file in Path(PLUGINS_DIR).iterdir()]
with open(join(PLUGINS_DIR, "InstallerSections.plist"), "wb") as f:
plist = {
"SectionOrder": [
"Introduction",
"ReadMe",
"License",
"Target",
"PackageSelection",
"Install",
*plugins,
]
}
plist_dump(plist, f)


def pkgbuild_script(name, info, src, dst='postinstall', **kwargs):
fresh_dir(SCRIPTS_DIR)
fresh_dir(PACKAGE_ROOT)
Expand All @@ -446,12 +523,13 @@ def create(info, verbose=False):
"installation! Aborting!"
)

global CACHE_DIR, PACKAGE_ROOT, PACKAGES_DIR, SCRIPTS_DIR
global CACHE_DIR, PACKAGE_ROOT, PACKAGES_DIR, PLUGINS_DIR, SCRIPTS_DIR

CACHE_DIR = info['_download_dir']
SCRIPTS_DIR = join(CACHE_DIR, "scripts")
PACKAGE_ROOT = join(CACHE_DIR, "package_root")
PACKAGES_DIR = join(CACHE_DIR, "built_pkgs")
PLUGINS_DIR = join(CACHE_DIR, "plugins")

fresh_dir(PACKAGES_DIR)
prefix = join(PACKAGE_ROOT, info.get("pkg_name", info['name']).lower())
Expand Down Expand Up @@ -499,31 +577,20 @@ def create(info, verbose=False):
shutil.copyfile(info['_conda_exe'], join(prefix, "_conda"))

# Sign conda-standalone so it can pass notarization
notarization_identity_name = info.get('notarization_identity_name')
if notarization_identity_name:
with NamedTemporaryFile(suffix=".plist", delete=False) as f:
plist = {
codesigner = None
if notarization_identity_name := info.get('notarization_identity_name'):
codesigner = CodeSign(
notarization_identity_name,
prefix=info.get("reverse_domain_identifier", info['name'])
)
entitlements = {
"com.apple.security.cs.allow-jit": True,
"com.apple.security.cs.allow-unsigned-executable-memory": True,
"com.apple.security.cs.disable-executable-page-protection": True,
"com.apple.security.cs.disable-library-validation": True,
"com.apple.security.cs.allow-dyld-environment-variables": True,
}
plist_dump(plist, f)
explained_check_call(
[
# hardcode to system location to avoid accidental clobber in PATH
"/usr/bin/codesign",
"--verbose",
'--sign', notarization_identity_name,
"--prefix", info.get("reverse_domain_identifier", info['name']),
"--options", "runtime",
"--force",
"--entitlements", f.name,
join(prefix, "_conda"),
]
)
os.unlink(f.name)
}
codesigner.sign_bundle(join(prefix, "_conda"), entitlements=entitlements)

# This script checks to see if the install location already exists and/or contains spaces
# Not to be confused with the user-provided pre_install!
Expand Down Expand Up @@ -579,14 +646,20 @@ def create(info, verbose=False):
explained_check_call(args)
modify_xml(xml_path, info)

if plugins := info.get("post_install_pages"):
create_plugins(plugins, codesigner=codesigner)

identity_name = info.get('signing_identity_name')
explained_check_call([
build_cmd = [
"/usr/bin/productbuild",
"--distribution", xml_path,
"--package-path", PACKAGES_DIR,
"--identifier", info.get("reverse_domain_identifier", info['name']),
"tmp.pkg" if identity_name else info['_outpath']
])
]
if plugins:
build_cmd.extend(["--plugins", PLUGINS_DIR])
build_cmd.append("tmp.pkg" if identity_name else info['_outpath'])
explained_check_call(build_cmd)
if identity_name:
explained_check_call([
# hardcode to system location to avoid accidental clobber in PATH
Expand Down
53 changes: 52 additions & 1 deletion constructor/signing.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@
import os
import shutil
from pathlib import Path
from plistlib import dump as plist_dump
from subprocess import PIPE, STDOUT, check_call, run
from tempfile import NamedTemporaryFile
from typing import Union

from .utils import check_required_env_vars, win_str_esc
from .utils import check_required_env_vars, explained_check_call, win_str_esc

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -218,3 +220,52 @@ def verify_signature(self, installer_file: Union[str, Path]):
except ValueError:
# Something else is in the output
raise RuntimeError(f"Unexpected signature verification output: {proc.stdout}")


class CodeSign(SigningTool):
def __init__(
self,
identity_name: str,
prefix: str = None,
):
# hardcode to system location to avoid accidental clobber in PATH
super().__init__("/usr/bin/codesign")
self.identity_name = identity_name
self.prefix = prefix

def get_signing_command(
self,
bundle: Union[str, Path],
entitlements: Union[str, Path] = None,
) -> list:
command = [
self.executable,
"--sign",
self.identity_name,
"--force",
"--options",
"runtime",
]
if self.prefix:
command.extend(["--prefix", self.prefix])
if entitlements:
command.extend(["--entitlements", str(entitlements)])
if logger.getEffectiveLevel() == logging.DEBUG:
command.append("--verbose")
command.append(str(bundle))
return command

def sign_bundle(
self,
bundle: Union[str, Path],
entitlements: Union[str, Path, dict] = None,
):
if isinstance(entitlements, dict):
with NamedTemporaryFile(suffix=".plist", delete=False) as ent_file:
plist_dump(entitlements, ent_file)
command = self.get_signing_command(bundle, entitlements=ent_file.name)
explained_check_call(command)
os.unlink(ent_file.name)
else:
command = self.get_signing_command(bundle, entitlements=entitlements)
explained_check_call(command)
27 changes: 24 additions & 3 deletions constructor/winexe.py
Original file line number Diff line number Diff line change
Expand Up @@ -302,14 +302,28 @@ def make_nsi(
# for the newlines business
replace['CONCLUSION_TEXT'] = "\r\n".join(conclusion_lines[1:])

for key in ['welcome_file', 'conclusion_file']:
for key in ['welcome_file', 'conclusion_file', 'post_install_pages']:
value = info.get(key, "")
if value and not value.endswith(".nsi"):
if not value:
continue
if isinstance(value, str) and not value.endswith(".nsi"):
logger.warning(
"On Windows, %s must be a .nsi file; %s will be ignored.",
"On Windows, %s must be an .nsi file; %s will be ignored.",
key,
value,
)
elif isinstance(value, list):
valid_values = []
for val in value:
if val.endswith(".nsi"):
valid_values.append(val)
else:
logger.warning(
"On Windows, %s must be .nsi files; %s will be ignored.",
key,
val,
)
info[key] = valid_values

for key, value in replace.items():
if value.startswith('@'):
Expand All @@ -332,6 +346,7 @@ def make_nsi(
ppd["has_conda"] = info["_has_conda"]
ppd["custom_welcome"] = info.get("welcome_file", "").endswith(".nsi")
ppd["custom_conclusion"] = info.get("conclusion_file", "").endswith(".nsi")
ppd["post_install_pages"] = bool(info.get("post_install_pages"))
data = preprocess(data, ppd)
data = fill_template(data, replace, exceptions=nsis_predefines)
if info['_platform'].startswith("win") and sys.platform != 'win32':
Expand Down Expand Up @@ -373,6 +388,12 @@ def make_nsi(
if ppd['custom_conclusion']
else ''
),
(
'@POST_INSTALL_PAGES@',
'\n'.join(
custom_nsi_insert_from_file(file) for file in info.get('post_install_pages', [])
),
),
('@TEMP_EXTRA_FILES@', '\n '.join(insert_tempfiles_commands(temp_extra_files))),
('@VIRTUAL_SPECS@', " ".join([f'"{spec}"' for spec in info.get("virtual_specs", ())])),

Expand Down
Loading
Loading