Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use poetry for bootstrap #478

Merged
merged 3 commits into from
Jan 9, 2024
Merged

Use poetry for bootstrap #478

merged 3 commits into from
Jan 9, 2024

Conversation

eloquence
Copy link
Member

@eloquence eloquence commented Dec 12, 2023

Resolves #468

This replaces our use of pip-compile with poetry to manage the dependencies for building other wheels, and updates the instructions accordingly.

To do so, it freezes all bootstrap dependencies at the current state, except for the ones we no longer need.

Context

Our bootstrap is quite fragile due to being so dated, and changes to individual dependencies can either cause breakage, changes to existing wheels, or both. Most notably, we cannot update to the latest Cython until we have removed our use of PyYAML (freedomofpress/securedrop-client#1681), which is underway. Instead of opportunistic piecemeal updates, #464 proposes a full update of all bootstrap requirements, which I would suggest we do after we've removed PyYAML so we don't have to worry about that piece.

Testing

To test this PR, try rebuilding the bootstrap by following the instructions in the README, with a test dependency such as cowsay, while on this branch:

# Ensure you are running in a cleanly boostrapped virtual environment
rm -rf .venv
make install-deps
source .venv/bin/activate
# Update the workstation-bootstrap/pyproject.toml file with a new dependency
# For example, add `cowsay = "6.0.0"` to `pyproject.toml`
# Then update the lockfile:
poetry -C workstation-bootstrap/ lock
# Now we are ready to build updated wheels:
./scripts/build-sync-wheels --project workstation-bootstrap --pkg-dir ./workstation-bootstrap
# Once the new wheels are ready, we recreate our sha256sums:
./scripts/sync-sha256sums ./workstation-bootstrap
# Sign the list of sha256sums
gpg --armor --output workstation-bootstrap/sha256sums.txt.asc --detach-sig  workstation-bootstrap/sha256sums.txt
# We can even verify if we want
./scripts/verify-sha256sum-signature ./workstation-bootstrap/
# Update the build-requirements.txt file
./scripts/update-requirements --pkg-dir ./workstation-bootstrap/ --project workstation-bootstrap

Check if you encounter errors or unexpected results (git diff) along the way.

@legoktm legoktm self-assigned this Dec 12, 2023
Copy link
Member

@legoktm legoktm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice :) walked through the test plan and it all seemed to work!

workstation-bootstrap/pyproject.toml Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@eloquence eloquence force-pushed the use-poetry-for-bootstrap branch from 96606e6 to 24fa403 Compare December 12, 2023 22:33
@eloquence eloquence force-pushed the use-poetry-for-bootstrap branch from 24fa403 to 1bc3b42 Compare January 8, 2024 16:43
Copy link
Member

@legoktm legoktm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!!

@legoktm legoktm merged commit ca81d0d into main Jan 9, 2024
19 checks passed
@legoktm legoktm deleted the use-poetry-for-bootstrap branch January 9, 2024 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Use poetry for bootstrap dependency mgmt
2 participants