Skip to content

Workflow

Jump to bottom
Ned Molter edited this page Mar 1, 2024 · 10 revisions

General Notes

All changes to the repository are made in branches on the developer's fork of this repository and submitted via a pull request (PR).

Each PR should be labeled with a step label (e.g. a PR to refpix should have a label refpix) and any other relevant labels. Each PR should be assigned a milestone (a build number or a future release version number).

A PR is merged after someone other than the submitter has reviewed and approved it. Usually this is one of the maintainers familiar with a particular step or pipeline. If nothing else, simply adding a LGTM comment should be present. Trivial changes can be merged by the submitter of the PR (use your best judgement). If the changes may affect another step, consider asking that step maintainer to review them.

CI tests will run on the repository and PRs performing as a minimum a PEP8 check and unit tests of all steps in the repo. Steps that depend on other steps should have some sanity unit tests if possible as well.

Example Workflow

  • Fork the main jwst repository
  • Create a local copy ("clone") of the repository you just forked and let it know where the main repository is.
git clone https://github.com/<your-git-user-name>/jwst.git
cd jwst
git remote add upstream https://github.com/spacetelescope/jwst.git

An Aside About GitHub Authentication

If you're cloning GitHub repositories using HTTPS, you can use a credential helper to tell Git to remember your GitHub username and password every time it talks to GitHub:

https://help.github.com/articles/caching-your-github-password-in-git/

If you clone GitHub repositories using SSH, then you authenticate using SSH keys instead of a username and password:

https://help.github.com/articles/generating-an-ssh-key/

Back to the Workflow

You should be able to look at all remotes now and see something like 

% git remote -v
upstream   https://github.com/spacetelescope/jwst.git (fetch)
upstream   https://github.com/spacetelescope/jwst.git (push)
origin     https://github.com/<your-git-user-name>/jwst.git (fetch)
origin     https://github.com/<your-git-user-name>/jwst.git (push)

The above operations to fork and clone the repo are normally performed once (not necessary every time you want to create a new PR branch).

Now start work on a new feature/change by making a separate branch that tracks upstream/master.

First, always fetch the upstream branch to get the latest changes into your clone. 

git fetch upstream

Make a new branch called feature1 off of upstream/master.

git checkout -b feature1 upstream/master

You will see a message like the following:

Branch feature1 set to track upstream/master

Work on this branch until the new feature is ready. Then commit the changes:

git add <file_names_to_commmit>
git commit -m "JP-nnnn: Adding feature 1" file_names_to_commit

Note that it's common (and encouraged) practice to include the Jira ticket number as a prefix in your commit message. This sets up an automatic link between the Jira ticket and the PR.

You should test that the changes actually work. It's easiest to do this by installing the jwst package in "editable" mode with the test and documentation dependencies:

pip install -e .[test,docs]

Sometimes development of a new feature takes a while, and perhaps upstream/master has passed you by, or you need a bug fix in your feature branch that someone else committed to upstream/master. So to update your branch with the latest from upstream:

git fetch upstream
git rebase upstream/master

It is better to use rebase than merge, because the commit log of upstream/master remains cleaner once your branch is merged back in.

When all changes are made, push to your forked repository (note: you are pushing the new branch to your own forked copy of the main repository):

git push origin feature1

Now go to your forked copy (http://github.com/<your_github_username>/jwst.git) to find the branch you just pushed. (There's a link to your github account in the upper right corner of any github page. Clicking on it will take you to your github profile and you can find the repository under the tab repositories.)

If it's a new branch you will see Compare and pull tab next to the branch name.

You can look at the changes online. If everything looks good, create a PR against upstream/master by clicking on the Create pull request button. This will take you to the main JWST repository on github.

  • Add relevant descriptions, labels, milestones, and ask for a review.
  • After the review and when CI and regression tests pass, merge using the web interface.

PEP8

We use ruff to help us maintain PEP8 standards. Simply run ruff check <filename> to check style, and ruff check --fix <filename> to fix style (if you agree with the suggested changes). Custom ruff settings are defined in this repo's pyproject.toml, but they should be automatically honored when running ruff locally within the repository.

Notes on how to configure Emacs and Vim to conform to PEP8.

Notes on VSCode extensions for linting and formatting.

Clone this wiki locally