-
Notifications
You must be signed in to change notification settings - Fork 5
lab 9
Friday November 19 by Midnight.
This week we are focused on managing project complexity through the use of Continuous Integration Pipelines and Test Automation. We will build on the work you did writing tests in Lab 8, and use automated ways to run our tests, and systems to continually guarantee that these tests are run before we merge or ship new versions.
NOTE: I highly recommend watching this week's video lectures before you start this lab.
This lab will help you learn to work with GitHub Actions Workflows, and also learn about test coverage.
Our test framework has been integrated and we've written a number of tests, but so far they only run if you remember to run them. Developers are human and make mistakes. It's easy to forget to run the tests before pushing new code or merging a pull request. What if that new code breaks our tests?
A central idea of testing on a large project is that no developer is ever allowed to break the main branch. If someone's changes would break the tests, we can't merge them; if we do, we'll introduce a bug that will derail the work of other developers on the project until it gets fixed, and also affect users.
To help us keep the default branch working at all times, we need to add Continuous Integration (aka CI) to our project. CI is a method whereby we build and run our tests (i.e., integrate everything in our project) automatically whenever anything is pushed to the repo, or a new pull request is made. With CI we do this continually and ideally on every change. This lets us monitor the effect of a given change and assess its quality.
For CI to work, we need dedicated machines to be available to checkout our code, build it, and run our tests. We could set up our own machines and have them listen for various events on GitHub in order to trigger a build; or we can use any number of existing CI cloud providers who will do this for us. Many of these are free for open source developers.
GitHub provides its own CI service called GitHub Actions. Actions allow us to automate workflows in response to events in a GitHub repo (e.g., merging to main, creating a pull request, etc).
We can add a GitHub Action via the GitHub UI for our repo. Doing so will create a new file YAML file with information about when to trigger the action, and which steps to perform.
You can read more about how to create a CI workflow for your language using these GitHub guides:
- Building and testing Node.js
- Building and testing Python
- Building and testing Java with Maven, Building and testing Java with Gradle, or Building and testing Java with Ant
- Building and testing .NET
- Building and testing Swift
There are also starter workflow files for many other languages, including dotnet core, C/C++, Rust, Go, etc.
In your repo, create a GitHub Actions Workflow that runs your tests. It should be triggered on any push to your default branch (e.g., main), and for any pull_request to your default branch. You can add, edit, and commit the file directly in GitHub. When you do, it should trigger a build that you can inspect in the Actions tab of your repo.
NOTE: if you commit the .yml workflow file via the GitHub web page, make sure you git pull origin master to your local repo so that you also get this change on your local machine's repo!
To make sure that our CI workflow is correct, we'll try creating a pull request against our own repo.
Pick a third area of your code that needs some tests, for example another function. Create a new branch for this work (e.g., add-more-tests) and write your new tests. Use your coverage tool to help you know when you've done enough work.
When all of your tests are passing, commit your new test cases to the add-more-tests branch, and push this branch to your origin (don't merge it to master):
$ git push origin add-more-testsWhen this completes, git will let you create a new Pull Request (or you can do it manually via the GitHub UI). Create a pull request from your add-more-tests branch to your main branch.
NOTE: this might seem odd to you--why would I ever make a pull request to my own repo? As strange as this sounds, it's often done in projects where a community of developers is working, and one of the people owns the repo. This allows the owner to also have their code reviewed and go through CI.
In GitHub, make sure your pull request has triggered your CI workflow, and you have a report of whether it is running, passed, or failed. If you don't see this in the pull request, something is wrong in Step 6, and you should go back to investigate.
If you do see your CI workflow being run, congratulations! Try making another commit that breaks the tests (e.g., make a new commit on your branch that breaks the logic in your function, so the tests will fail). Push this broken commit to GitHub and watch your CI build to make sure it fails as you expect. Go and inspect the workflow's logs to see the error message that your tests produced. Does it make sense to you?
Once you can confirm that a failed test will break your CI workflow, and know how to inspect the logs, make another change to fix the failing test and push it. Confirm that this change fixes the CI and everything is green.
Once you have successfully passed, failed, and passed CI again, go ahead and merge your pull request to master.
Now that you've done all the hard work of setting up automated tests, code coverage, and CI, you might as well include more more tests!
For this step, find a partner in the class and send each other a pull request that adds one more test case to their project. If you're feeling brave, pick a partner that uses a different language/framework than you. It's great to get experience writing tests in different environments.
Work with your partner to figure out which function or file you should add tests for, and read their CONTRIBUTING.md document to learn how to use their testing tools. If anything is unclear to you, file issues to have the owner improve their repo's documentation.
Do all of your work in a separate branch and send a pull request when you are finished and the tests run locally. Make sure that GitHub's CI runs the pull request's new tests and passes before this gets merged. If anything fails or needs fixing, do that in the pull request by adding more commits.
In the same way that we added automated testing to our CI workflow, you can also add a command line script to run your linter and other static analysis tools. You might do this by overloading your test script to call both the linter and your test runner one after the other; or you might add a dedicated script just for the linter.
Many projects run linters before the automated tests, and a successful CI build will mean passing all lint and test conditions. This is done to help developers remember to run the linting steps, and avoid merging silly typos and bugs that could be easily caught at development time. It also reduces the amount that a reviewer has to do, since they know that linting and test cases are being checked by the CI machines.
When you have completed all of the required steps above, please write a detailed blog post. In your post, discuss the following:
- How did you set up your GitHub Actions CI Workflow? What did the YAML for this workflow look like?
- How did your partner's repo and testing setup differ from yours? What was it like writing tests for a project you didn't create?
- What do you think of CI now that you've set it up for yourself?
When you have completed all the requirements above, please add your details to the table below.