Testing Dozen is:
- Testing Specialist training / mentoring program run by Maaret Pyhäjärvi, co-facilitated by Mirja Pyhäjärvi
- In Finland for people seeking testing jobs in Finland
- Pilot batch runs 6 months from November 16th, 2022
- Svetlana Egorov @SvetlanaS1212
- Pia @piavesala
- samuel @samuelnarteyGH @samuelnomis
- Olga Larintseva @Loranpire
- Katri Kiero @katrikiero
- Gold Lola @GoldLola
- Olayinka @Olayinka_Dele
- Sanna Tawah @nanahamjam
- Martha @Lopezmartha
- Sergey @embedded-sergey
What we did:
- Create 1st version of this page on Github
- Remote control tech check: writing names over Zoom remote control on this page on Maaret's computer
- Making changes through forks, branches and pull requests
Targeted Capability:
- Place for creating portfolio of your learning - your Github
- Linking your Github with course page
Topics:
- Github: repositories, forks and branches, pull requests
Homework:
- Improve this page through pull requests
- Install Git and VSCode
Recommended reading:
- Git tutorial
- Markdown manual
- Catalin Pit: How to Create a Kickass GitHub Profile Page
- Ship, show ask - branching strategies
- Git commit messages
Targeted Capability:
- Writing basic tests for a web app with Python and Playwright
- Exploring through automation to find bugs
Topics:
- Contemporary exploratory testing with automationist's gambit
- Basic concepts: bug, test, coverage, oracle, heuristics
- Basic Python, pytest and Playwright
Target app:
Gist of what session produced:
Homework:
- Try to get python-pytest-playwright tests to run on your computer
- Think about more inputs that could reveal bugs
Recommended readings:
- Courseware: Exploratory Testing Foundations -course
- pytest
- playwright in python
- Top Programming Languages 2022
- Elisabeth Hendrickson: Test Heuristics Cheat Sheet
- Github Naughty Strings
- Exploratory Testing Dynamics
- QA Engineer Roadmap - conceptual learning map
- Computing tools for CS studies course material
Checklist to get tooling installed:
- install vscode (win/mac: download and install) -> see it starts
- verify what is installed from vscode terminal:
git --version, win:python --version, macpython3 --version,node --version - install tooling managing programmer tooling on mac: homebrew
- install tooling: git (win: download&install, mac:
brew install git) - install programming languages: python (win: download&install, mac:
brew install python3.11) - install programming languages: nodejs (win: download&install, mac:
brew install node - verify what is installed from vscode terminal:
git --version, win:python --version, macpython3 --version,node --version
Checklist for further configuring tooling:
- add local folder and open folder in vscode
- change terminal on windows: git bash or cmd over powershell
- add simple file.py with
print('whatever')in it and run it from terminal withpython file.py - sandbox your folder as python virtual environment with
python -m venv .venv - activate your virtual environment (win: restart terminal, mac:
source .venv/bin/activate - install dependencies: pytest (win/mac:
pip install pytest) - create simple test_file.py with one test method
def test_method(): pass - run tests from terminal with
pytest test_file.py - define pytest as test framework for test lab in vscode to see test_files and test_method() in runner
- create requirements.txt file and write in dependencies, each on own line:
pytest,playwright,pytest-playwright - install repo dependencies based on requirements.txt with
pip install -r requirements.txton terminal - install playwright node tooling with `playwright install on terminal
- clone a project from github and open that folder in vscode, sandbox with virtual environment as before
- make changes in files, try commit&sync in vscode
- to commit successfully, define git.name and git.email from terminal how to
- to sync successfully, create keys for github access on git bash how to
Extra practice:
- learn to navigate folders with
cd ..andcd folder_nameand check files in folder with win:dir, mac:ls/ls -la - learn to use autocomplete - write only max 3 letters and allow tab to autocomplete
Extra test playwright operational without sample project:
- create test_playwright.py -file with
from playwright_sync_api import Pageanddef test_pw(page: Page):andpage.goto("https://exploratorytestingacademy.com")each on own line as the skeleton test case - run tests
pytest --headed test_playwright.py - create pytest.ini -file with contents
[pytest]andaddopts = --headed --browser chromium --slowmo 2000to continuously run tests showing the browser, using chromium and slowing steps by 2 seconds
Targeted Capability:
- Extending inputs to find bugs
- Increased comfort with editing automated tests to extend with data samples
Topics:
- Testing as discovery
- Basic pytest-bdd in gherkin + pytest + playwright
- Bug and other words that mean almost the same
Target app:
Gist of what session produced:
Homework:
- Think about how to reveal more bugs - preview 'known bugs' and think about how you could find them
Recommended readings:
Targeted Capability:
- Learning about an entirely different type of system (text generator not a search engine)
- High level idea of ML in software systems
Topics:
- Black-box testing
- Modeling, inputs and outputs
- Test environment, test target readiness / availability
- Exploratory testing, scale from technique to approach
Target app:
Model of testing the session produced:
Recommended materials:
Targeted Capability:
- Understanding that reporting matters on ease of seeing what you think is a problem
- Reproducing a problem and reporting a problem
Topics:
- Reproduce from other people's notes - why customers reporting and tester's reporting is different
- A sample bug report
- Reproducing from a known list of bugs enables finding bugs outside the list
- Invisible ink of relevant results from testing - usually we don't have a list of bugs but must imagine one
- Bug is anything that might bug a user / other stakeholders
Target app:
Listing of what reproduced issues session produced:
Homework:
- Update statistics on what you could reproduce for next time demo of bugs around the group
Targeted Capability:
- Show and explain a problem
- A tour of all (known) issues
- Prioritizing conversations
Topics:
- Crowdsourcing as practice possibility eg. Testlio, Utest) and bug bounties for security bugs esim. Hackr
- Ensure everyone in group has seen and understood the versatility of "bugs"
- Bugs as anything that might bug a user - improvement suggestions, missing features, implementation mistakes
- Research while testing, grounding with priority: fixing relevance of the project
- Completing round of problems with this application
Targeted Capability:
- Listing functionalities not only bugs
- Analysis: function and structure
Topics:
- Ensemble exploratory testing, emergent constraints
- Functionality and bug note listing
- Local and hosted test environments
- Intro to REST APIs
- Debugging in IDE
- Python requests library
Target App:
- Restful Booker Platform by Mark Winteringham
- Our ensemble notes https://github.com/Testing-Dozen/dozen_restful_booker
Recommended materials:
Homework:
- Inspect in browser, network tab and requests and responses
Targeted Capability:
- First basic API test for GET endpoint on local and remote deployments
Topics:
- Localhost and hosted on some computer (cloud) deployments
- Unauthenticated endpoints with requests-library
- GET and 200 response, verifying whole, contains, element contains and element exists conditions
- Setup and teardown with pytest fixtures
Target App:
- Restful Booker Platform by Mark Winteringham
Our Tests:
Homework:
- Create GET and POST calls on the front page APIs that are unauthenticated
Targeted Capability:
- Exploratory unit testing applied on roman numerals function beyond most obvious tests
Topics:
- Applying positions as trainee / junior test engineer
- pytest foundations: test naming, asserts, approvals, parametrizing, conftest
- Testing fundamentals: coverage, results, oracles
Target App:
- Github Copilot generated roman numerals function
- Available at lines 1-16 on https://github.com/exploratory-testing-academy/do-a-thing-and-call-it-foo-solution/blob/main/important_program.py
Our tests:
Homework:
- Study traditional and simplified roman numerals in excel
- Review code for Excel and Know the Romans -page reference implementations
Recommended Readings:
- How to Make Your VSCode Sparkle
- VSCode extensions: Mermaid theme
- VSCode extensions: Power mode
Targeted Capability:
- Using earlier tests to remember earlier testing
- Test styles beyond examples of asserts
Topics:
- ApprovalTests - Run (red), Approve, Run (green), and combinationapprovals
- SoftAssertions - Collating multiple assertion results instead of failing on first
- Snapshots - Run (save), Run (green), fail when output changes
- Properties, hypothesis library - programmatic rules, generated falsifiable hypothesis tests
- xfail-decorator - expecting a failure
Target App:
- Github Copilot generated roman numerals function
- Available as https://github.com/exploratory-testing-academy/do-a-thing-and-call-it-foo-solution/blob/main/important_program.py
Our tests:
Recommended Reading:
Homework:
- Study restful booker API testing project current state
Targeted Capability:
- Running containerized local environment
- Creating and using pytest fixtures
- GET, POST, PUT and authentication in cookies
Topics:
- Starting with someone else's python test project, setting virtual environment and installing dependencies
- Authenticated endpoints with requests-library using pytest fixtures
- GET, POST, PUT and 200 vs. 400 response
- 'docker-compose up' and 'docker-compose down' to run test target on localhost
- Reading API tests in Java to turn same tests to Python
Target App:
- Restful Booker Platform by Mark Winteringham
- Code: https://github.com/mwinteringham/restful-booker-platform
Our Tests:
Homework:
- Pull requests adding more tests for restful booker APIs
Targeted Capability:
- Finding bugs in APIs
- Extending set of tests to cover more of the APIs
Topics:
- Small group practice of API tests
- Creating pull request with new tests
- Reviewing pull request and merging it
Target App:
- Restful Booker Platform by Mark Winteringham
- Code: https://github.com/mwinteringham/restful-booker-platform
Our Tests:
Homework:
- Pull requests extending and improving structurefor tests of restful booker APIs
Targeted Capability:
- Skills inventory
- Exploring an application and naming its features
- Seeing options in browser drivers: playwright and selenium
Topics:
- Skills inventory, recognizing and naming what skills we have been learning
- Testing when quality is not inherently bad
- Pair testing an application to name its features
- Basic test in python+playwright and python+selenium
- When would you care to drive real browsers
Test Target:
Bugs:
- Chosen colors on the app make recognizing elements and seeing texts difficult
- Instructions to use app in small font and located at end of list while there is space to display them more prominently
- Selecting all tasks functionality icon is not clearly communicating its existence
- 404 on favicon, no favicon defined and is visible as error in devtools
Our samples:
Targeted Capability:
- CV that gets you to interview
- Understanding your tester personality and how it impacts your likely style of testing
Topics:
- Reviewing CVs and proposing improvements
- Sharing experiences of getting to interview
- Four tester personalities and tailoring your message to more specific for CVs
Exercise:
- Lloyd Roden's good old Test Team / Tester personality Test
- Slides from 'personality astrology' books from earlier
Targeted Capability:
- From functional issues to security issues
Topics:
- Security issues in software: Adverse users, function in unintended use
- Guessing passwords
- Editing client side HTML
- Connecting available information to find security issues
Exercise:
Material:
Homework:
- Continue with juice shop exercises
Targeted Capability:
- Discussing plans of testing work
Topics:
- Plan tasks and timeboxes not test cases or requirements
- Task: work you complete until its done to expected results
- Timebox: work you complete until you have used your budget of time
- Test planning is timebox planning
Exercise:
- Havannah board game sprint 1, 2 and N
Targeted Capability:
- Browser testing with Selenium and Playwright on Python
Topics:
- Browser testing driver options: Selenium and Playwright
- APIs for browser testing
- Syntactic sugar and wrapper methods
Exercise:
- Run and extend: https://github.com/Testing-Dozen/todo-mvc-playwright-selenium
Targeted Capability:
- Pipelines as automatic way to run tests
- Naming few pipeline tools
Topics:
- Implement a pipeline to run tests with GitHub Actions
- Review pipelines on Jenkins and GitLab CI
Exercise:
- https://qaautomationlabs.com/automate-playwright-test-cases-with-ci-cd-github-actions/ + debugging missing steps
- Running sample: https://github.com/Testing-Dozen/playwright-on-github-actions
Targeted Capability:
- Navigating in code
- Starting with green after sorting dependencies
Topics:
- Dependencies, lifecycle of 3rd party components including your test framework
- Maintenance of dependencies, change strategies
- Fixtures with setup and teardown
- Modeling page in class 'page objects'
- Separating data
- Utilities and helpers
- Run parameters
- Logging
Exercise:
- Read and clean up https://github.com/exploratory-testing-academy/todomvc-python-playwright
Homework:
- Read the code to understand the repository we are refactoring for test_basic.py to page object and data separation
Recommended reading:
Targeted Capability:
- Test case management tool basics
- Tasks (one time) and tests (reusable tasks)
Topics:
- Prerequisites, Tests, Test Plans, Test Sets, Test Repository, Test Executions as Jira ticket types
- Plan is to report on results
- Tasks and Tests
- Reasons to avoid steps
- Consise and useful documentation for posterity
Targeted Capability:
- Tally of acquired skills and knowledge
- Questions and Answers
Topics:
- What we remember - restful booker, juice shop, testing personalities, learning together
- Next steps on learning journey, social learning
- Finding work to practice in
Can do the testing job?
- Can look at an application and make sense of it
- Can create UI and API tests on it (using programming language)
- Can structure test code so that reading them make sense for later
- Can define epics acceptance criteria
- Can ask good questions about features
- Can suggest ways to improve testability
- Can make sense and extend pipelines
- Can analyze changes
- Can define experiments
- Can fail gracefully with pride of learning
- Can actively learn useful things
- Can apply good judgement, prioritise and decide
- Can install software on various OSs, configure and operate it
- Can say when they are unsure and actively seek understanding
- Can create models and use them to derive coverage
- Can communicate results and status
- Can figure out systems of systems impact on testing