Skip to content

Commit

Permalink
MCR-3080: bring back a11y tests in ci (#2715)
Browse files Browse the repository at this point in the history
* ADd and implement cypress-axe

* Add accessibility commands

* Fix launch darkly intercepts for local deployment.

* Add two accessibility tests.

* Reverting bad merge

* ADd and implement cypress-axe

* Add accessibility commands

* Fix launch darkly intercepts for local deployment.

* Add two accessibility tests.

* Reverting bad merge

* lock file

* Reverting bad merge again

* Fix lock file.

* Fix lock file.

* Remove pa11y.

* Add axe-core and cypress-axe to root package.json

* Cleanup

* Use WCAG 2.2 Level AA

* Lock file

* Add documentation

* Rename tests

* Update ADR

* Replace depreciated command

* Skipping violations and made tickets to address them.

* Update doc
  • Loading branch information
JasonLin0991 authored Aug 30, 2024
1 parent 21dc063 commit a270f44
Show file tree
Hide file tree
Showing 15 changed files with 378 additions and 133 deletions.
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Define Frontend Toolset for Accessibility Standards

## This ADR has been superseded by [028-updated-frontend-a11y-toolset](028-updated-frontend-a11y-toolset.md)

Decide on a toolset to help ensure MCRRS frontend development follows accessibility standards (specifically WCAG AA guidelines).

Tooling is not a replacement for engineers/designers with strong expertise in HTML or manual testing with assistive technology. Thus, the toolset will focus on the types of accessibility issues where tooling is useful. This includes validating HTML attributes, checking basic page structure, ensuring media content has descriptive tags, ensuring that buttons are used appropriately, and checking for basic color contrast.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Updated Frontend Toolset for Accessibility Standards

Decide on a toolset to help ensure MCRRS frontend development follows accessibility standards (specifically WCAG 2.2 AA guidelines).

Tooling is not a replacement for engineers/designers with strong expertise in HTML or manual testing with assistive technology. Thus, the toolset will focus on the types of accessibility issues where tooling is useful. This includes validating HTML attributes, checking basic page structure, ensuring media content has descriptive tags, ensuring that buttons are used appropriately, and checking for basic color contrast.

### Considerations

- Must check React component design for accessibility standards
- Must validate raw HTML markup for accessibility standards
- Must include end to end testing for accessibility standards rather than only unit tests
- Preference for easy options to run in CI

### Decision Changes

Updated toolset for accessibility testing and supersedes [005-frontend-a11y-toolset](005-frontend-a11y-toolset.md)

[storybook addon-a11y][storybook-addon-a11y] - Provides integration with Storybook GUI which allows the team to surface a11y concerns to non-developer stakeholders. This helps make a11y issues visible beyond the code editor or command line.

[eslint jsx-a11y][eslint-a11y] Provides React-friendly code linting which allows developers to find accessibility issues during local development.

[cypress-axe][cypress-axe] and [axe-core][axe-core] - Accessibility test runner for end to end testing in the command line or Node.js. End to end testing can catch concerns that are not noticeable in linting or unit tests since tests run on browser. Our previous choice [pa11y][pa11y] and [pa11y-ci][pa11y-ci] seems to not be maintained anymore and no longer works with Cypress end to end testing.

[jest-axe][jest-axe] - We are no longer using this tool and has been removed `package.json`

### Pro/Cons

#### [@axe-core/react][axe-core-react], [@axe-core/cli][axe-core-cli]

- `+` This react plugin outputs accessibility warnings in Chrome Devtools when the app is running. The cli plugin surfaces a command for running in CI.
- `+` Base standard for most accessibility tools. Customizable for different levels of accessibility compliance.
- `-` Documentation. Couldn’t get it set up quickly, they are in progress of moving their docs and consolidating their npm modules. Minimal documentation and example on best use in CI.
- `-` Heard from other teams that have struggled customization - when used with standard flags it is more strict than needed.

#### [cypress-axe][cypress-axe] and [axe-core][axe-core]
- `+` `axe-core` is a well known and maintained a11y testing engine
- `+` `axe-core` has great documentation
- `+` `cypress-axe` can run tests with specific a11y standards like `WCAG 2.2 AA`.
- `+` `cypress-axe` is a Cypress plugin for axe-core and most of the configuration for this plugin follows `axe-core` which has great documentation.
- `-` `cypress-axe` itself has sparse documentation
- `-` `cypress-axe` is very minimal in features and if we wanted specific things like reporting, we would have to implement that ourselves.

#### [Cypress Accessibility][cypress-accessibility]
- `+` Built in accessibility testing by Cypress.
- `-` Still in early access and not much is known about implementation or documentation.
- `-` As of now it looks like it would cost money in addition to what we already pay for Cypress.
- `-` To get access we have to sign up for early access as a trial.

[storybook-addon-a11y]: https://storybook.js.org/addons/@storybook/addon-a11y
[eslint-a11y]: https://github.com/jsx-eslint/eslint-plugin-jsx-a11y
[pa11y]: https://github.com/pa11y/pa11y
[pa11y-ci]: https://github.com/pa11y/pa11y-ci
[jest-axe]: https://github.com/nickcolley/jest-axe
[axe-core-react]: https://www.npmjs.com/package/@axe-core/react
[axe-core-cli]: https://www.npmjs.com/package/@axe-core/cli
[cypress-audit]: https://github.com/mfrachet/cypress-audit
[lighthouse-ci]: https://github.com/GoogleChrome/lighthouse-ci
[cypress-axe]: https://github.com/component-driven/cypress-axe
[axe-core]: https://github.com/dequelabs/axe-core
[cypress-accessibility]: https://www.cypress.io/blog/introducing-cypress-accessibility
2 changes: 2 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@
"devDependencies": {
"c8": "^10.1.2",
"cypress": "^13.13.2",
"axe-core": "^4.10.0",
"cypress-axe": "^1.5.0",
"danger": "^11.2.6",
"husky": "^9.1.5",
"lint-staged": "^15.2.7",
Expand Down
102 changes: 39 additions & 63 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit a270f44

Please sign in to comment.