Skip to content

Commit

Permalink
test(storybook): support running visual tests in Chromatic
Browse files Browse the repository at this point in the history
To better make use of visual tests:
- "Avatar/In Dropdown" story renders with the dropdown open
- Modals, comboboxes, dropdowns, and popovers are now opened
  automatically (when not in docs) through interaction tests
- SkipLink: added a story showing the SkipLink when it has become visible
- Switch: added examples of enabled and hovered switches
- Added a default 1rem padding to stories, to account for normally
  overflowing content like focus styles, floating badges etc
- Several stories have custom styling to ensure elements that have been
  removed from the normal layout flow (e.g. with absolute positioning)
  or are rendered outside the Story root element (e.g. with a React
  portal) are actually visible in the snapshots. To easier facilitate
  this, a global customStylesDecorator has been added, which adds styles
  configured through `parameters.customStyles`.

This commit also replaces story-specific decorators which only added
styling with `parameter.customStyles`, simplifying the stories a bit.

Visual tests on 320px wide screen revealed a bug where Combobox was
wider than the screen. This has been fixed.

With the changes that open modals, add pseudo states etc, some new
accessibility violations surfaced.
- Fixed axe violation `svg-img-alt` in Dropdown stories
- Disabled axe rule `color-contrast` when the pseudo-class :active is
  emulated through `storybook-addon-pseudo-states`, since we concluded
  that 4.5:1 text contrast during press actions is unnecessary

Chromatic is run automatically for pull requests through Github Actions
-- see `.github/workflows/test.yml`.

If you want to trigger visual tests from your own machine, add (or edit)
the file `apps/storybook/.env`. This file is in .gitignore and will not be
committed.

Add the following to `.env`:
```
CHROMATIC_PROJECT_TOKEN=<token>
```

Replace `<token>` with the token found [here](https://www.chromatic.com/manage?appId=66fe736b9d639fe6801bf130&setup=true), under "Setup Chromatic with this project token".

Then run these commands:
```
yarn build:storybook
yarn chromatic
```

You can also replace the last command with e.g.
```
yarn chromatic --no-only-changed --only-story-names "Komponenter/Modal/*"
```
...to only run tests for the Modal components
  • Loading branch information
unekinn committed Oct 8, 2024
1 parent 1898de7 commit 13c3433
Show file tree
Hide file tree
Showing 41 changed files with 799 additions and 301 deletions.
5 changes: 5 additions & 0 deletions .changeset/slimy-buttons-train.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@digdir/designsystemet-css': patch
---

Combobox: fix overflow on screens narrower than ~340px
14 changes: 14 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,15 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
# Necessary for Chromatic
fetch-depth: 0
- uses: ./.github/actions/gh-setup
- name: Build
run: yarn build
- name: Types
run: yarn types:react

- name: Test
run: yarn test
- name: 'Report Coverage'
Expand All @@ -36,8 +40,10 @@ jobs:
check_name: Unit Test Report
check_annotations: true
check_title_template: '{{FILE_NAME}} / {{TEST_NAME}}'

- name: Test CLI (create tokens, then build the theme)
run: yarn test:cli

- name: Install Playwright
run: npx playwright install --with-deps
- name: Build storybook
Expand All @@ -61,3 +67,11 @@ jobs:
check_name: Storybook Test Report
check_annotations: true
check_title_template: '{{FILE_NAME}} / {{TEST_NAME}}'

- name: Run Chromatic (visual and interaction tests)
uses: chromaui/action@latest
with:
workingDir: apps/storybook
projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
exitZeroOnChanges: true
autoAcceptChanges: '{main,next}'
20 changes: 20 additions & 0 deletions .yarn/patches/@storybook-addon-a11y-npm-8.3.4-1c07bc384c.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
diff --git a/dist/preview.js b/dist/preview.js
index 2ebc8126dbb113e1d43f39f70f0ef9016b96f53f..9392d52eff52f083ced74c93b68680dc718f741a 100644
--- a/dist/preview.js
+++ b/dist/preview.js
@@ -3,4 +3,4 @@
var previewApi = require('storybook/internal/preview-api');
var global = require('@storybook/global');

-var ADDON_ID="storybook/a11y";var RESULT=`${ADDON_ID}/result`,REQUEST=`${ADDON_ID}/request`,RUNNING=`${ADDON_ID}/running`,ERROR=`${ADDON_ID}/error`,MANUAL=`${ADDON_ID}/manual`,EVENTS={RESULT,REQUEST,RUNNING,ERROR,MANUAL};var{document}=global.global,channel=previewApi.addons.getChannel(),active=!1,activeStoryId,defaultParameters={config:{},options:{}},handleRequest=async(storyId,input)=>{input?.manual||await run(storyId,input??defaultParameters);},run=async(storyId,input=defaultParameters)=>{activeStoryId=storyId;try{if(!active){active=!0,channel.emit(EVENTS.RUNNING);let{default:axe}=await import('axe-core'),{element="#storybook-root",config,options={}}=input,htmlElement=document.querySelector(element);if(!htmlElement)return;axe.reset(),config&&axe.configure(config);let result=await axe.run(htmlElement,options),resultJson=JSON.parse(JSON.stringify(result));activeStoryId===storyId?channel.emit(EVENTS.RESULT,resultJson):(active=!1,run(activeStoryId));}}catch(error){channel.emit(EVENTS.ERROR,error);}finally{active=!1;}};channel.on(EVENTS.REQUEST,handleRequest);channel.on(EVENTS.MANUAL,run);
+var ADDON_ID="storybook/a11y";var RESULT=`${ADDON_ID}/result`,REQUEST=`${ADDON_ID}/request`,RUNNING=`${ADDON_ID}/running`,ERROR=`${ADDON_ID}/error`,MANUAL=`${ADDON_ID}/manual`,EVENTS={RESULT,REQUEST,RUNNING,ERROR,MANUAL};var{document}=global.global,channel=previewApi.addons.getChannel(),active=!1,activeStoryId,defaultParameters={config:{},options:{}},handleRequest=async(storyId,input)=>{input?.manual||await run(storyId,input??defaultParameters);},run=async(storyId,input=defaultParameters)=>{activeStoryId=storyId;try{if(!active){active=!0,channel.emit(EVENTS.RUNNING);let{default:axe}=await import('axe-core'),{element="#storybook-root",config,options={}}=input,htmlElement=document.querySelectorAll(element);if(!htmlElement)return;axe.reset(),config&&axe.configure(config);let result=await axe.run(htmlElement,options),resultJson=JSON.parse(JSON.stringify(result));activeStoryId===storyId?channel.emit(EVENTS.RESULT,resultJson):(active=!1,run(activeStoryId));}}catch(error){channel.emit(EVENTS.ERROR,error);}finally{active=!1;}};channel.on(EVENTS.REQUEST,handleRequest);channel.on(EVENTS.MANUAL,run);
diff --git a/dist/preview.mjs b/dist/preview.mjs
index 6e52b1b0b1c137af769e84a59ca32a0e8c91bbb7..dee1c98bfde648dd5f63037f16e954ef69913bb8 100644
--- a/dist/preview.mjs
+++ b/dist/preview.mjs
@@ -1,4 +1,4 @@
import { addons } from 'storybook/internal/preview-api';
import { global } from '@storybook/global';

-var ADDON_ID="storybook/a11y";var RESULT=`${ADDON_ID}/result`,REQUEST=`${ADDON_ID}/request`,RUNNING=`${ADDON_ID}/running`,ERROR=`${ADDON_ID}/error`,MANUAL=`${ADDON_ID}/manual`,EVENTS={RESULT,REQUEST,RUNNING,ERROR,MANUAL};var{document}=global,channel=addons.getChannel(),active=!1,activeStoryId,defaultParameters={config:{},options:{}},handleRequest=async(storyId,input)=>{input?.manual||await run(storyId,input??defaultParameters);},run=async(storyId,input=defaultParameters)=>{activeStoryId=storyId;try{if(!active){active=!0,channel.emit(EVENTS.RUNNING);let{default:axe}=await import('axe-core'),{element="#storybook-root",config,options={}}=input,htmlElement=document.querySelector(element);if(!htmlElement)return;axe.reset(),config&&axe.configure(config);let result=await axe.run(htmlElement,options),resultJson=JSON.parse(JSON.stringify(result));activeStoryId===storyId?channel.emit(EVENTS.RESULT,resultJson):(active=!1,run(activeStoryId));}}catch(error){channel.emit(EVENTS.ERROR,error);}finally{active=!1;}};channel.on(EVENTS.REQUEST,handleRequest);channel.on(EVENTS.MANUAL,run);
+var ADDON_ID="storybook/a11y";var RESULT=`${ADDON_ID}/result`,REQUEST=`${ADDON_ID}/request`,RUNNING=`${ADDON_ID}/running`,ERROR=`${ADDON_ID}/error`,MANUAL=`${ADDON_ID}/manual`,EVENTS={RESULT,REQUEST,RUNNING,ERROR,MANUAL};var{document}=global,channel=addons.getChannel(),active=!1,activeStoryId,defaultParameters={config:{},options:{}},handleRequest=async(storyId,input)=>{input?.manual||await run(storyId,input??defaultParameters);},run=async(storyId,input=defaultParameters)=>{activeStoryId=storyId;try{if(!active){active=!0,channel.emit(EVENTS.RUNNING);let{default:axe}=await import('axe-core'),{element="#storybook-root",config,options={}}=input,htmlElement=document.querySelectorAll(element);if(!htmlElement)return;axe.reset(),config&&axe.configure(config);let result=await axe.run(htmlElement,options),resultJson=JSON.parse(JSON.stringify(result));activeStoryId===storyId?channel.emit(EVENTS.RESULT,resultJson):(active=!1,run(activeStoryId));}}catch(error){channel.emit(EVENTS.ERROR,error);}finally{active=!1;}};channel.on(EVENTS.REQUEST,handleRequest);channel.on(EVENTS.MANUAL,run);
11 changes: 11 additions & 0 deletions apps/storybook/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@

##
## This file contains examples of useful environment variables for the Storybook project.
## To actually use them, create a .env file in this directory, and set the desired
## environment variables. That file is in .gitignore and will never be commited.
##

# To run Chromatic locally, set this var and replace <token> with the token from
# https://www.chromatic.com/manage?appId=66fe736b9d639fe6801bf130&view=configure
# ...under "Setup Chromatic with this project token".
CHROMATIC_PROJECT_TOKEN=<token>
1 change: 1 addition & 0 deletions apps/storybook/.storybook/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ const config: StorybookConfig = {
getAbsolutePath('@chromatic-com/storybook'),
getAbsolutePath('@storybook/addon-storysource'),
'@storybook/addon-themes',
'storybook-addon-pseudo-states',
],
staticDirs: ['../assets'],
framework: {
Expand Down
23 changes: 22 additions & 1 deletion apps/storybook/.storybook/preview.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,11 @@ import type { Preview } from '@storybook/react';
import type { LinkProps } from '@digdir/designsystemet-react';
import { Link, List, Paragraph, Table } from '@digdir/designsystemet-react';

import { customStylesDecorator } from '../story-utils/customStylesDecorator';
import { allModes, viewportWidths } from '../story-utils/modes';
import customTheme from './customTheme';

const viewports: Record<string, object> = {};
const viewportWidths = [320, 375, 576, 768, 992, 1200, 1440];

for (const width of viewportWidths) {
viewports[`${width}px`] = {
Expand Down Expand Up @@ -143,10 +144,30 @@ const preview: Preview = {
viewport: {
viewports,
},
chromatic: {
modes: {
mobile: allModes[320],
desktop: allModes[1200],
},
},
backgrounds: {
disable: true,
},
a11y: {
element: ['#storybook-root', '[data-floating-ui-portal]'],
config: {
rules: [
{
// Ignore the color-contrast rule for the ":active" pseudo-state
id: 'color-contrast',
selector:
'#storybook-root:not(.pseudo-active-all) *:not(.pseudo-active)',
},
],
},
},
},
decorators: [customStylesDecorator],
};

/* Add this back when https://github.com/storybookjs/storybook/issues/29189 is fixed */
Expand Down
4 changes: 2 additions & 2 deletions apps/storybook/.storybook/test-runner.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,15 @@ const config: TestRunnerConfig = {
*/

await configureAxe(page, {
// Apply story-level a11y rules
// Apply a11y rules set through parameters (global, component or story level)
rules: storyContext.parameters?.a11y?.config?.rules,
});

const isA11yDisabled = storyContext.parameters?.a11y?.disable === true;
if (!isA11yDisabled) {
await checkA11y(
page,
'#storybook-root',
storyContext.parameters?.a11y?.element ?? ['#storybook-root'],
{
detailedReport: true,
detailedReportOptions: {
Expand Down
8 changes: 8 additions & 0 deletions apps/storybook/chromatic.config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"$schema": "https://www.chromatic.com/config-file.schema.json",
"onlyChanged": true,
"projectId": "Project:66fe736b9d639fe6801bf130",
"storybookBaseDir": "apps/storybook",
"storybookBuildDir": "apps/storybook/dist",
"zip": true
}
5 changes: 3 additions & 2 deletions apps/storybook/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
"build": "storybook build -o ./dist",
"static-storybook": "npx http-server dist --port 6006 --silent",
"test-storybook": "rimraf test-report.xml && wait-on tcp:6006 && test-storybook --junit",
"run-and-test-storybook": "concurrently -k -s first -n 'SB,TEST' -c 'magenta,blue' 'yarn static-storybook' 'yarn test-storybook'"
"run-and-test-storybook": "concurrently -k -s first -n 'SB,TEST' -c 'magenta,blue' 'yarn static-storybook' 'yarn test-storybook'",
"chromatic": "npx chromatic"
},
"author": "",
"license": "ISC",
Expand All @@ -22,7 +23,7 @@
"@digdir/designsystemet-css": "workspace:^",
"@digdir/designsystemet-react": "workspace:^",
"@digdir/designsystemet-theme": "workspace:^",
"@storybook/addon-a11y": "^8.3.4",
"@storybook/addon-a11y": "patch:@storybook/addon-a11y@npm%3A8.3.4#~/.yarn/patches/@storybook-addon-a11y-npm-8.3.4-1c07bc384c.patch",
"@storybook/addon-essentials": "^8.3.4",
"@storybook/addon-interactions": "^8.3.4",
"@storybook/addon-links": "^8.3.4",
Expand Down
54 changes: 54 additions & 0 deletions apps/storybook/story-utils/customStylesDecorator.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
import type { Decorator } from '@storybook/react';

/**
* This decorator is used to customize the style of the root story element.
* It is useful for customizing the layout, or when you need to account
* for elements that would otherwise not be visible in Chromatic's visual snapshots.
*
* The decorator is added globally, and can be configured through `parameters.customStyles`
* at the meta or story level. E.g.
* ```ts
* parameters: {
* customStyles: {
* // These apply both in docs mode and story mode
* display: 'flex',
* gap: '8px',
* docs: {
* // These apply only when the story renders in a docs page
* height: '200px'
* },
* story: {
* // These apply only when the story is viewed individually
* height: '100vh'
* }
* }
* }
* ```
*
* By default, the decorator sets `overflow: hidden` so you can see in Storybook exactly
* what Chromatic's snapshot will be, and `padding: 1rem` to account for most overflowing
* elements like focus styles, badges etc.
*
* From Chromatic's documentation:
* > Snapshots can sometimes exclude outline and other focus styles because Chromatic
* > trims each snapshot to the dimensions of the root node of the story. To capture
* > those styles, wrap the story in a decorator that adds slight padding.
*/
export const customStylesDecorator: Decorator = (Story, ctx) => {
const { docs, story, ...style } = ctx.parameters.customStyles ?? {};

return (
<div
className='storybook-decorator'
style={{
overflow: 'hidden',
padding: '1rem',
...style,
...(ctx.viewMode === 'docs' && docs),
...(ctx.viewMode === 'story' && story),
}}
>
<Story />
</div>
);
};
6 changes: 6 additions & 0 deletions apps/storybook/story-utils/modes.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
import { fromPairs } from 'ramda';
export const viewportWidths = [320, 375, 576, 768, 992, 1200, 1440] as const;

export const allModes = fromPairs(
viewportWidths.map((width) => [width, { viewport: { width } }]),
);
147 changes: 147 additions & 0 deletions apps/storybook/story-utils/type-extensions.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
import type {} from '@storybook/types';
import type { ElementContext, Spec } from 'axe-core';
import type { configureAxe } from 'axe-playwright';
import type { CSSProperties } from 'react';

type AxeConfig = Parameters<typeof configureAxe>[1];

type ChromaticViewport = {
width?: number | `${string}px`;
height?: number | `${string}px`;
};

declare module '@storybook/types' {
type PseudoState =
| 'hover'
| 'active'
| 'focusVisible'
| 'focusWithin'
| 'focus'
| 'visited'
| 'link'
| 'target';

type PseudoValue = boolean | string | string[];

interface Parameters {
/**
* Set custom styling for the story's root element. The default styling is:
* ```css
* { overflow: hidden; padding: 1rem; }
* ```
*
* This is a custom parameter, implemented by `customStylesDecorator.ts`.
* */
customStyles?: CSSProperties & {
/** Styles that only apply when viewing a docs page */
docs?: CSSProperties;
/** Styles that only apply when viewing an individual story */
story?: CSSProperties;
};

/**
* Set the story layout.
*
* This is a standard Storybook parameter,
* [see the docs](https://storybook.js.org/docs/configure/story-layout)
*/
layout?: 'centered' | 'fullscreen' | 'padded';

/**
* Configure `@storybook/addon-a11y`. See [the documentation](https://storybook.js.org/addons/@storybook/addon-a11y)
*/
a11y?: {
disable?: boolean;
element?: ElementContext;
config?: Spec;
manual?: boolean;
};

/**
* Configure Chromatic. See [the documentation](https://www.chromatic.com/docs/config-with-story-params/).
*/
chromatic?: {
/** Disable visual snapshots at the component or story level */
disableSnapshot?: true;
/**
* By default, CSS animations are paused at the end of their animation cycle
* when tests are run in Chromatic. Setting this to false will pause animations
* at the first frame instead.
*/
pauseAnimationAtEnd?: false;
/** Delay in ms before running tests in Chromatic */
delay?: number;
/**
* Allows you to fine-tune the threshold for visual change between snapshots before
* Chromatic flags them. Must be a number from 0 to 1. 0 is the most accurate, while
* 1 is the least accurate.
*
* @default 0.063
*/
diffThreshold?: number;
/**
* Modes allow separate snapshots and baselines for a collection
* of parameters like viewport size, theme etc.
*/
modes?: Record<
string,
{
/**
* Disable a mode that has been enabled at a higher level.
* E.g. disable a global mode for a specific story.
**/
disable?: true;
/**
* The viewport to use.
*
* This parameter can either be an object with height and/or width (in px), or
* the name of one of the viewports configured in `parameters.viewports` in `.storybook/preview.tsx`
*/
viewport?: ChromaticViewport | string;
// ...any other globals from Storybook, addons or decorators which we want
// to use in modes can also be added here
}
>;
};

/**
* Toggle pseudo states. Supported states are listed in {@link PseudoState}.
* Read [Storybook Pseudo States documentation](https://github.com/chromaui/storybook-addon-pseudo-states)
* for more info.
*
* Each state can be toggled on/off:
* ```ts
* export const Hover = () => <Button>Label</Button>
* Hover.parameters = { pseudo: { hover: true } }
* ```
*
* You can also use CSS selectors to target the elements you want to enable the state for:
* ```ts
* export const Buttons = () => (
* <>
* <Button id="one">Hover</Button>
* <Button id="two">Hover focus</Button>
* <Button id="three">Hover focus active</Button>
* </>
* )
* Buttons.parameters = {
* pseudo: {
* hover: ["#one", "#two", "#three"],
* focus: ["#two", "#three"],
* active: "#three",
* },
* }
* ```
*/
pseudo?: {
/**
* If you need to render elements outside Storybook's root element, you can set
* rootSelector to override it. This is convenient for portals, dialogs, tooltips, etc.
* @default "#storybook-root"
*/
rootSelector?: string;
} & {
[K in PseudoState]?: PseudoValue;
};
}
}
Loading

0 comments on commit 13c3433

Please sign in to comment.