Unit Tester
The Evinced Unit Tester integrates with new or existing Jest environments to automatically detect accessibility issues. By adding just one unit test to your component library project, you can check that your component has the required accessibility semantics and that you have implemented the correct keyboard interactions. This will ensure that each tested component is fully compatible with assistive technologies (e.g. screen readers) and operable by keyboard. Since the JSDOM environment does not visually render the code, we recommend using downstream tools and/or manual review to evaluate visual requirements such as focus indication and color contrast. At the conclusion of each test, a rich and comprehensive report is generated to easily track issues to resolution.
Interested in seeing this in action? Contact us to get started!
Prerequisites
- Node version 16 or higher
- A testing framework like Jest or Vitest is installed and configured in your project. For running tests with other testing frameworks, please contact us.
- Jest version >= 25 is required for Jest users.
- Testing Library for your relevant framework is installed in your project. This is required to simulate user interactions in your tests.
- JSDOM version >= 15.2.1 is supported. If you encounter issues with JSDOM, please contact us.
Get Started
The following section will guide you through the process of setting up the Evinced Unit Tester in your project. You might also be interested in the Common Troubleshooting section.
Add EvincedUT as a dependency
In your project directory install Unit Tester using npm or yarn. The Unit Tester package is not publicly available, and requires a token. Contact us to get started!
1// This will only be available after you received an access token to the Evinced scope2npm install "@evinced/unit-tester"
Import Unit Tester into your test files
In your test file, import Unit Tester and initialize the EvincedUT object.
1import EvincedUT from "@evinced/unit-tester";
Import EvincedUT into your test environment
Instead of importing EvincedUT into each test file, you can make the EvincedUT object available globally in your test environment. Simply add the following lines to your Jest setup file.
1import EvincedUT from "@evinced/unit-tester";23Object.defineProperty(global, "EvincedUT", {4 value: EvincedUT,5});
If your Jest configuration does not include a setup file, you can create one by adding the following line to your Jest configuration file. Make sure to use the setupFilesAfterEnv
configuration option as the Evinced library must be initialized after Jest is loaded.
1setupFilesAfterEnv: ["<rootDir>/setup-jest.js"],
NOTE: If your eslint rules raise a no-undef
error on the EvincedUT global object, you can add the following lines to your eslint.rc
:
1globals: {2 EvincedUT: "readonly";3}
Authentication
In order to write and run tests with the Evinced Unit Tester, you must set up authentication. The unit tester supports two modes of authentication. User authentication is intended for test authors, it enables the user to write and run tests. Anonymous authentication is intended for test runners and CI environments, it enables the user to run tests.
User authentication
To set up user authentication, you must run the login command from your project root. After installing the Evinced Unit Tester, run the following command in your project directory:
1$ npx --package=@evinced/unit-tester login
This will open a browser window where you will be asked to confirm the login code which will be printed to the terminal.
1Opening the browser to continue the login process, your user code is ABCD-EFGH.2If the browser does not open automatically, please open it manually and navigate3to https://auth.evinced.com/activate?user_code=ABCD-EFGH
After confirming the login code, you will be asked to log in to your Evinced account. If you do not have an Evinced account, contact the administrator of your Evinced organization or contact us to get started.
Once you have logged in, the login command will print a success message to the terminal.
1Login successful
You can now write and run tests with the Evinced Unit Tester. Each time your authentication token expires, the unit tester will automatically refresh it, you should not need to run the login command again. If you want to change the user that is logged in, you must delete the saved user token. To do this, run the following command from your project root:
1rm .cache/userToken
To ensure that your authentication token is not accidentally committed to your source control, you should add the .cache
directory to your .gitignore
file.
Anonymous authentication
Anonymous authentication is intended for test runners and CI environments. It enables the user to run tests without having to log in to an Evinced account. In order to user anonymous authentication, you must find your evinced service account id and secret. Contact the administrator of your Evinced organization or contact us if you do not have these. There are two ways to set up anonymous authentication:
- You can set the
EVINCED_SERVICE_ACCOUNT_ID
andEVINCED_SERVICE_ACCOUNT_SECRET
environment variables. - You can call the
configure
method exposed by the unit tester package before running your tests, for instance in your Jest setup file.
1import EvincedUT, { configure } from "@evinced/unit-tester";23configure({4 serviceAccountId: "your-service-account-id",5 serviceAccountSecret: "your-service-account-secret",6});
Once you have set up anonymous authentication, you can run tests with the Evinced Unit Tester without having to log in to an Evinced account. An authentication token will be saved to your .cache
directory, you should not commit this to your source control.
Your first test
Here are simple examples of how to add an Evinced accessibility scan to a test. Please note the inline comments that give detail on each test step.
Vanilla JS
1it("Evinced unit tester basic example", async () => {2 // Create your component3 const myComponent = document.createElement("div");4 myComponent.setAttribute("role", "button");5 myComponent.setAttribute("class", "custom-button");6 document.body.appendChild(myComponent);78 // Scan for a11y issues and assert on the results9 const results = await EvincedUT.analyzeButton(myComponent);10 expect(results).toHaveNoFailures();11});
React
1import { screen, render } from "@testing-library/react";23it("Evinced unit tester basic example", async () => {4 // Render your component5 render(<MyButton>Click Me!</MyButton>);6 const myComponent = screen.getByRole("button");78 // Scan for a11y issues and assert on the results9 const results = await EvincedUT.analyzeButton(myComponent);10 expect(results).toHaveNoFailures();11});
Configuration
The Evinced Unit Tester exposes a global configuration object that can be used to configure the behavior of the unit tester. You can update a specific configuration setting by passing a Config object to the configure function in your test setup file. The object should contain the key of the setting you want to update and its new value. Here's an example of how you can update configuration setting:
1import EvincedUT, { configure } from "@evinced/unit-tester";23configure({ reportSkippedTests: true, detailedMembersReport: false });
Additionally, you can set the configuration options inline in your test file. This is useful when you want to tailor the configuration for individual analyses. Here's an example of how you can set the configuration options inline:
1const results = await EvincedUT.analyzeButton(myComponent, {2 containerElement: document.querySelector("#my-container"),3 extendedKeyboardTests: true,4});
The following configuration options are available:
Option | Type | Default | Description |
---|---|---|---|
reportSkippedTests | boolean | false | If set to true , the unit tester will report which tests were skipped, as well as passed INFO level tests. |
debugLevel | boolean / 'dom' / 'roles' | false | If set to dom or true , the unit tester report will include the DOM representation. If set to roles , the report will include all elements with roles found in the DOM. |
detailedMembersReport | boolean | true | If set to true , the unit tester's report will contain detailed information about the members of the tested objects, instead of uniting the results. This is useful for debugging failing tests in complex components. |
extendedKeyboardTests | boolean | false | If set to true , the analysis will be extended to include all the optional keyboard interactions. This configuration is not fully supported yet. |
sendUsageAnalytics | boolean | true | If set to true , the unit tester will send usage analytics to Evinced. |
proxy | string / URL / Proxy | undefined | Optional proxy configuration to use for the unit tester. May be string, URL or Proxy object. You can find more about configuring proxies here. |
serviceAccountId | string | EVINCED_SERVICE_ACCOUNT_ID environment variable | The service account id to use for anonymous authentication |
serviceAccountSecret | string | EVINCED_SERVICE_ACCOUNT_SECRET environment variable | The service account secret to use for anonymous authentication |
suppressAnonymousAuthenticationWarning | boolean | CI environment variable | If set to true , the unit tester will not warn if anonymous authentication is used. This is useful for CI environments where anonymous authentication is the only option. |
Components summary
Component | API Method | Knowledge Base |
---|---|---|
Accordion | analyzeAccordion | Accordion entry |
Alert | analyzeAlert | Alert entry |
Breadcrumb | analyzeBreadcrumb | Breadcrumb entry |
Button | analyzeButton | Button entry |
Carousel | analyzeCarousel | Carousel entry |
Checkbox | analyzeCheckbox | Checkbox entry |
Combobox | analyzeCombobox | Combobox entry |
Disclosure | analyzeDisclosure | Disclosure entry |
Feed | analyzeFeed | Feed entry |
Link | analyzeLink | Link entry |
Listbox | analyzeListbox | Listbox entry |
Meter | analyzeMeter | Meter entry |
Modal | analyzeModal | Modal entry |
Site Navigation | analyzeSiteNavigation | Site Navigation entry |
Radio Group | analyzeRadioGroup | Radio Group entry |
Slider | analyzeSlider | Slider entry |
Spin Button | analyzeSpinButton | Spin Button entry |
Multi Thumb Slider | analyzeSliderMultiThumb | Multi Thumb Slider entry |
Switch | analyzeSwitch | Switch entry |
Table | analyzeTable | Table entry |
Tab List | analyzeTabList | Tab List entry |
Toggle Button | analyzeToggleButton | Toggle Button entry |
API
about EvincedUT.analyzeComponent
Overview
Scans the specified component. An array is returned containing the results of each accessibility check that was performed. This is the generic API signature for most components. For more specific component types, see the following methods.
Parameters
locator: TargetLocator
- The locator for the component to be scanned. This can be a CSS selector string, an element, or a function that returns an element.options?: BaseComponentOptions
- An optional object containing additional options for the scan.containerElement?: Element
- An optional container used as the root of the document instead of the document elementallowTargetReset?: boolean
- An optional parameter that allows Unit Tester to pick the target element. Useful when the component is wrapped inside several containers.userEventOptions?: @testing-library/user-event Options
- An optional object containing options to be passed to the user-event library
Example
In the following example, a selector is passed to locate the switch component. The containerElement
option is used to specify the root container for the scan. The userEventOptions
option is used to specify the delay between user interactions in the test.
1const results = await EvincedUT.analyzeSwitch("#my-switch", {2 containerElement: "#my-container",3 userEventOptions: { delay: 10 },4});5expect(results).toHaveNoFailures();
Default options
1{2 containerElement: document.documentElement;3 allowTargetReset: false;4 userEventOptions: undefined;5}
Return value
1Promise<AnalysisResult[]>;
analyzeAccordion
Overview
Scans the specified accordion container. An array is returned containing the results of each accessibility check that was performed.
This test uses the signature specified in the analyzeDisclosure section.
analyzeAlert
Overview
Scans the specified alert element. Make sure the element is visible before scanning.
An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeBreadcrumb
Overview
Scans the specified breadcrumb container. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeButton
Overview
Scans the specified button. An array is returned containing the results of each accessibility check that was performed.
Parameters
locator: TargetLocator
- The locator for the component to be scanned. This can be a CSS selector string, an element, or a function that returns an element.options?: AnalyzeButtonOptions
- An optional object containing additional options for the scan.wasActivatedCallback?: () => boolean
- an optional boolean callback to evaluate whether the element was activated after interactions- The other parameters of the BaseComponentOptions type.
Example
1const checkActivation = () => {2 return window.wasButtonActivated === true;3};45const results = await EvincedUT.analyzeButton('div[role="button"]', {6 wasActivatedCallback: checkActivation,7});8expect(results).toHaveNoFailures();
analyzeCheckbox
Overview
Scans the specified checkbox. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeCarousel
Overview
Scans the specified carousel. An array is returned containing the results of each accessibility check that was performed.
Parameters
The carousel scan requires additional options. It is mandatory to specify the automatic rotation option, and either the next/previous slide controls or the slider picker control.
If your implementation of carousel does not include any controls, it is considered as an accessibility violation.
locator: TargetLocator
- The locator for the component to be scanned. This can be a CSS selector string, an element, or a function that returns an element.options: AnalyzeCarouselOptions
- A mandatory object containing additional options for the scan.automaticRotation: boolean
- a boolean parameter that determines whether the carousel automatically rotates upon initializationnextSlideControl?: TargetLocator
an optional target locator for the next slide controlprevSlideControl?: TargetLocator
an optional target locator for the previous slide controlpickerControl?: TargetLocator
an optional target locator for the slider picker controlrotationControl?: TargetLocator
an optional target locator for the rotation control- The other parameters of the BaseComponentOptions type.
Example
1const checkActivation = () => {2 return window.wasButtonActivated === true;3};45const results = await EvincedUT.analyzeCarousel("div.carousel", {6 automaticRotation: false,7 nextSlideControl: "div.next-slide",8 prevSlideControl: "div.prev-slide",9 rotationControl: "div.rotation-control",10});11expect(results).toHaveNoFailures();
analyzeCombobox
Overview
Scans the specified combobox element. Currently only supports the listbox combobox pattern.
An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeDisclosure
Overview
Scans the specified disclosure button. An array is returned containing the results of each accessibility check that was performed.
Parameters
locator: TargetLocator
- The locator for the disclosure button to be scanned. This can be a CSS selector string, an element, or a function that returns an element.options?: AnalyzeDisclosureOptions
- An optional object containing additional options for the scan.evalEscape?: boolean
- An optional boolean parameter that determines whether to evaluate the escape key accessibility. Default is false.- The other parameters of the BaseComponentOptions type.
Example
1const results = await EvincedUT.analyzeDisclosure('div[role="button"]', {2 evalEscape: true,3});4expect(results).toHaveNoFailures();
analyzeFeed
Overview
Scans the specified feed element. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeLink
Overview
Scans the specified link. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeListbox
Overview
Scans the specified listbox element. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeMeter
Overview
Scans the specified meter element. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeModal
Overview
Scans the specified modal dialog and launcher element. An array is returned containing the results of each accessibility check that was performed.
Parameters
launcherLocator: TargetLocator
- The locator for the modal launcher. This can be a CSS selector string, an element, or a function that returns an element.modalLocator: TargetLocator
- The locator for the modal container. This can be a CSS selector string, an element, or a function that returns an element.options?: BaseComponentOptions
- An optional object containing additional options for the scan.
Example
1const results = await EvincedUT.analyzeModal(2 buttonElement,3 'div[role="dialog"]'4);5expect(results).toHaveNoFailures();
analyzeRadioGroup
Overview
Scans the specified radio group. An array is returned containing the results of each accessibility check that was performed.
Parameters
locator: TargetLocator
- The locator for the component to be scanned. This can be a CSS selector string, an element, or a function that returns an element.options?: AnalyzeRadioGroupOptions
- An optional object containing additional options for the scan.isToolbar?: boolean
- an optional boolean parameter to specify if the radio group is part of a toolbar.- The other parameters of the BaseComponentOptions type.
Example
1const results = await EvincedUT.analyzeRadioGroup('div[role="radiogroup"]', {2 isToolbar: true,3});4expect(results).toHaveNoFailures();
Default options
1{2 containerElement: document.documentElement;3 isToolbar: false;4}
analyzeSiteNavigation
Overview
Scans the specified site navigation container. Supports all kinds of site navigation components.
An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeSlider
Overview
Scans the specified slider. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeSpinButton
Overview
Scans the specified Spin button. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeSliderMultiThumb
Overview
Scans the specified multi thumb slider container. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeSwitch
Overview
Scans the specified switch. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeTable
Overview
Scans the specified table. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeTabList
Overview
Scans the specified tablist container. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
analyzeToggleButton
Overview
Scans the specified toggle button, which is a subcategory of the button pattern. An array is returned containing the results of each accessibility check that was performed.
This test uses the generic signature specified in the About analyzeComponent section.
Assertions
toHaveNoFailures
Asserts that the scan results contain no failures. If the scan results contain failures, the test will fail and the results will be printed to the console. This assertion is useful when you want to ensure that your component has the required accessibility semantics and that you have implemented the correct keyboard interactions.
Example
1const results = await EvincedUT.analyzeButton('div[role="button"]');2expect(results).toHaveNoFailures();
toHaveNoWarnings
Asserts that the scan results contain no warnings. If the scan results contain warnings, the test will fail and the results will be printed to the console. This assertion is a strict version of the toHaveNoFailures
assertion. It is useful when you want to ensure that your component have implemented recommended accessibility semantics and keyboard interactions.
Example
1const results = await EvincedUT.analyzeButton('div[role="button"]');2expect(results).toHaveNoWarnings();
toHaveResult
Asserts that the scan results contain a specific result. If the scan results do not contain the specified result, the test will fail and the results will be printed to the console. This assertion is useful when you want to ensure that your component complies with specific tests.
Parameters
expected: Partial<AnalysisResult>
- The expected result. The assertion will pass if the scan results contain a result that matches the expected result.
Example
1const results = await EvincedUT.analyzeButton('div[role="button"]');2expect(results).toHaveResult({3 component: "Button",4 test: "button name",5 pass: true,6});
Typings
TargetLocator
1string | HTMLElement | SVGElement | (() => HTMLElement | SVGElement);
AnalysisResult
1{2 // The type of component that was scanned3 component: string;45 // The test that was run6 test: string;78 // The result of the test9 pass: boolean;1011 // The message associated with the test12 message: string;1314 // The element that were related to the test message15 members?: string[];1617 // The actions that were related to the test message18 actions?: string[];1920 // The type of the result (PASS, FAIL, WARN or SKIP)21 type?: string;22}
BaseComponentOptions
1{2 // The container element to use as the root of the document3 containerElement?: Element;4 // An optional parameter that allows Unit Tester to pick the target element.5 // Useful when the component is wrapped inside several containers.6 allowTargetReset?: boolean;7 // An optional object containing options to be passed to the user-event library8 // https://testing-library.com/docs/user-event/options/9 userEventOptions?: @testing-library/user-event Options;10}
Proxy
Configuring a proxy might be necessary when running the unit tester in an isolated corporate environment. To configure a proxy you can set the proxy URL in one of the env vars HTTPS_PROXY
or EVINCED_HTTPS_PROXY
in the format of http(s)://user:pass@proxy.url:port/
, where user and pass are optional if your proxy requires it. You could also set the proxy URL using the configure
method, which supports a URL string in the same format as the environment variables, a URL object or a Proxy
object with the following properties:
1/**2 * Proxy configuration for the unit tester3 * @protocol: The protocol of the proxy, either `http` or `https`4 * @host: The host of the proxy5 * @port: The port of the proxy6 * @auth: Optional authentication details for the proxy7 * @username: The username to use for the proxy8 * @password: The password to use for the proxy9 */10{11 protocol: 'http' | 'https'12 host: string;13 port: number;14 auth?: {15 username: string;16 password: string;17 }18}
Troubleshooting common issues
Cannot install the package using npm install
, receiving a 404 error
If you are unable to install the package using npm install
, and receive the following error npm ERR! 404 Not Found
, you may need to add the Evinced package registry token to your .npmrc
file. You can do this by running the following command from your project root. Make sure to insert your token:
1echo "@evinced:registry=https://evinced.jfrog.io/artifactory/api/npm/restricted-npm/2//evinced.jfrog.io/artifactory/api/npm/restricted-npm/:_authToken=<YOUR_TOKEN_HERE>" >> ~/.npmrc & npm i @evinced/unit-tester
I'm unable to authenticate using the login command
There are a few reasons why you might be unable to authenticate using the login command. If you are having trouble, you can contact us for assistance. The most common reason for authentication issues is network connectivity problems. Make sure that your organization network is not blocking the authentication process. You can run the following snippet to verify that you are able to reach the Evinced authentication server with node js:
1const https = require("https");23https.get(4 "https://auth.evinced.com/.well-known/openid-configuration",5 (res) => {6 console.log("Got response: " + res.statusCode);7 res.on("data", (chunk) => {8 console.log("BODY: " + chunk);9 }).on("error", (err) => {10 console.error(err);11 });12 }13);
Here is a list of endpoint URLs that the Unit Tester uses, make sure that your network allows access to these URLs:
https://auth.evinced.com
https://app.evinced.com
https://ingestion.evinced.com
If your corporate network is blocking the Evinced authentication server, you might want to set up a proxy. You can find more information on how to configure a proxy here.
I'm having trouble configuring my unit tests environment
If you are having trouble configuring your unit tests environment, you can contact us for assistance. A helpful step could be to set up a new testing environment, with clean configurations, like in this example.
I'm getting a Jest encountered an unexpected token
error
If you are getting a Jest encountered an unexpected token
error, it is likely to happen if your project wasn't previously configured to run unit tests on components. Often, this error is caused by the Jest environment not being set up to handle the ES6 syntax, or a failure to transform images and css files. You can fix this issue by adding the following content to your Jest configuration:
1module.exports = {2 ..., // Your Jest configuration3 moduleNameMapper: {4 '\\.(css|less|scss|sass)$': 'PATH/TO/EMPTY_FILE.js',5 '\\.(jpg|jpeg|png|gif|eot|otf|webp|svg|ttf|woff|woff2|mp4)$':6 'PATH/TO/EMPTY_FILE.js'7 },8 // lodash is a dependency of many packages, and it's not transpiled by default9 transformIgnorePatterns: ['/node_modules/(?!lodash-es)/'],10};
The EMPTY_FILE.js
file in this examples is an empty file, that exports an empty object like so:
1module.exports = {};
The tests seems inaccurate, I'm sure that my component has a role
The Unit Tester uses the specified selector to locate the component and then scans the component for accessibility issues. If the selector was incorrect, the scan results may not be accurate. Make sure that the selector is correct and that the component is rendered before scanning.
You can use the test configuration debugLevel: 'roles'
to see the roles of the elements in the DOM and their selectors. You can also use the test configuration debugLevel: 'dom'
to print the current DOM representation.
Also, you can use the allowTargetReset
option to allow Unit Tester to pick the target element. This is useful when the component is wrapped inside several containers.
My component is disabled and the scan issues a warning. How can I suppress this warning?
The Unit Tester will issue a warning if it detects that the component is disabled. If you want to suppress this warning, you can use the toHaveNoFailures
assertion instead of the toHaveNoWarnings
assertion. The toHaveNoFailures
assertion will pass if the scan results contain no failures, and will ignore warnings.
A certain test is failing, but I prefer to skip it instead of fixing it
If you want to skip a certain test, you can always filter out the test from the AnalysisResult
array that is returned by the Unit Tester. You can utilize js and filter tests by any property of the AnalysisResult
object. For example:
1const results = await EvincedUT.analyzeButton("button[aria-disabled]");2// We want the disabled button to remain on the focus sequence for discoverability3const filteredResults = results.filter(4 (result) =>5 result.message !==6 "Button element is a native html control so it's best to use disabled instead of aria-disabled"7);8expect(filteredResults).toHaveNoFailures();
I get a weird error - SyntaxError: slot):not([inert] is not a valid selector
This error is caused by the nwsapi
package, which is a dependency of the jsdom
package. The nwsapi
package is used to parse CSS selectors, and in certain versions of the package, it does not support the :not
pseudo-class. This issue has been fixed in the latest version of the nwsapi
package, so you can fix this issue by updating the nwsapi
package to the latest version. You can do this by running the following command from your project root:
1npm install nwsapi@latest
Support
Please feel free to reach out to support@evinced.com with any questions.