Unit Tester

The Evinced Unit Tester integrates with new or existing Jest environments to automatically detect accessibility issues. By adding just one unit test to your component library project, you can check that your component has the required accessibility semantics and that you have implemented the correct keyboard interactions. This will ensure that each tested component is fully compatible with assistive technologies (e.g. screen readers) and operable by keyboard. Since the JSDOM environment does not visually render the code, we recommend using downstream tools and/or manual review to evaluate visual requirements such as focus indication and color contrast. At the conclusion of each test, a rich and comprehensive report is generated to easily track issues to resolution.

Interested in seeing this in action? Contact us to get started!

Prerequisites

• Node version 16 or higher
• A testing framework like Jest or Vitest is installed and configured in your project. For running tests with other testing frameworks, please contact us.
  • Jest version >= 25 is required for Jest users.
• Testing Library for your relevant framework is installed in your project. This is required to simulate user interactions in your tests.
• JSDOM version >= 15.2.1 is supported. If you encounter issues with JSDOM, please contact us.

Get Started

The following section will guide you through the process of setting up the Evinced Unit Tester in your project. You might also be interested in the Common Troubleshooting section.

Add EvincedUT as a dependency

In your project directory install Unit Tester using npm or yarn. The Unit Tester package is not publicly available, and requires a token. Contact us to get started!

1// This will only be available after you received an access token to the Evinced scope
2npm install "@evinced/unit-tester"

Import Unit Tester into your test files

In your test file, import Unit Tester and initialize the EvincedUT object.

1import EvincedUT from "@evinced/unit-tester";

Import EvincedUT into your test environment

Instead of importing EvincedUT into each test file, you can make the EvincedUT object available globally in your test environment. Simply add the following lines to your Jest setup file.

1import EvincedUT from "@evinced/unit-tester";
2
3Object.defineProperty(global, "EvincedUT", {
4    value: EvincedUT,
5});

If your Jest configuration does not include a setup file, you can create one by adding the following line to your Jest configuration file. Make sure to use the setupFilesAfterEnv configuration option as the Evinced library must be initialized after Jest is loaded.

1setupFilesAfterEnv: ["<rootDir>/setup-jest.js"],

NOTE: If your eslint rules raise a no-undef error on the EvincedUT global object, you can add the following lines to your eslint.rc:

1globals: {
2    EvincedUT: "readonly";
3}

Authentication

In order to write and run tests with the Evinced Unit Tester, you must set up authentication. The unit tester supports two modes of authentication. User authentication is intended for test authors, it enables the user to write and run tests. Anonymous authentication is intended for test runners and CI environments, it enables the user to run tests.

User authentication

To set up user authentication, you must run the login command from your project root. After installing the Evinced Unit Tester, run the following command in your project directory:

1$ npx --package=@evinced/unit-tester login

This will open a browser window where you will be asked to confirm the login code which will be printed to the terminal.

1Opening the browser to continue the login process, your user code is ABCD-EFGH.
2If the browser does not open automatically, please open it manually and navigate
3to https://auth.evinced.com/activate?user_code=ABCD-EFGH

After confirming the login code, you will be asked to log in to your Evinced account. If you do not have an Evinced account, contact the administrator of your Evinced organization or contact us to get started.

Once you have logged in, the login command will print a success message to the terminal.

1Login successful

You can now write and run tests with the Evinced Unit Tester. Each time your authentication token expires, the unit tester will automatically refresh it, you should not need to run the login command again. If you want to change the user that is logged in, you must delete the saved user token. To do this, run the following command from your project root:

1rm .cache/userToken

To ensure that your authentication token is not accidentally committed to your source control, you should add the .cache directory to your .gitignore file.

Anonymous authentication

Anonymous authentication is intended for test runners and CI environments. It enables the user to run tests without having to log in to an Evinced account. In order to user anonymous authentication, you must find your evinced service account id and secret. Contact the administrator of your Evinced organization or contact us if you do not have these. There are two ways to set up anonymous authentication:

1. You can set the EVINCED_SERVICE_ACCOUNT_ID and EVINCED_SERVICE_ACCOUNT_SECRET environment variables.
2. You can call the configure method exposed by the unit tester package before running your tests, for instance in your Jest setup file.

1import EvincedUT, { configure } from "@evinced/unit-tester";
2
3configure({
4    serviceAccountId: "your-service-account-id",
5    serviceAccountSecret: "your-service-account-secret",
6});

Once you have set up anonymous authentication, you can run tests with the Evinced Unit Tester without having to log in to an Evinced account. An authentication token will be saved to your .cache directory, you should not commit this to your source control.

Your first test

Here are simple examples of how to add an Evinced accessibility scan to a test. Please note the inline comments that give detail on each test step.

Vanilla JS

1it("Evinced unit tester basic example", async () => {
2    // Create your component
3    const myComponent = document.createElement("div");
4    myComponent.setAttribute("role", "button");
5    myComponent.setAttribute("class", "custom-button");
6    document.body.appendChild(myComponent);
7
8    // Scan for a11y issues and assert on the results
9    const results = await EvincedUT.analyzeButton(myComponent);
10    expect(results).toHaveNoFailures();
11});

React

1import { screen, render } from "@testing-library/react";
2
3it("Evinced unit tester basic example", async () => {
4    // Render your component
5    render(<MyButton>Click Me!</MyButton>);
6    const myComponent = screen.getByRole("button");
7
8    // Scan for a11y issues and assert on the results
9    const results = await EvincedUT.analyzeButton(myComponent);
10    expect(results).toHaveNoFailures();
11});

Configuration

The Evinced Unit Tester exposes a global configuration object that can be used to configure the behavior of the unit tester. You can update a specific configuration setting by passing a Config object to the configure function in your test setup file. The object should contain the key of the setting you want to update and its new value. Here's an example of how you can update configuration setting:

1import EvincedUT, { configure } from "@evinced/unit-tester";
2
3configure({ reportSkippedTests: true, detailedMembersReport: false });

Additionally, you can set the configuration options inline in your test file. This is useful when you want to tailor the configuration for individual analyses. Here's an example of how you can set the configuration options inline:

1const results = await EvincedUT.analyzeButton(myComponent, {
2    containerElement: document.querySelector("#my-container"),
3    extendedKeyboardTests: true,
4});

The following configuration options are available:

Components summary

API

about EvincedUT.analyzeComponent

Overview

Scans the specified component. An array is returned containing the results of each accessibility check that was performed. This is the generic API signature for most components. For more specific component types, see the following methods.

Parameters

• locator: TargetLocator - The locator for the component to be scanned. This can be a CSS selector string, an element, or a function that returns an element.
• options?: BaseComponentOptions - An optional object containing additional options for the scan.
  • containerElement?: Element - An optional container used as the root of the document instead of the document element
  • allowTargetReset?: boolean - An optional parameter that allows Unit Tester to pick the target element. Useful when the component is wrapped inside several containers.
  • userEventOptions?: @testing-library/user-event Options - An optional object containing options to be passed to the user-event library

Example

In the following example, a selector is passed to locate the switch component. The containerElement option is used to specify the root container for the scan. The userEventOptions option is used to specify the delay between user interactions in the test.

1const results = await EvincedUT.analyzeSwitch("#my-switch", {
2    containerElement: "#my-container",
3    userEventOptions: { delay: 10 },
4});
5expect(results).toHaveNoFailures();

Default options

1{
2    containerElement: document.documentElement;
3    allowTargetReset: false;
4    userEventOptions: undefined;
5}

Return value

1Promise<AnalysisResult[]>;

See AnalysisResult

analyzeAccordion

Overview

Scans the specified accordion container. An array is returned containing the results of each accessibility check that was performed.

This test uses the signature specified in the analyzeDisclosure section.

analyzeAlert

Overview

Scans the specified alert element. Make sure the element is visible before scanning.

An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeBreadcrumb

Overview

Scans the specified breadcrumb container. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeButton

Overview

Scans the specified button. An array is returned containing the results of each accessibility check that was performed.

Parameters

• locator: TargetLocator - The locator for the component to be scanned. This can be a CSS selector string, an element, or a function that returns an element.
• options?: AnalyzeButtonOptions - An optional object containing additional options for the scan.
  • wasActivatedCallback?: () => boolean - an optional boolean callback to evaluate whether the element was activated after interactions
  • The other parameters of the BaseComponentOptions type.

Example

1const checkActivation = () => {
2    return window.wasButtonActivated === true;
3};
4
5const results = await EvincedUT.analyzeButton('div[role="button"]', {
6    wasActivatedCallback: checkActivation,
7});
8expect(results).toHaveNoFailures();

analyzeCheckbox

Overview

Scans the specified checkbox. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeCarousel

Overview

Scans the specified carousel. An array is returned containing the results of each accessibility check that was performed.

Parameters

The carousel scan requires additional options. It is mandatory to specify the automatic rotation option, and either the next/previous slide controls or the slider picker control.

If your implementation of carousel does not include any controls, it is considered as an accessibility violation.

• locator: TargetLocator - The locator for the component to be scanned. This can be a CSS selector string, an element, or a function that returns an element.
• options: AnalyzeCarouselOptions - A mandatory object containing additional options for the scan.
  • automaticRotation: boolean - a boolean parameter that determines whether the carousel automatically rotates upon initialization
  • nextSlideControl?: TargetLocator an optional target locator for the next slide control
  • prevSlideControl?: TargetLocator an optional target locator for the previous slide control
  • pickerControl?: TargetLocator an optional target locator for the slider picker control
  • rotationControl?: TargetLocator an optional target locator for the rotation control
  • The other parameters of the BaseComponentOptions type.

Example

1const checkActivation = () => {
2    return window.wasButtonActivated === true;
3};
4
5const results = await EvincedUT.analyzeCarousel("div.carousel", {
6    automaticRotation: false,
7    nextSlideControl: "div.next-slide",
8    prevSlideControl: "div.prev-slide",
9    rotationControl: "div.rotation-control",
10});
11expect(results).toHaveNoFailures();

analyzeCombobox

Overview

Scans the specified combobox element. Currently only supports the listbox combobox pattern.

An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeDisclosure

Overview

Scans the specified disclosure button. An array is returned containing the results of each accessibility check that was performed.

Parameters

• locator: TargetLocator - The locator for the disclosure button to be scanned. This can be a CSS selector string, an element, or a function that returns an element.
• options?: AnalyzeDisclosureOptions - An optional object containing additional options for the scan.
  • evalEscape?: boolean - An optional boolean parameter that determines whether to evaluate the escape key accessibility. Default is false.
  • The other parameters of the BaseComponentOptions type.

Example

1const results = await EvincedUT.analyzeDisclosure('div[role="button"]', {
2    evalEscape: true,
3});
4expect(results).toHaveNoFailures();

analyzeFeed

Overview

Scans the specified feed element. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeLink

Overview

Scans the specified link. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeListbox

Overview

Scans the specified listbox element. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeMeter

Overview

Scans the specified meter element. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeModal

Overview

Scans the specified modal dialog and launcher element. An array is returned containing the results of each accessibility check that was performed.

Parameters

• launcherLocator: TargetLocator - The locator for the modal launcher. This can be a CSS selector string, an element, or a function that returns an element.
• modalLocator: TargetLocator - The locator for the modal container. This can be a CSS selector string, an element, or a function that returns an element.
• options?: BaseComponentOptions - An optional object containing additional options for the scan.

Example

1const results = await EvincedUT.analyzeModal(
2    buttonElement,
3    'div[role="dialog"]'
4);
5expect(results).toHaveNoFailures();

analyzeRadioGroup

Overview

Scans the specified radio group. An array is returned containing the results of each accessibility check that was performed.

Parameters

• locator: TargetLocator - The locator for the component to be scanned. This can be a CSS selector string, an element, or a function that returns an element.
• options?: AnalyzeRadioGroupOptions - An optional object containing additional options for the scan.
  • isToolbar?: boolean - an optional boolean parameter to specify if the radio group is part of a toolbar.
  • The other parameters of the BaseComponentOptions type.

Example

1const results = await EvincedUT.analyzeRadioGroup('div[role="radiogroup"]', {
2    isToolbar: true,
3});
4expect(results).toHaveNoFailures();

Default options

1{
2    containerElement: document.documentElement;
3    isToolbar: false;
4}

analyzeSiteNavigation

Overview

Scans the specified site navigation container. Supports all kinds of site navigation components.

An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeSlider

Overview

Scans the specified slider. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeSpinButton

Overview

Scans the specified Spin button. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeSliderMultiThumb

Overview

Scans the specified multi thumb slider container. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeSwitch

Overview

Scans the specified switch. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeTable

Overview

Scans the specified table. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeTabList

Overview

Scans the specified tablist container. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeToggleButton

Overview

Scans the specified toggle button, which is a subcategory of the button pattern. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

Assertions

toHaveNoFailures

Asserts that the scan results contain no failures. If the scan results contain failures, the test will fail and the results will be printed to the console. This assertion is useful when you want to ensure that your component has the required accessibility semantics and that you have implemented the correct keyboard interactions.

Example

1const results = await EvincedUT.analyzeButton('div[role="button"]');
2expect(results).toHaveNoFailures();

toHaveNoWarnings

Asserts that the scan results contain no warnings. If the scan results contain warnings, the test will fail and the results will be printed to the console. This assertion is a strict version of the toHaveNoFailures assertion. It is useful when you want to ensure that your component have implemented recommended accessibility semantics and keyboard interactions.

Example

1const results = await EvincedUT.analyzeButton('div[role="button"]');
2expect(results).toHaveNoWarnings();

toHaveResult

Asserts that the scan results contain a specific result. If the scan results do not contain the specified result, the test will fail and the results will be printed to the console. This assertion is useful when you want to ensure that your component complies with specific tests.

Parameters

• expected: Partial<AnalysisResult> - The expected result. The assertion will pass if the scan results contain a result that matches the expected result.

Example

1const results = await EvincedUT.analyzeButton('div[role="button"]');
2expect(results).toHaveResult({
3    component: "Button",
4    test: "button name",
5    pass: true,
6});

Typings

TargetLocator

1string | HTMLElement | SVGElement | (() => HTMLElement | SVGElement);

AnalysisResult

1{
2    // The type of component that was scanned
3    component: string;
4
5    // The test that was run
6    test: string;
7
8    // The result of the test
9    pass: boolean;
10
11    // The message associated with the test
12    message: string;
13
14    // The element that were related to the test message
15    members?: string[];
16
17    // The actions that were related to the test message
18    actions?: string[];
19
20    // The type of the result (PASS, FAIL, WARN or SKIP)
21    type?: string;
22}

BaseComponentOptions

1{
2    // The container element to use as the root of the document
3    containerElement?: Element;
4    // An optional parameter that allows Unit Tester to pick the target element.
5    // Useful when the component is wrapped inside several containers.
6    allowTargetReset?: boolean;
7    // An optional object containing options to be passed to the user-event library
8    // https://testing-library.com/docs/user-event/options/
9    userEventOptions?: @testing-library/user-event Options;
10}

Proxy

Configuring a proxy might be necessary when running the unit tester in an isolated corporate environment. To configure a proxy you can set the proxy URL in one of the env vars HTTPS_PROXY or EVINCED_HTTPS_PROXY in the format of http(s)://user:pass@proxy.url:port/, where user and pass are optional if your proxy requires it. You could also set the proxy URL using the configure method, which supports a URL string in the same format as the environment variables, a URL object or a Proxy object with the following properties:

1/**
2 * Proxy configuration for the unit tester
3 * @protocol: The protocol of the proxy, either `http` or `https`
4 * @host: The host of the proxy
5 * @port: The port of the proxy
6 * @auth: Optional authentication details for the proxy
7 *  @username: The username to use for the proxy
8 *  @password: The password to use for the proxy
9 */
10{
11  protocol: 'http' | 'https'
12  host: string;
13  port: number;
14  auth?: {
15    username: string;
16    password: string;
17  }
18}

Troubleshooting common issues

Cannot install the package using npm install, receiving a 404 error

If you are unable to install the package using npm install, and receive the following error npm ERR! 404 Not Found, you may need to add the Evinced package registry token to your .npmrc file. You can do this by running the following command from your project root. Make sure to insert your token:

1echo "@evinced:registry=https://evinced.jfrog.io/artifactory/api/npm/restricted-npm/
2//evinced.jfrog.io/artifactory/api/npm/restricted-npm/:_authToken=<YOUR_TOKEN_HERE>" >> ~/.npmrc & npm i @evinced/unit-tester

I'm unable to authenticate using the login command

There are a few reasons why you might be unable to authenticate using the login command. If you are having trouble, you can contact us for assistance. The most common reason for authentication issues is network connectivity problems. Make sure that your organization network is not blocking the authentication process. You can run the following snippet to verify that you are able to reach the Evinced authentication server with node js:

1const https = require("https");
2
3https.get(
4    "https://auth.evinced.com/.well-known/openid-configuration",
5    (res) => {
6        console.log("Got response: " + res.statusCode);
7        res.on("data", (chunk) => {
8            console.log("BODY: " + chunk);
9        }).on("error", (err) => {
10            console.error(err);
11        });
12    }
13);

Here is a list of endpoint URLs that the Unit Tester uses, make sure that your network allows access to these URLs:

• https://auth.evinced.com
• https://app.evinced.com
• https://ingestion.evinced.com

If your corporate network is blocking the Evinced authentication server, you might want to set up a proxy. You can find more information on how to configure a proxy here.

I'm having trouble configuring my unit tests environment

If you are having trouble configuring your unit tests environment, you can contact us for assistance. A helpful step could be to set up a new testing environment, with clean configurations, like in this example.

I'm getting a Jest encountered an unexpected token error

If you are getting a Jest encountered an unexpected token error, it is likely to happen if your project wasn't previously configured to run unit tests on components. Often, this error is caused by the Jest environment not being set up to handle the ES6 syntax, or a failure to transform images and css files. You can fix this issue by adding the following content to your Jest configuration:

1module.exports = {
2    ..., // Your Jest configuration
3    moduleNameMapper: {
4        '\\.(css|less|scss|sass)$': 'PATH/TO/EMPTY_FILE.js',
5        '\\.(jpg|jpeg|png|gif|eot|otf|webp|svg|ttf|woff|woff2|mp4)$':
6            'PATH/TO/EMPTY_FILE.js'
7    },
8    // lodash is a dependency of many packages, and it's not transpiled by default
9    transformIgnorePatterns: ['/node_modules/(?!lodash-es)/'],
10};

The EMPTY_FILE.js file in this examples is an empty file, that exports an empty object like so:

1module.exports = {};

The tests seems inaccurate, I'm sure that my component has a role

The Unit Tester uses the specified selector to locate the component and then scans the component for accessibility issues. If the selector was incorrect, the scan results may not be accurate. Make sure that the selector is correct and that the component is rendered before scanning. You can use the test configuration debugLevel: 'roles' to see the roles of the elements in the DOM and their selectors. You can also use the test configuration debugLevel: 'dom' to print the current DOM representation. Also, you can use the allowTargetReset option to allow Unit Tester to pick the target element. This is useful when the component is wrapped inside several containers.

My component is disabled and the scan issues a warning. How can I suppress this warning?

The Unit Tester will issue a warning if it detects that the component is disabled. If you want to suppress this warning, you can use the toHaveNoFailures assertion instead of the toHaveNoWarnings assertion. The toHaveNoFailures assertion will pass if the scan results contain no failures, and will ignore warnings.

A certain test is failing, but I prefer to skip it instead of fixing it

If you want to skip a certain test, you can always filter out the test from the AnalysisResult array that is returned by the Unit Tester. You can utilize js and filter tests by any property of the AnalysisResult object. For example:

1const results = await EvincedUT.analyzeButton("button[aria-disabled]");
2// We want the disabled button to remain on the focus sequence for discoverability
3const filteredResults = results.filter(
4    (result) =>
5        result.message !==
6        "Button element is a native html control so it's best to use disabled instead of aria-disabled"
7);
8expect(filteredResults).toHaveNoFailures();

I get a weird error - SyntaxError: slot):not([inert] is not a valid selector

This error is caused by the nwsapi package, which is a dependency of the jsdom package. The nwsapi package is used to parse CSS selectors, and in certain versions of the package, it does not support the :not pseudo-class. This issue has been fixed in the latest version of the nwsapi package, so you can fix this issue by updating the nwsapi package to the latest version. You can do this by running the following command from your project root:

1npm install nwsapi@latest

Support

Please feel free to reach out to support@evinced.com with any questions.