Unit Tester

What is Unit Tester?

Evinced’s Unit Tester is a tool that integrates with test runners like Jest or Vitest to automatically detect accessibility issues in web components. It can be used to add unit tests to your frontend project. Unlike common accessibility unit testing libraries that conduct purely static syntactic tests, the unit tester invokes a scan that checks if your component has the required accessibility semantics for all the required component states and that you have implemented the correct keyboard interactions to transition between those states. This ensures that each tested component is fully compatible with assistive technologies (e.g. screen readers) and operable by keyboard.

The unit tester offers test suites for an extensive set of component patterns. These test suites were defined by conducting extensive research on each pattern, drawing from the APG, Open UI, and other relevant resources. While the APG serves as a primary benchmark, the team is mindful of its limitations and uses additional resources to ensure comprehensive testing. APG prioritizes patterns that are commonly used but lack corresponding semantic elements. The Unit Tester's test plans are reviewed by in-house accessibility experts - its tests are designed to support both native and non-native implementations, with a preference for semantic HTML.

Since the JSDOM environment does not visually render the code, we recommend using downstream tools and/or manual review to evaluate visual requirements such as focus indication and color contrast. At the conclusion of each test, a rich and comprehensive report is generated to easily track issues to resolution.

Interested in seeing this in action? Contact us to get started!

Prerequisites

• Node version 16 or higher
• A testing framework like Jest or Vitest is installed and configured in your project. For running tests with other testing frameworks, please contact us.
  • Jest version >= 25 is required for Jest users.
• Testing Library for your relevant framework is installed in your project. This is required to simulate user interactions in your tests.
• JSDOM version >= 15.2.1 is supported. If you encounter issues with JSDOM, please contact us.

You can run this command to install the required packages. Make sure to install the correct package of Testing Library for your framework (e.g. @testing-library/react for React):

1# Example for React
2npm install --save-dev jest @testing-library/react

Get Started

The following section will guide you through the process of setting up the Evinced Unit Tester in your project. You might also be interested in the Common Troubleshooting section.

Obtain EvincedUT Package

The Evinced Unit Tester tool is currently only available in this private repository: https://evinced.jfrog.io/artifactory/restricted-npm/. If you need access contact us, reach out to your Evinced Account Team, or support@evinced.com for access to the EvincedUT Package.

Integrate the UT dependency

The Unit Tester package is not publicly available, as it is found in a restricted npm registry, which requires an access token.

Fetching from the Restricted Repo

If you haven't used an Evinced product before, you may need to add the Evinced package registry token to your .npmrc file or defer to a local registry used by your team.

You can gain access to the private Evinced Repository by running the following command. Make sure to insert your token; contact us or your account team to get the token:

1echo "@evinced:registry=https://evinced.jfrog.io/artifactory/api/npm/restricted-npm/
2//evinced.jfrog.io/artifactory/api/npm/restricted-npm/:_authToken=<YOUR_TOKEN_HERE>" >> ~/.npmrc

Once the Evinced package registry is already set up in your .npmrc file, you can install the package using the following command:

1// This will only be available after you received an access token to the Evinced scope
2npm install "@evinced/unit-tester"

Installing from Local Tarball

WARNING: this will only work locally, if other team members or CICD systems need the Unit Tester Package follow the Restricted Repo steps.

If your Evinced Account team has provided a .tgz tarball for you, you may install it to your local dependencies. No unzipping needed.

1# Using NPM
2npm install -D {path to unit-tester.tgz package}

Add EvincedUT to your tests

In your test file, import Unit Tester and initialize the EvincedUT object.

1import EvincedUT from "@evinced/unit-tester";

Globally Import EvincedUT into your test environment

Instead of importing EvincedUT into each test file, you can make the EvincedUT object available globally in your test environment. Simply add the following lines to your Jest setup file.

1import EvincedUT from "@evinced/unit-tester";
2
3Object.defineProperty(global, "EvincedUT", {
4    value: EvincedUT,
5});

If your Jest configuration does not include a setup file, you can create one by adding the following line to your Jest configuration file. Make sure to use the setupFilesAfterEnv configuration option as the Evinced library must be initialized after Jest is loaded.

1// jest.config.js
2const customJestConfig = {
3  setupFilesAfterEnv: ["<rootDir>/jest.setup.js"],
4  ...
5};

NOTE: If your eslint rules raise a no-undef error on the EvincedUT global object, you can add the following lines to your eslint.rc:

1globals: {
2    EvincedUT: "readonly";
3}

Authentication

In order to write and run tests with the Evinced Unit Tester, you must set up authentication. The unit tester supports two modes of authentication: User and Anonymous authentication. If you are testing from a local workstation with access to a browser you can login with your Evinced username & password via User Authentication. If you need to setup automated runs of Unit Tester you can use environment variable secrets with Anonymous Auth.

It’s highly recommended to use User or Anonymous authentication methods with the Evinced Unit Tester. This will allow Evinced to provide analytics and usage data upon request as well as prevent disruptions when JWT tokens expire. If these methods aren't an option please reach out to your Evinced account team to be provided with an offline token.

• User authentication is intended for test authors, it enables the user to write and run tests locally.

• Anonymous authentication is intended for test runners and CI environments, it enables the user to run tests. Setting up one of these authentication methods is enough to run tests with the Evinced Unit Tester.

• Offline authentication is for any situation where you cannot reach our licensing servers, no HTTP calls will be performed by the SDK.

User authentication

To set up user authentication, you must run the login command from your project root. After installing the Evinced Unit Tester, run the following command in your project directory:

1$ npx --package=@evinced/unit-tester login

This will open a browser window where you will be asked to confirm the login code which will be printed to the terminal.

1Opening the browser to continue the login process, your user code is ABCD-EFGH.
2If the browser does not open automatically, please open it manually and navigate
3to https://auth.evinced.com/activate?user_code=ABCD-EFGH

After confirming the login code, you will be asked to log in to your Evinced account. If you do not have an Evinced account, contact the administrator of your Evinced organization or contact us to get started.

Once you have logged in, the login command will print a success message to the terminal.

1Login successful

You can now write and run tests with the Evinced Unit Tester. Each time your authentication token expires, the unit tester will automatically refresh it, you should not need to run the login command again. If you want to change the user that is logged in, you must delete the saved user token. To do this, run the following command from your project root:

1rm .cache/userToken

To ensure that your authentication token is not accidentally committed to your source control, you should add the .cache directory to your .gitignore file.

Anonymous authentication

Anonymous authentication is intended for test runners and CI environments, you must authenticate using the Service ID and API key. Typically this is for CICD pipelines or automated testing. It enables the user to run tests without having to log in to an Evinced account. If you have already logged in using the user authentication, this method is not required, and vice versa.

In order to use anonymous authentication, you must find your Evinced Service ID and API key which are available in our product hub. Contact the Evinced administrator of your organization or Evinced support if you cannot login.

1. Obtain the Service ID and API key credentials at [Evinced Product Hub] > Automation for Web > "Get SDK" button.

2. See the Service Account ID and API Key at the bottom of the page.

3. Save the credentials as environment variables and continue below.

Once you get your tokens, set them in the EVINCED_SERVICE_ACCOUNT_ID and EVINCED_SERVICE_ACCOUNT_SECRET environment variables. There are many ways to set up the environment variables for anonymous authentication, each method would suffice. Here are a few examples:

• If your project uses a .env file, you can add the following lines to it:

1EVINCED_SERVICE_ACCOUNT_ID=your-service-account-id
2EVINCED_SERVICE_ACCOUNT_SECRET=your-service-account-secret

• If your project uses a CI environment, you can set the environment variables in the CI Secret configuration.
• You can set the environment variables locally on any host before running your tests. On Mac or Linux, you should add these lines to your .bash_profile or .zshrc file:

1export EVINCED_SERVICE_ACCOUNT_ID=your-service-account-id
2export EVINCED_SERVICE_ACCOUNT_SECRET=your-service-account-secret

On Windows, you should add them to your system environment variables. You can set the environment variables in your CMD terminal with the following commands:

1setx EVINCED_SERVICE_ACCOUNT_ID=your-service-account-id
2setx EVINCED_SERVICE_ACCOUNT_SECRET=your-service-account-secret

• You can set the environment variables inline in your test script:

1EVINCED_SERVICE_ACCOUNT_ID=your-id EVINCED_SERVICE_ACCOUNT_SECRET=your-secret npm run <ACCESSIBILITY_TEST_SCRIPT>

If you are unable to or don't wish to use the default environment variables (EVINCED_SERVICE_ACCOUNT_ID and EVINCED_SERVICE_ACCOUNT_SECRET), You can call the configure method exposed by the unit tester package before running your tests, for instance in your Jest setup file. You can find more about it here. You should still use environment variables or other secret management techniques to provide these values. Do not commit secrets directly to your code.

1// setup-jest.js
2import EvincedUT, { configure } from "@evinced/unit-tester";
3
4configure({
5    serviceAccountId: process.env.EVINCED_SERVICE_ACCOUNT_ID,
6    serviceAccountSecret:
7        process.env.EVINCED_SERVICE_ACCOUNT_SECRET,
8});

Once you have set up anonymous authentication, you can run tests with the Evinced Unit Tester without having to log in to an Evinced account. An authentication token will be saved to your .cache directory, you should not commit this to your source control.

Offline Authentication

In the event you have no access to the public internet the Evinced SDK can use a JSON Web Token(JWT) to perform the normal Authentication & Authorization checks. These checks will happen locally when an Evinced employee provides a JWT for your entire organization to use instead of the normal Secret or npx login command.

You can either add EVINCED_OFFLINE_TOKEN as local environment variable or CI Secret. The value being the JWT provided to your orgnaization.

Or you can explicitly use the JWT as the serviceAccountSecret value in the configure step:

1// setup-jest.js
2import EvincedUT, { configure } from "@evinced/unit-tester";
3
4configure({
5    serviceAccountId: process.env.SERVICE_ACCOUNT_ID,
6    offlineToken: process.env.EVINCED_OFFLINE_TOKEN,
7});

Your first test

Here are simple examples of how to add an Evinced accessibility scan to a test. Please note the inline comments that give detail on each test step.

Vanilla JS

1it("Evinced unit tester basic example", async () => {
2    // Create your component
3    const myComponent = document.createElement("div");
4    myComponent.setAttribute("role", "button");
5    myComponent.setAttribute("class", "custom-button");
6    myComponent.setAttribute("data-testid", "myComponent");
7    document.body.appendChild(myComponent);
8
9    // Scan for a11y issues and assert on the results
10    const results = await EvincedUT.analyzeButton({ testId: "myComponent" });
11    expect(results).toHaveNoFailures();
12});

React

1import { render } from "@testing-library/react";
2
3it("Evinced unit tester basic example", async () => {
4    // Render your component
5    render(<MyButton>Click Me!</MyButton>);
6
7    // Scan for a11y issues and assert on the results
8    const results = await EvincedUT.analyzeButton({ role: "button" });
9    expect(results).toHaveNoFailures();
10});

Configuration

The Evinced Unit Tester exposes a global configuration object that can be used to configure the behavior of the unit tester. You can update a specific configuration setting by passing a Config object to the configure function in your test setup file. The object should contain the key of the setting you want to update and its new value. Here's an example of how you can update configuration setting:

1// setup-jest.js
2import EvincedUT, { configure } from "@evinced/unit-tester";
3
4configure({ reportSkippedTests: true, detailedMembersReport: false });

Additionally, you can set the configuration options inline in your test file. This is useful when you want to tailor the configuration for individual analyses. Here's an example of how you can set the configuration options inline:

1const results = await EvincedUT.analyzeButton(
2    { element: myComponent },
3    {
4        containerElement: document.querySelector("#my-container"),
5        extendedKeyboardTests: true,
6    }
7);

The following configuration options are available:

Components summary

To use Unit Tester the component must be selected according to the ARIA authoring practices guide. Example of the components and their tests are found below.

The ARIA authoring practices guide (APG) defines 30 UX patterns or component types (some of which contain multiple sub-patterns) that can work with assistive technologies and details the implementation requirements for both accessibility semantics as well as keyboard interactions. The unit tester provides a way for developers to fully automate the testing of a component to ensure that it meets the requirements laid out in the APG.

API

analyzeComponent

Overview

Scans the specified component. An array is returned containing the results of each accessibility check that was performed. This is the generic API signature for most components. For more specific component types, see the following methods.

Parameters

• locator: TargetQuery | TargetLocator - The locator for the component to be scanned. TargetQuery is an object with many optional properties that allows to find a target element by different means. For legacy support we also allow TargetLocator usage - which can be a CSS selector string, an element, or a function that returns an element.
• options?: BaseComponentOptions - An optional object containing additional options for the scan. It extends the configuration object to allow inline configuration for specific scans.

Example

In the following example, a selector is passed to locate the switch component. The userEventOptions option is used to specify the delay between user interactions in the test.

1const results = await EvincedUT.analyzeSwitch(
2    { id: "my-switch" },
3    {
4        userEventOptions: { delay: 10 },
5    }
6);
7expect(results).toHaveNoFailures();

Return value

1Promise<AnalysisResult[]>;

See AnalysisResult

analyzeAccordion

Overview

Scans the specified accordion container. An array is returned containing the results of each accessibility check that was performed.

This test uses the signature specified in the analyzeDisclosure section.

analyzeAlert

Overview

Scans the specified alert element. Make sure the element is visible before scanning.

An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeBreadcrumb

Overview

Scans the specified breadcrumb container. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeButton

Overview

Scans the specified button. An array is returned containing the results of each accessibility check that was performed.

Parameters

• locator: TargetQuery | TargetLocator - The locator for the component to be scanned. TargetQuery is an object with many optional properties that allows to find a target element by different means. For legacy support we also allow TargetLocator usage - which can be a CSS selector string, an element, or a function that returns an element.
• options?: AnalyzeButtonOptions - An optional object containing additional options for the scan.
  • wasActivatedCallback?: () => boolean - an optional boolean callback to evaluate whether the element was activated after interactions
  • The other parameters of the BaseComponentOptions type.

Example

1const checkActivation = () => {
2    return window.wasButtonActivated === true;
3};
4
5const results = await EvincedUT.analyzeButton(
6    { role: "button" },
7    {
8        wasActivatedCallback: checkActivation,
9    }
10);
11expect(results).toHaveNoFailures();

analyzeCheckbox

Overview

Scans the specified checkbox. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeCarousel

Overview

Scans the specified carousel. An array is returned containing the results of each accessibility check that was performed.

Parameters

The carousel scan requires additional options. It is mandatory to specify the automatic rotation option, and either the next/previous slide controls or the slider picker control.

If your implementation of carousel does not include any controls, it is considered as an accessibility violation.

• locator: TargetQuery | TargetLocator - The locator for the component to be scanned. TargetQuery is an object with many optional properties that allows to find a target element by different means. For legacy support we also allow TargetLocator usage - which can be a CSS selector string, an element, or a function that returns an element.
• options: AnalyzeCarouselOptions - A mandatory object containing additional options for the scan.
  • automaticRotation: boolean - a boolean parameter that determines whether the carousel automatically rotates upon initialization
  • nextSlideControl?: TargetQuery | TargetLocator an optional target locator for the next slide control
  • prevSlideControl?: TargetQuery | TargetLocator an optional target locator for the previous slide control
  • pickerControl?: TargetQuery | TargetLocator an optional target locator for the slider picker control
  • rotationControl?: TargetQuery | TargetLocator an optional target locator for the rotation control
  • The other parameters of the BaseComponentOptions type.

Example

1const checkActivation = () => {
2    return window.wasButtonActivated === true;
3};
4
5const results = await EvincedUT.analyzeCarousel(
6    { selector: "div.carousel" },
7    {
8        automaticRotation: false,
9        nextSlideControl: { accName: "next" },
10        prevSlideControl: { accName: "previous" },
11        rotationControl: { selector: "div.rotation-control" },
12    }
13);
14expect(results).toHaveNoFailures();

analyzeCombobox

Overview

Scans the specified combobox element. Currently only supports the listbox combobox pattern.

An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeDataGrid

Overview

Scans the specified data grid element. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeDisclosure

Overview

Scans the specified disclosure button. An array is returned containing the results of each accessibility check that was performed.

Parameters

• locator: TargetQuery | TargetLocator - The locator for the component to be scanned. TargetQuery is an object with many optional properties that allows to find a target element by different means. For legacy support we also allow TargetLocator usage - which can be a CSS selector string, an element, or a function that returns an element.
• options?: AnalyzeDisclosureOptions - An optional object containing additional options for the scan.
  • evalEscape?: boolean - An optional boolean parameter that determines whether to evaluate the escape key accessibility. Default is false.
  • The other parameters of the BaseComponentOptions type.

Example

1const results = await EvincedUT.analyzeDisclosure(
2    { id: "myDisclosure" },
3    {
4        evalEscape: true,
5    }
6);
7expect(results).toHaveNoFailures();

analyzeFeed

Overview

Scans the specified feed element. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeLink

Overview

Scans the specified link. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeListbox

Overview

Scans the specified listbox element. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeMenu

Overview

Scans the specified menu element. This API call is intended to be used on visible menu bars, not popup menus. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeMenuButton

Overview

Scans the specified menu button element. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeMeter

Overview

Scans the specified meter element. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeModal

Overview

Scans the specified modal dialog and launcher element. An array is returned containing the results of each accessibility check that was performed.

In the simplest case you only need to specify the locator that will trigger the opening of the modal. Once the modal is opened by Unit Tester the default setting is to scan for an element with the role of 'dialog'. No further input or params needed.

Parameters

• launcherLocator: TargetQuery | TargetLocator - Required. The locator of the element or button that opens the modal.
  • TargetQuery is an object with many optional properties that allows Unit Tester to find the element by different means.
  • TargetLocator can be a CSS selector string, an element, or a function that returns an element.
• options?: AnalyzeModalOptions - An optional object containing additional options for the scan.
  • modalLocator?: TargetQuery | TargetLocator - Optional. The locator for the modal container. Defaults to { role: 'dialog' }
  • The other parameters of the BaseComponentOptions type.

Example

1// simple example
2const results = await EvincedUT.analyzeModal("#openButton");
3expect(results).toHaveNoFailures();
4
5// advanced example
6const results = await EvincedUT.analyzeModal(
7    { element: buttonElement },
8    { modalLocator: { id: "myModal" } }
9);
10expect(results).toHaveNoFailures();

analyzeRadioGroup

Overview

Scans the specified radio group. An array is returned containing the results of each accessibility check that was performed.

Parameters

• locator: TargetQuery | TargetLocator - The locator for the component to be scanned. TargetQuery is an object with many optional properties that allows to find a target element by different means. For legacy support we also allow TargetLocator usage - which can be a CSS selector string, an element, or a function that returns an element.
• options?: AnalyzeRadioGroupOptions - An optional object containing additional options for the scan.
  • isToolbar?: boolean - an optional boolean parameter to specify if the radio group is part of a toolbar.
  • The other parameters of the BaseComponentOptions type.

Example

1const results = await EvincedUT.analyzeRadioGroup(
2    { role: "radiogroup" },
3    {
4        isToolbar: true,
5    }
6);
7expect(results).toHaveNoFailures();

analyzeSiteNavigation

Overview

Scans the specified site navigation container. Supports all kinds of site navigation components.

An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeSlider

Overview

Scans the specified slider. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeSpinButton

Overview

Scans the specified Spin button. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeSliderMultiThumb

Overview

Scans the specified multi thumb slider container. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeSwitch

Overview

Scans the specified switch. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeTable

Overview

Scans the specified table. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeTabList

Overview

Scans the specified tablist container. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

analyzeTextInput

Overview

Scans the specified textinput element. An array is returned containing the results of each accessibility check that was performed.

Parameters

• locator: TargetQuery | TargetLocator - The locator for the component to be scanned. TargetQuery is an object with many optional properties that allows to find a target element by different means. For legacy support we also allow TargetLocator usage - which can be a CSS selector string, an element, or a function that returns an element.
• options?: AnalyzeRadioGroupOptions - An optional object containing additional options for the scan.
  • isRequired?: boolean - an optional boolean parameter to specify if the text input is required.
  • invalidValueExample?: string - an optional parameter to specify an invalid input for the input validation evaluation.
  • submitButtonLocator?: TargetLocator | TargetQuery - an optional locator to the form submit button, used for input validation evaluation.
  • The other parameters of the BaseComponentOptions type.

Example

1const results = await EvincedUT.analyzeTextInput(
2    { selector: 'input#my-input' },
3    {
4        isRequired: true,
5        invalidValueExample: 'Invalid value',
6        submitButtonLocator: { role: 'button' },
7    }
8);
9expect(results).toHaveNoFailures();

analyzeToggleButton

Overview

Scans the specified toggle button, which is a subcategory of the button pattern. An array is returned containing the results of each accessibility check that was performed.

This test uses the generic signature specified in the About analyzeComponent section.

Assertions

toHaveNoFailures

Asserts that the scan results contain no failures. If the scan results contain failures, the test will fail and the results will be printed to the console. This assertion is useful when you want to ensure that your component has the required accessibility semantics and that you have implemented the correct keyboard interactions.

Example

1const results = await EvincedUT.analyzeButton({ role: "button" });
2expect(results).toHaveNoFailures();

toHaveNoWarnings

Asserts that the scan results contain no warnings. If the scan results contain warnings, the test will fail and the results will be printed to the console. This assertion is a strict version of the toHaveNoFailures assertion. It is useful when you want to ensure that your component have implemented recommended accessibility semantics and keyboard interactions.

Example

1const results = await EvincedUT.analyzeButton({ role: "button" });
2expect(results).toHaveNoWarnings();

toHaveResult

Asserts that the scan results contain a specific result. If the scan results do not contain the specified result, the test will fail and the results will be printed to the console. This assertion is useful when you want to ensure that your component complies with specific tests.

Parameters

• expected: Partial<AnalysisResult> - The expected result. The assertion will pass if the scan results contain a result that matches the expected result.

Example

1const results = await EvincedUT.analyzeButton({ role: "button" });
2expect(results).toHaveResult({
3    component: "Button",
4    test: "button name",
5    pass: true,
6});

Typings

TargetQuery

TargetQuery allows to find a target element by different means:

Example usage

1{
2    role: "button",
3    accName: "My Button"
4}

TargetLocator

A string such as #my-element or .my-element is sufficient to locate an HTML element.

1string | HTMLElement | SVGElement | (() => HTMLElement | SVGElement);

AnalysisResult

1{
2    // The type of component that was scanned
3    component: string;
4
5    // The test that was run
6    test: string;
7
8    // The result of the test
9    pass: boolean;
10
11    // The message associated with the test
12    message: string;
13
14    // The element that were related to the test message
15    members?: string[];
16
17    // The actions that were related to the test message
18    actions?: string[];
19
20    // The type of the result (PASS, FAIL, WARN or SKIP)
21    type?: string;
22}

BaseComponentOptions

Each component scan object extends the configuration object, which allows inline configuration for specific scans. For example, the following configuration could be passed to each analyzeComponent method:

1{
2    // An optional object containing options to be passed to the user-event library
3    userEventOptions: {
4        delay: 30;
5    }
6    // An optional parameter that will add to the report which tests were skipped, as well as passed INFO level tests.
7    reportSkippedTests: true;
8}

Proxy

Configuring a proxy might be necessary when running the unit tester in an isolated corporate environment. To configure a proxy you can set the proxy URL in one of the env vars HTTPS_PROXY or EVINCED_HTTPS_PROXY in the format of http(s)://user:pass@proxy.url:port/, where user and pass are optional if your proxy requires it. You could also set the proxy URL using the configure method, which supports a URL string in the same format as the environment variables, a URL object or a Proxy object with the following properties:

1/**
2 * Proxy configuration for the unit tester
3 * @protocol: The protocol of the proxy, either `http` or `https`
4 * @host: The host of the proxy
5 * @port: The port of the proxy
6 * @auth: Optional authentication details for the proxy
7 *  @username: The username to use for the proxy
8 *  @password: The password to use for the proxy
9 */
10{
11  protocol: 'http' | 'https'
12  host: string;
13  port: number;
14  auth?: {
15    username: string;
16    password: string;
17  }
18}

Troubleshooting common issues

Receiving a 404 error when trying to install the package using npm install

If you are unable to install the package using npm install, and receive the following error npm ERR! 404 Not Found, you may need to add the Evinced package registry token to your .npmrc file. You can do this by running the following command from your project root. Make sure to insert your token:

1echo "@evinced:registry=https://evinced.jfrog.io/artifactory/api/npm/restricted-npm/
2//evinced.jfrog.io/artifactory/api/npm/restricted-npm/:_authToken=<YOUR_TOKEN_HERE>" >> ~/.npmrc & npm i @evinced/unit-tester

I'm unable to authenticate

Most authentication issues are caused by network connectivity problems. Make sure that your organization network is not blocking the authentication process.

Here is a list of endpoint base URLs that the Unit Tester uses, make sure that your network allows access to these URLs:

• https://app.evinced.com (for user and anonymous authentication)
• https://auth.evinced.com (for user authentication only)
• https://ingestion.evinced.com (for sending usage analytics)

In most cases, if your corporate network is blocking these API calls, you just need to configure a proxy.

You can run the following snippet to verify that you are able to reach the Evinced user authentication server with node js:

1const https = require("https");
2
3// Automatically pick up proxy settings from environment variables if set
4const proxy =
5    process.env.EVINCED_HTTPS_PROXY ||
6    process.env.HTTPS_PROXY ||
7    process.env.https_proxy ||
8    process.env.EVINCED_HTTP_PROXY ||
9    process.env.ALL_PROXY ||
10    process.env.all_proxy;
11
12if (proxy) {
13    console.log(`Using proxy settings from environment variable: ${proxy}\n`);
14    const { HttpsProxyAgent } = require("hpagent");
15    const httpsAgent = new HttpsProxyAgent({ proxy });
16    https.globalAgent = httpsAgent;
17}
18
19function testUrl(url) {
20    return https.get(url, (res) => {
21        if (res.statusCode === 200) {
22            console.log("No connection problems (status code 200)\n");
23        } else {
24            console.log(`Connection problem: ${res.statusCode}\n`);
25        }
26        res.on("error", (err) => {
27            console.error(err);
28        });
29    });
30}
31
32console.log(
33    "Testing connection to user authentication server (https://auth.evinced.com/)\n"
34);
35testUrl("https://auth.evinced.com/.well-known/openid-configuration").on(
36    "close",
37    () => {
38        console.log(
39            "\nTesting connection to Evinced app server (https://app.evinced.com/)\n"
40        );
41        testUrl("https://app.evinced.com/public/healthz/");
42    }
43);

If you are unable to resolve issues with authentication, please contact us for assistance.

I'm having trouble configuring my unit tests environment

If you are having trouble configuring your unit tests environment, you can contact us for assistance. A helpful step could be to set up a new testing environment, with clean configurations, like in this example.

I'm getting a "Jest encountered an unexpected token" error

If you are getting a Jest encountered an unexpected token error, it is likely to happen if your project wasn't previously configured to run unit tests on components. Often, this error is caused by the Jest environment not being set up to handle the ES6 syntax, or a failure to transform images and css files. You can fix this issue by adding the following content to your Jest configuration:

1module.exports = {
2    ..., // Your Jest configuration
3    moduleNameMapper: {
4        '\\.(css|less|scss|sass)$': 'PATH/TO/EMPTY_FILE.js',
5        '\\.(jpg|jpeg|png|gif|eot|otf|webp|svg|ttf|woff|woff2|mp4)$':
6            'PATH/TO/EMPTY_FILE.js'
7    },
8    // lodash is a dependency of many packages, and it's not transpiled by default
9    transformIgnorePatterns: ['/node_modules/(?!lodash-es)/'],
10};

The EMPTY_FILE.js file in this examples is an empty file, that exports an empty object like so:

1module.exports = {};

The tests seems inaccurate, I'm sure that my component has a role

The Unit Tester uses the specified selector to locate the component and then scans the component for accessibility issues. If the selector was incorrect, the scan results may not be accurate. Make sure that the selector is correct and that the component is rendered before scanning. You can use the test configuration debugLevel: 'roles' to see the roles of the elements in the DOM and their selectors. You can also use the test configuration debugLevel: 'dom' to print the current DOM representation. Also, you can use the allowTargetReset option to allow Unit Tester to pick the target element. This is useful when the component is wrapped inside several containers.

My component is disabled and the scan issues a warning. How can I suppress this warning?

The Unit Tester will issue a warning if it detects that the component is disabled. If you want to suppress this warning, you can use the toHaveNoFailures assertion instead of the toHaveNoWarnings assertion. The toHaveNoFailures assertion will pass if the scan results contain no failures, and will ignore warnings.

A certain test is failing, but I prefer to skip it instead of fixing it

If you want to skip a certain test, you can always filter out the test from the AnalysisResult array that is returned by the Unit Tester. You can utilize js and filter tests by any property of the AnalysisResult object. For example:

1const results = await EvincedUT.analyzeButton({
2    selector: "button[aria-disabled]",
3});
4// We want the disabled button to remain on the focus sequence for discoverability
5const filteredResults = results.filter(
6    (result) =>
7        result.message !==
8        "Button element is a native html control so it's best to use disabled instead of aria-disabled"
9);
10expect(filteredResults).toHaveNoFailures();

React issues a warning - a test was not wrapped in act

If you are using React, you might encounter a warning that an interaction with the component was not wrapped in act. This warning occurs when the unit tester is using a different version of testing library than the one that was used to render the component. Examine your project dependencies and make sure that you only have one version of the testing library installed.

Sometimes it can be very difficult to find the cause of these warnings, especially if you are using a lot of dependencies. Nonetheless, they should not affect the test results, so if desired, you can suppress the warning by wrapping the interactions with the component in a waitFor block - just note that if the component analysis will throw an error, the test will always reach the timeout limit. Here is an example of how you can do this:

1import React from "react";
2import { render, waitFor } from "@testing-library/react";
3import EvincedUT from "@evinced/unit-tester";
4import { MyButton } from "./button";
5
6test("My Button accessibility analysis", async () => {
7    render(<MyButton>Click me!</MyButton>);
8
9    const results = await waitFor(() =>
10        EvincedUT.analyzeButton({ role: "button" })
11    );
12    expect(results).toHaveNoWarnings();
13}, 10000);

If you need to include additional code in the waitFor block, you can do it like this:

1const results = await waitFor(
2    async () => {
3        // ...your code
4        return await EvincedUT.analyzeButton({ selector: "button" });
5    },
6    // Optional extended timeout
7    { timeout: 10000 }
8);

I get a weird error - SyntaxError: slot):not([inert] is not a valid selector

This error is caused by the nwsapi package, which is a dependency of the jsdom package. The nwsapi package is used to parse CSS selectors, and in certain versions of the package, it does not support the :not pseudo-class. This issue has been fixed in the latest version of the nwsapi package, so you can fix this issue by updating the nwsapi package to the latest version. You can do this by running the following command from your project root:

1npm install nwsapi@latest

I'm getting a false positives relating to focus management, focus order or focus trap

If your code depends directly on the tabbable package or you're using a library that depends on it such as focus-trap or focus-trap-react, you might get false positives in the focus management or focus trap tests. This is because the tabbable package does not fully support JSDOM. Specifically the tabbable package performs display checks using DOM APIs such as Element.getClientRects() and Element.getBoundingClientReact(), which are not fully supported in JSDOM.

tabbable has an option to disable display checks but since you probably don't want to do that in your production code, you can disable it only for the tests using mocking. For more information and instructions on how to do this, please refer to the tabbable documentation.

I'm getting timeouts and/or false positives when my component is animated or uses transitions

JSDOM does not fully support CSS animations and transitions, for instance, it doesn't support the animationend event. This can cause timeouts and false positives in the tests. Many libraries that depend on animations and transitions, such as react-transition-group have a way to disable animations for testing. For instance, react-transition-group has a global configuration that can be set to disable transitions.

If you are using another library, please refer to its documentation to see if it has a similar feature.

I'm getting errors about node:os or other node libraries

For errors like: no such file or directory, open 'node:os'

These kind of errors are most likely due an older jest version. The preferred solution is to upgrade the jest version but if it's not possible to upgrade jest then you'll have to mock these node libraries:

1. create __mocks__ folder in your project's root level
2. add a js file with the name of the library you need to mock, example: node:os.js
3. in your `jest.config.js' file, add module mapping

   1moduleNameMapper: {
   2  'node:os': '<rootDir>/__mocks__/node:os.js',
   3}

Support

Please feel free to reach out to support@evinced.com with any questions.