WebdriverIO JS SDK (Web)

The Evinced WebdriverIO JS SDK (Web) integrates with new or existing WebdriverIO tests to automatically detect accessibility issues. By adding a few lines of code to your WebdriverIO project, you can begin analyzing all the web pages and DOM changes to provide a dynamic view of how your site can become more accessible. As a result of the test, a rich and comprehensive report is generated to easily track issues to resolution.

Interested in seeing this in action? Contact us to get started!

Prerequisites

• WebdriverIO version 7 or higher
• Node version 14+
• ChromeDriver

In order to add import to your config file make sure that you have Babel config added to your project.

Babel Setup

Get Started

Installation

The WebdriverIO SDK package is not publicly available. Contact us to get started.

In your project directory install the Evinced SDK using npm .

1npm install @evinced/webdriverio-sdk

Your First Test

SDK Initialization

To work with WebdriverIO JS SDK (Web) you need to have authentication token. Please refer to licensing section for details. Add following lines to your wdio.conf.js:

1import Evinced from "@evinced/webdriverio-sdk";
2
3exports.config = {
4    // ...
5    services: [[Evinced.WdioService]],
6}

1before: async function(capabilities, specs) {
2    // Set offline credentials method
3    Evinced.setOfflineCredentials({
4        serviceId: '<serviceID>',
5        token: 'token'
6    });
7
8    // Set online creadentials method (through Evinced lience server)
9    await Evinced.setCredentials({
10        serviceId: '<serviceID>',
11        secret: '<secret>'
12    });
13}

Add Evinced accessibility checks (Single Run Mode)

This is a simple example of how to add an Evinced accessibility scan to a test. Please note the inline comments that give detail on each test step.

1test("evAnalyze", async () => {
2    await browser.url("https://example.com/");
3    // Scan the page for accessibility issues
4    const issues = await browser.evAnalyze();
5    expect(issues).toHaveLength(0);
6});

Add Evinced accessibility checks (Continuous Mode)

This is an example of how to add a continuous Evinced accessibility scan to a test. Using the evStart() and evStop() methods, the Evinced engine will continually scan in the background capturing all DOM changes and page navigations as the test is executed. This will capture all accessibility issues as clicking on drop-downs or similar interactions reveals more of the page. The advantage of continuous mode is that no interaction with the actual test code is needed.

1test("evStart/evStop", async () => {
2    await browser.url("https://example.com/");
3    await browser.evStart();
4    await $("#button-1").click();
5    await $("#button-2").click();
6    const issues = await browser.evStop();
7    expect(issues.report).toHaveLength(0);
8});

API

Evinced.Wdioservice

Initializes the Evinced object within the project.

Example

1services: [
2    [
3        Evinced.WdioService,
4        {
5            axeConfig: {
6                rules: {
7                    'link-name': { enabled: false }
8                }
9            },
10            logging: {
11                LOGGING_LEVEL: 'error',
12                ADD_LOGGING_CONTEXT: true
13            },
14            skip_validations: [],
15            toggles: {},
16        },
17    ]
18]

Please refer to configuration to see examples of using init with options.

evAnalyze(options)

Scans the current page and returns a list of accessibility issues.

Note: This method is not supported if evStart() is already running.

Please refer to configuration to see examples of using init with options.

Example

1const issues = await browser.evAnalyze({
2    axeConfig: {
3        rules: {
4            "link-name": { enabled: false },
5        },
6    },
7});

Return value Promise<Issue[]>

A Report object is returned containing accessibility issues. This is the recommended method for static page analysis. For more information regarding reports as well as the Report object itself, please refer to our detailed Web Reports page.

evStart(options)

Continually watches for DOM mutations and page navigation and records all accessibility issues until the evStop() method is called. This method is recommended for dynamic page flows.

Example

1await browser.evStart({
2    axeConfig: {
3        rules: {
4            "link-name": { enabled: false },
5        },
6    },
7});
8await $("#button-1").click();
9await $("#button-2").click();
10const issues = await browser.evStop();

Return value Promise<void>

evStop(options)

Stops the process of issue gathering started by evStart() command.

Example

1await browser.evStart();
2await $("#button-1").click();
3await $("#button-2").click();
4const issues = await browser.evStop();

Return value Promise<Issue[]>

Returns an object containing recorded accessibility issues from the point in time at which the evStart() method was instantiated. For more information regarding reports as well as the returned object itself, please refer to detailed Web Reports page.

evSaveFile(issues, format, destination)

Saves list of the issues in a file, with the specified format and location. For example, format could be ‘json', 'html, 'sarif' or 'csv'. Please find detailed information on Web Reports page.

Example

1await browser.url("https://demo.evinced.com/");
2
3const issues = await browser.evAnalyze();
4
5const filePath = path.resolve(__dirname, "evinced-report.html");
6await browser.evSaveFile(issues, "html", filePath);

Return value Promise<void>

Aggregated Report

The aggregated report feature allows you to have a general aggregated report for the whole run (not only for one test or suite). This report will contain all the issues found by the tests where evStart() and evStop() commands were called. It is still possible to use the evSaveFile() command in any place of your code along with this Aggregated Report feature.

Example

Licensing

To work with WebdriverIO JS SDK (Web) you need to have Authentication token. There are two methods to provide the token: offline mode and online mode. The difference is if you use online mode the request to obtain the token will be sent to Evinced Licencing Server. You need to provide Service ID and Secret API key in such a case. They are available via the Evinced Product Hub.

Offline mode assumes that you already have token and can provide it directly. If an offline token is required please reach out to your account team or support@evinced.com.

We encourage to use environment variables to store credentials and read them in code

1# Offline mode
2export AUTH_SERVICE_ID=<serviceId>
3export AUTH_TOKEN=<token>
4
5# Online mode
6export AUTH_SERVICE_ID=<serviceId>
7export AUTH_SECRET=<secret>

Example

1// Set offline credentials method
2const serviceId = process.env.AUTH_SERVICE_ID;
3const token = process.env.AUTH_TOKEN;
4Evinced.setOfflineCredentials({
5    serviceId,
6    token
7});
8
9// OR
10
11// Set online creadentials method (through Evinced lience server)
12const serviceId = process.env.AUTH_SERVICE_ID;
13const secret = process.env.AUTH_SECRET;
14await Evinced.setCredentials({
15    serviceId,
16    secret
17})

Configuration

The same configuration object can be used in Evinced.Wdioservice, evStart() and evAnalyze() methods but with a bit different consequences. By providing some options in Evinced.Wdioservice method you define a global configuration for all calls of evStart() or evAnalyze() methods. Please note that global configuration is not intended to be altered from tests due to it will affect all the rest tests as well as tests running in parallel threads. To alter configuration in specific tests you can provide config on the method's level, for instance with evStart() it defines local configuration for this particular session until evStop() is called. If provided in both levels, the command's level config will override the global one.

Engines configuration

Evinced uses two separate engines when scanning for accessibility issues, one is the aXe engine and the other is the Evinced engine. By default, Evinced disables the aXe Needs Review and Best Practices issues based on customer request and due to the fact they are mostly false positives. Please note this setting when comparing issue counts directly. See an example of how to enable Needs Review and Best practices issues in our toggles section.

Configuration object type

1export type TInitOptions = {
2    enableScreenshots?: false,
3    rootSelector?: string,
4    enableShadowDom?: false,
5    filterIssues?: IssuesFilter,
6    axeConfig?: axe.RunOptions,
7    logging?: {
8        LOGGING_LEVEL?: 'error'|'debug'|'warn'|'info',
9        ADD_LOGGING_CONTEXT?: boolean
10    }
11    skip_validations?: {
12        selector: string,
13        urlRegex: string,
14        validationTypes: string[]
15    }[],
16};

Root Selector

Instructs engine to return issues which belong to or which are nested to element with the specified selector. By default the issues for all elements from document root will be returned.

null by default.

Example

1const issues = await browser.evAnalyze({
2    rootSelector: "#some .selector"
3});

AXE Configuration

Evinced leverages Axe open-source accessibility toolkit as part of its own accessibility detection engine. The Evinced engine is able to pinpoint far more issues than Axe alone.

For the full Axe config options, see Axe Core API.

Example

1await browser.evStart({
2    axeConfig: {
3        rules: {
4            "link-name": { enabled: false },
5        },
6    },
7});
8const issues = await browser.evStop();

Errors Strict Mode

When set to true it switches SDK in mode when Evinced SDK errors are thrown as runtime errors and stops current test execution. Otherwise errors are printed into console except critical ones.

false by default.

Example

Engine Logging

Switches logging on for engine. Log messages from the engine will be printed to console.

Levels are: debug, info, warn, error

null by default.

error is a default level on engine side.

1const issues = await browser.evAnalyze({
2    logging: {
3        LOGGING_LEVEL: 'debug',
4        ADD_LOGGING_CONTEXT: true
5    }
6});

Reports Screenshots

The Screenshots feature allows to include screenshots to the Evinced reports. When the feature is enabled, Evinced will take screenshots of the page, highlight an element with a specific accessibility issue, and include them to the report.

false by default.

Note: The Screenshots feature may affect the test run performance.

Enabling the Screenshots feature

1exports.config = {
2    services: [
3        [Evinced.WdioService, {
4            enableScreenshots: true
5        }]
6    ]
7};

Toggles

Toggles are feature flags which controls SDK and Engine behavior. With toggles we introduce experimental features or manage the way how accessibility issues are gathered. The list of toggles is not specified so we suggest not to rely on specific toggle name or value and use them in investigation purposes only.

Example enabling aXe Best Practices and Needs Review Issues

1exports.config = {
2    services: [
3        [Evinced.WdioService, {
4            experimentalFlags: {
5                USE_AXE_NEEDS_REVIEW: true,
6                USE_AXE_BEST_PRACTICES: true
7            }
8        }]
9    ]
10};

Skip Validations

Skips specific validations for the given URL pattern and the selector.

SkipValidation[] by default.

Example

1const skippedValidation1 = {
2    "selector": 'selector1',
3    "urlRegex": "www\.mysite\.org",
4    "validationTypes": [
5        "WRONG_SEMANTIC_ROLE",
6        "NO_DESCRIPTIVE_TEXT",
7    ]
8}
9const issues = await browser.evAnalyze({ skip_validations: [skippedValidation1] });

How to identify ID for the specific issue type is described in web reports section.

Knowledge Base Link overrides

This custom parameter helps to customize knowledge base links in the reports. Those links are displayed in the reports as follows:

The knowledge base link can be overridden for every issue type ID.

Example

Shadow DOM support

Example

1const issues = await browser.evAnalyze({ enableShadowDom: true });

IFrames support

When set to true, the accessibility tests run analysis on iframes that exist inside the page. Default is false.

Global Switch

Global Switch allows to disable or enable Evinced functionality. It could be needed, for example, while working on functional tests in your local environment or for running some CI jobs that are not intended to gather information regarding accessibility issues.

When switched off

• evStart() and evSaveFile() will be bypassed.
• evStop() and evAnalyze() will return an empty report.
• evValid() will never fail.

true by default.

Important! Global Switch environment variable overrides the global configuration option.

Switch on/off Evinced functionality in config

1exports.config = {
2    services: [
3        [Evinced.WdioService, {
4            switchOn: false
5        }]
6    ]
7};

Switching off Evinced functionality with environment variable

1export EV_SWITCH_ON=false

Tutorials

You can find fully functional example projects on our GitHub

Tutorial 1

Generating a comprehensive accessibility report for your application

In this tutorial, we will enhance our existing WebdriverIO UI test with the Evinced WebdriverIO SDK in order to check our application for accessibility issues. In order to get started you will need the following:

1. All of the prerequisites for the Evinced WebdriverIO SDK should be met
2. Evinced WebdriverIO SDK should be added to your project

Preface - existing UI test overview

Let’s consider the following basic UI test as our starting point.

1describe("Evinced Demo Site tests", () => {
2    test("Search Test", async () => {
3        await browser.url("https://demo.evinced.com/");
4        const BASE_FORM_SELECTOR =
5            "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container";
6        const SELECT_HOME_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > div.dropdown.line`;
7        const SELECT_WHERE_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > div.dropdown.line`;
8        const TINY_HOME_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > ul > li:nth-child(2)`;
9        const EAST_COST_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > ul > li:nth-child(3)`;
10        const SUBMIT_BUTTON = `${BASE_FORM_SELECTOR} > .search-btn`;
11        await $(SELECT_HOME_DROPDOWN).click();
12        await $(TINY_HOME_OPTION).click();
13        await $(SELECT_WHERE_DROPDOWN).click();
14        await $(EAST_COST_OPTION).click();
15        await $(SUBMIT_BUTTON).click();
16        expect(browser).toHaveUrlContaining("/results");
17    });
18});

We wrote this test for a demo travel site called TRVL that has a few known accessibility issues.

The purpose of this test is to check the functionality of the main application screen and ensure a user can successfully select their desired trip. For now, this test is only concerned with the functional testing of the app. However, with the help of the Evinced WebdriverIO SDK, we can also check it for accessibility issues along the way. Let’s go through this process with the following step-by-step instructions.

Step #1 - Verify the Evinced WebdriverIO service

Remove optional configuration in wdio.conf.js from Evinced.WdioService service.

Step #2 - Start the Evinced engine

Now that we have everything we need to scan for accessibility issues, let’s start the Evinced engine. Since we are going to use it scan throughout our test, the best place for its initialization will be after we navigate to the desired website.

1describe("Evinced Demo Site tests", () => {
2    test("Search Test", async () => {
3        await browser.url("https://demo.evinced.com/");
4        // Add command to start recording issues
5        await browser.evStart();
6        const BASE_FORM_SELECTOR =
7            "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container";
8        const SELECT_HOME_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > div.dropdown.line`;
9        const SELECT_WHERE_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > div.dropdown.line`;
10        const TINY_HOME_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > ul > li:nth-child(2)`;
11        const EAST_COST_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > ul > li:nth-child(3)`;
12        const SUBMIT_BUTTON = `${BASE_FORM_SELECTOR} > .search-btn`;
13        await $(SELECT_HOME_DROPDOWN).click();
14        await $(TINY_HOME_OPTION).click();
15        await $(SELECT_WHERE_DROPDOWN).click();
16        await $(EAST_COST_OPTION).click();
17        await $(SUBMIT_BUTTON).click();
18        expect(browser).toHaveUrlContaining("/results");
19    });
20});

Step #3 - Stop the Evinced engine and create reports

As our test was executed we collected a lot of accessibility information. We can now perform accessibility assertions at the end of our test suite. Referring back again to our UI test the best place for this assertion will be the end of test actions. To stop the Evinced engine and generate the actual object representation of your accessibility report simply call the evStop() method. We can then output the report files in JSON format.

1describe("Evinced Demo Site tests", () => {
2    test("Search Test", async () => {
3        await browser.url("https://demo.evinced.com/");
4        // Add command to start recording issues
5        await browser.evStart();
6        const BASE_FORM_SELECTOR =
7            "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container";
8        const SELECT_HOME_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > div.dropdown.line`;
9        const SELECT_WHERE_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > div.dropdown.line`;
10        const TINY_HOME_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > ul > li:nth-child(2)`;
11        const EAST_COST_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > ul > li:nth-child(3)`;
12        const SUBMIT_BUTTON = `${BASE_FORM_SELECTOR} > .search-btn`;
13        await $(SELECT_HOME_DROPDOWN).click();
14        await $(TINY_HOME_OPTION).click();
15        await $(SELECT_WHERE_DROPDOWN).click();
16        await $(EAST_COST_OPTION).click();
17        await $(SUBMIT_BUTTON).click();
18        expect(browser).toHaveUrlContaining("/results");
19        // Add a command to stop recording and retrieve issues list
20        const issues = await browser.evStop();
21        // Save issues to a report file
22        await browser.evSaveFile(issues, "json", "./test/issues.json");
23        await browser.evSaveFile(issues, "sarif", "./test/issues.sarif.json");
24        await browser.evSaveFile(issues, "html", "./test/issues.html");
25        await browser.evSaveFile(issues, "csv", "./test/issues.csv");
26        // Assert issues count
27        expect(issues).toHaveLength(0);
28    });
29});

Support

Please feel free to reach out to support@evinced.com with any questions.

FAQ

1. Can I configure which validations to run?

Yes, see the configuration section for details on how to configure Axe validations to your needs.

2. Can I run tests with Evinced using cloud-based services like Sauce Labs, Perfecto, or BrowserStack?

Yes, we have tested the Evinced SDK on many of these types of cloud-based services and expect no issues.