Developer Documentation

WebdriverIO SDK (Web)

The Evinced WebdriverIO SDK integrates with new or existing WebdriverIO tests to automatically detect accessibility issues. With the addition of as few as 5 lines of code to your WebdriverIO framework, you can begin to analyze all the pages and DOM changes to offer a dynamic view of how your site can become more accessible. At the conclusion of the test, a rich and comprehensive JSON or HTML report is generated to track issues in any reporting tool.

Interested in seeing this in action? Contact us to get started!

Prerequisites

  • WebdriverIO v7+
  • Node version 12+
  • ChromeDriver

Get Started

Add Evinced as a dependency

In your project directory install the Evinced SDK using npm . The WebdriverIO SDK package is not publicly available. Contact us to get started!

1npm install @evinced/webdriverio-sdk

Update WebdriverIO config file and provide auth token

A Service ID and API key are required to use the Cypress SDK. These are available via the Evinced Product Hub. If an offline token is required please reach out to your account team or support@evinced.com.

In your test suite you can either provide token as is (offline mode) or obtain it from Evinced licensing server (online mode).

Add following lines to your wdio.conf.js:

1import Evinced from "@evinced/webdriverio-sdk";
2

3exports.config = {
4    // ...
5    services: [[Evinced.WdioService]],
6},
1before: function (capabilities, specs) {
2    browser.evincedState = {
3        isRecording: false,
4        lastRecordingUrl: null,
5        recordingStateByUrl: {},
6        initOptions: null
7    };
8

9    // Set offline credentials method
10    Evinced.setOfflineCredentials({
11        serviceId: '<serviceID>',
12        token: 'token'
13    });
14

15    // Set online creadentials method (through Evinced lience server)
16    Evinced.setCredentials({
17        serviceId: '<serviceID>',
18        secret: '<secret>'
19    });
20},

We encourage to use environment variables to store credentials and read them in code

1# Offline mode
2export AUTH_SERVICE_ID=<serviceId>
3export AUTH_TOKEN=<token>
4

5# Online mode
6export AUTH_SERVICE_ID=<serviceId>
7export AUTH_SECRET=<secret>

Example

1// Set offline credentials method
2const serviceId = process.env.AUTH_SERVICE_ID;
3const token = process.env.AUTH_TOKEN;
4Evinced.setOfflineCredentials({
5    serviceId,
6    token
7});
8

9// OR
10

11// Set online creadentials method (through Evinced lience server)
12const serviceId = process.env.AUTH_SERVICE_ID;
13const secret = process.env.AUTH_SECRET;
14Evinced.setCredentials({
15    serviceId,
16    token
17},

In order to add import to your config file make sure that you have Babel config added to your project.

Babel Setup | WebdriverIO

Your first test

Issue Type Configuration - aXe Needs Review and Best Practices

Evinced uses two separate engines when scanning for accessibility issues, one is the aXe engine and the other is the Evinced engine. By default, Evinced disables the aXe Needs Review and Best Practices issues based on customer request and due to the fact they are mostly false positives. Please note this setting when comparing issue counts directly. Below is an example of how to enable aXe Needs Review and Best Practices issues.

1import Evinced from '@evinced-private/webdriverio-sdk';
2

3exports.config = {
4    // ...
5    services: [
6        [Evinced.WdioService, {
7        experimentalFlags:{
8          USE_AXE_NEEDS_REVIEW: true,
9          USE_AXE_BEST_PRACTICES: true
10          }
11        ]
12    ]
13};

Add Evinced accessibility checks (Single Run Mode)

This test shows how to use the evAnalyze command to scan the current page for accessibility issues. The command returns a list of issues within an array. In this example, we are verifying the expected number of accessibility issues found.

1test("evAnalyze", async () => {
2    await browser.url("https://example.com/");
3    // Scan the page for accessibility issues
4    const issues = await browser.evAnalyze();
5    expect(issues).toHaveLength(0);
6});

Add Evinced accessibility checks (Continuous Mode)

This is an example of how to add a continuous Evinced accessibility scan to a test. Using the evStart and evStop methods, the Evinced engine will continually scan in the background capturing all DOM changes and page navigations as the test is executed. This will capture all accessibility issues as clicking on drop-downs or similar interactions reveals more of the page. The advantage of continuous mode is that no interaction with the actual test code is needed.

1test("evStart/evStop", async () => {
2    await browser.url("https://example.com/");
3    await browser.evStart();
4    await $("#button-1").click();
5    await $("#button-2").click();
6    const issues = await browser.evStop();
7    expect(issues.report).toHaveLength(0);
8});

Creating an HTML report

This example shows how the results of the evAnalyze can be saved in a report file. The supported formats are: ‘html’, ‘html-v2’ and ‘json’. Following example will generate a web page that can be opened and all the issues can be viewed.

1test("create HTML report", async () => {
2    await browser.url("https://example.com/");
3    const issues = await browser.evAnalyze();
4    const filePath = path.resolve(__dirname, "evinced-report.html");
5    await browser.evSaveFile(issues, "html-v2", filePath);
6});

Validating report by issues filters

Here’s an example of excluding the issues with Critical severity from the result. The received array will have issues with all severities except Critical.

1test("apply issue filter", async () => {
2    await browser.url("https://example.com/");
3    const issues = await browser.evAnalyze({
4        filterIssues: {
5            exclude: {
6                severities: ["CRITICAL"],
7            },
8        },
9    });
10    expect(issues).toHaveLength(12);
11});

API

EvincedConfig (Evinced.WdioService config)

Default options

1{
2    rootSelector: null, // Set to scan only a subset of the DOM
3    filterIssues: null, // Specify which issues should be omitted from the report (IssuesFilter)
4    axeConfig: null // Set Axe configuration. https://github.com/dequelabs/axe-core/blob/develop/doc/API.md#api-name-axeconfigure
5}

browser.evAnalyze(options)

Default options

1// Same as EvincedConfig
2{
3    rootSelector: null,
4    filterIssues: null,
5    axeConfig: null
6}

Return value

1Promise<Issue[]>;

Example:

1const issues = await browser.evAnalyze({
2    axeConfig: {
3        rules: {
4            "link-name": { enabled: false },
5        },
6    },
7});

browser.evStart(options)

Watches for DOM mutations and records all accessibility issues until browser.evStop() is called.

Default options

1// Same as EvincedConfig
2{
3    rootSelector: null,
4    filterIssues: null,
5    axeConfig: null
6}

Example:

1await browser.evStart({
2    axeConfig: {
3        rules: {
4            "link-name": { enabled: false },
5        },
6    },
7});
8await $("#button-1").click();
9await $("#button-2").click();
10const issues = await browser.evStop();

browser.evStop()

Stops the Evinced engines and returns the issues object

Return value

1Promise<Issue[]>;

browser.evSaveFile(issues, format, destination)

Stores a report on the file system of the chosen format.

Parameters

1// Same as EvincedConfig
2issues: Issue[];
3format: 'json' | 'html' | 'html-v2';
4destination: string;

Example:

1await browser.url("https://example.com/");
2

3const issues = await browser.evAnalyze();
4

5const filePath = path.resolve(__dirname, "evinced-report.html");
6await browser.evSaveFile(issues, "html", filePath);

Evinced.inject(browser)

Injects the Evinced library into the page. Usually this is not needed since calling Evinced commands (evAnalyze() for example) will internally inject the lib if it is not already injected.

Example:

1await Evinced.inject(browser);

Types

1type Issue = {
2    id: string;
3    index: string;
4    signature: string;
5    type: {
6        id: string;
7        name: string;
8    };
9    severity: {
10        id: string;
11        name: string;
12    };
13    summary: string;
14    description: string;
15    additionalInformation: any;
16    duplicates?: string;
17    elements: IssueElement[];
18    firstSeenTime: number;
19    tags: IssueTag[];
20    knowledgeBaseLink?: string;
21};
22

23type IssueElement = {
24    componentId: string;
25    domSnippet: string;
26    id: string;
27    index: string;
28    pageUrl: string;
29    selector: string;
30};
31

32type IssueTag = {
33    description: string;
34    id: string;
35    link: string;
36};
37

38type IssuesFilter = {
39    exclude?: {
40        urls?: (string | RegExp)[]
41        selectors?: string[]
42        types?: string[] | ValidationType[]
43        severities?: IssueSeverity[]
44        signatures?: string[]
45        tags?: IssueTagId[]
46    },
47    customFilter?: CustomIssuesFilter
48};
49

50interface CustomIssuesFilter {
51    (issue: Issue): boolean
52}

Tutorials

Generating a comprehensive accessibility report for your application

In this tutorial, we will enhance our existing WebdriverIO UI test with the Evinced WebdriverIO SDK in order to check our application for accessibility issues. In order to get started you will need the following:

  1. All of the prerequisites for the Evinced WebdriverIO SDK should be met
  2. Evinced WebdriverIO SDK should be added to your project

Preface - existing UI test overview

Let’s consider the following basic UI test as our starting point.

1describe("Evinced Demo Site tests", () => {
2    test("Search Test", async () => {
3        await browser.url("https://demo.evinced.com/");
4        const BASE_FORM_SELECTOR =
5            "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container";
6        const SELECT_HOME_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > div.dropdown.line`;
7        const SELECT_WHERE_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > div.dropdown.line`;
8        const TINY_HOME_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > ul > li:nth-child(2)`;
9        const EAST_COST_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > ul > li:nth-child(3)`;
10        const SUBMIT_BUTTON = `${BASE_FORM_SELECTOR} > .search-btn`;
11        await $(SELECT_HOME_DROPDOWN).click();
12        await $(TINY_HOME_OPTION).click();
13        await $(SELECT_WHERE_DROPDOWN).click();
14        await $(EAST_COST_OPTION).click();
15        await $(SUBMIT_BUTTON).click();
16        expect(browser).toHaveUrlContaining("/results");
17    });
18});

We wrote this test for a demo travel site called TRVL that has a few known accessibility issues.

The purpose of this test is to check the functionality of the main application screen and ensure a user can successfully select their desired trip. For now, this test is only concerned with the functional testing of the app. However, with the help of the Evinced WebdriverIO SDK, we can also check it for accessibility issues along the way. Let’s go through this process with the following step-by-step instructions.

Step #1 - Verify the Evinced WebdriverIO service

Remove optional configuration in wdio.conf.js from Evinced.WdioService service.

Step #2 - Start the Evinced engine

Now that we have everything we need to scan for accessibility issues, let’s start the Evinced engine. Since we are going to use it scan throughout our test, the best place for its initialization will be after we navigate to the desired website.

1describe("Evinced Demo Site tests", () => {
2    test("Search Test", async () => {
3        await browser.url("https://demo.evinced.com/");
4        // Add command to start recording issues
5        await browser.evStart();
6        const BASE_FORM_SELECTOR =
7            "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container";
8        const SELECT_HOME_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > div.dropdown.line`;
9        const SELECT_WHERE_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > div.dropdown.line`;
10        const TINY_HOME_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > ul > li:nth-child(2)`;
11        const EAST_COST_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > ul > li:nth-child(3)`;
12        const SUBMIT_BUTTON = `${BASE_FORM_SELECTOR} > .search-btn`;
13        await $(SELECT_HOME_DROPDOWN).click();
14        await $(TINY_HOME_OPTION).click();
15        await $(SELECT_WHERE_DROPDOWN).click();
16        await $(EAST_COST_OPTION).click();
17        await $(SUBMIT_BUTTON).click();
18        expect(browser).toHaveUrlContaining("/results");
19    });
20});

Step #3 - Stop the Evinced engine and create reports

As our test was executed we collected a lot of accessibility information. We can now perform accessibility assertions at the end of our test suite. Referring back again to our UI test the best place for this assertion will be the end of test actions. To stop the Evinced engine and generate the actual object representation of your accessibility report simply call the evStop() method. We can then output the report files in JSON format.

1describe("Evinced Demo Site tests", () => {
2    test("Search Test", async () => {
3        await browser.url("https://demo.evinced.com/");
4        // Add command to start recording issues
5        await browser.evStart();
6        const BASE_FORM_SELECTOR =
7            "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container";
8        const SELECT_HOME_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > div.dropdown.line`;
9        const SELECT_WHERE_DROPDOWN = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > div.dropdown.line`;
10        const TINY_HOME_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(1) > div > ul > li:nth-child(2)`;
11        const EAST_COST_OPTION = `${BASE_FORM_SELECTOR} > div:nth-child(2) > div > ul > li:nth-child(3)`;
12        const SUBMIT_BUTTON = `${BASE_FORM_SELECTOR} > .search-btn`;
13        await $(SELECT_HOME_DROPDOWN).click();
14        await $(TINY_HOME_OPTION).click();
15        await $(SELECT_WHERE_DROPDOWN).click();
16        await $(EAST_COST_OPTION).click();
17        await $(SUBMIT_BUTTON).click();
18        expect(browser).toHaveUrlContaining("/results");
19        // Add a command to stop recording and retrieve issues list
20        const issues = await browser.evStop();
21        // Save issues to a report file
22        await browser.evSaveFile(issues, "json", "./test/issues.json");
23        // Assert issues count
24        expect(issues).toHaveLength(0);
25    });
26});

The demo website contains several accessibility issues for the showcase purpose. With the recording we will be able to identify the number of issues and verify their count. For more information regarding reports as well as the Report object itself, please see our detailed Web Reports page.

You are now set to run the test and ensure the accessibility of your application! So, go ahead and run it via your IDE or any other tooling you use for JavaScript development.

Support

Please feel free to reach out to support@evinced.com with any questions.

FAQ

  1. Do you have support for other JavaScript automation frameworks? Yes! We have SDK versions for Cypress and Protractor as well.
  2. Can I configure which validations to run? Yes, see the API section for details on how to configure validations to your needs.
  3. Can I run tests with Evinced using cloud-based services like Sauce Labs, Perfecto, or BrowserStack? Yes, we have tested the Evinced SDK on many of these types of cloud-based services and expect no issues.