Developer Documentation

Playwright Java SDK

The Evinced Playwright Java SDK integrates with new or existing Playwright tests to automatically detect accessibility issues. By adding a few lines of code to your Playwright project, you can begin analyzing all the web pages and DOM changes to provide a dynamic view of how your site can become more accessible. As a result of the test, a rich and comprehensive report is generated to easily track issues to resolution.

Interested in seeing this in action? Contact us to get started!

Prerequisites

  • Playwright version 1.29 or higher

Get Started

Installation

Install Playwright Java SDK from a local ZIP archive

You can perform installation right from the standalone .jar distribution. In this case, you need to do the following:

  1. Download the .jar file.
  2. Unpack the provided playwright-java-sdk.zip to any desirable location
  3. Add the following dependencies entries pointing to java-playwright-sdk-version.jar
    1. Gradle:
      1 implementation files('/Users/<path-to-your-upacked-folder>/java-playwright-sdk-<version>.jar')
    2. Maven:
      1. First, install the “all“ jar into your local Maven repository by the following command:
        1mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file –Dfile=java-playwright-sdk-<version>.jar -DpomFile=java-playwright-sdk-<version>.pom
        Example:
        1mvn org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file -Dfile=java-playwright-sdk-1.6.1.jar -DpomFile=java-playwright-sdk-1.6.1.pom
      2. Add the corresponding dependency into your pom.xml:
        1 <dependency>
        2     <groupId>com.evinced</groupId>
        3     <artifactId>java-playwright-sdk</artifactId>
        4     <version>1.6.1</version>
        5 </dependency>

Your First Test

SDK Initialization

To work with Playwright Java SDK you need to have authentication token. Please refer to licensing section for details. The only command you need to add is to wrap the existing WebDriver object.

1Playwright playwright = Playwright.create();
2Browser browser = playwright.chromium().launch();
3EvPage page = EvPageFactory.create(browser.newPage());

Add Evinced accessibility checks (Single Run Mode)

This is a simple example of how to add an Evinced accessibility scan to a test. Please note the inline comments that give detail on each test step.

1@Test
2public void evAnalyzeTest() {
3    Playwright playwright = Playwright.create();
4    Browser browser = playwright.chromium().launch();
5    EvPage page = EvPageFactory.create(browser.newPage());
6

7    try{
8        page.navigate("https://demo.evinced.com");
9        Report report = page.evAnalyze();
10        List<Issue> issues = report.getIssues();
11        assertEquals(6, issues.size());
12    } finally {
13        playwright.close();
14    }
15}

Add Evinced accessibility checks (Continuous Mode)

This is an example of how to add a continuous Evinced accessibility scan to a test. Using the evStart() and evStop() methods, the Evinced engine will continually scan in the background capturing all DOM changes and page navigations as the test is executed. This will capture all accessibility issues as clicking on drop-downs or similar interactions reveals more of the page. The advantage of continuous mode is that no interaction with the actual test code is needed.

1@Test
2public void evAnalyzeTest() {
3    Playwright playwright = Playwright.create();
4    Browser browser = playwright.chromium().launch();
5    EvPage page = EvPageFactory.create(browser.mewPage());
6

7    try{
8        page.navigate("https://demo.evinced.com");
9        page.evStart();
10

11        // More test code to interact with the page
12

13        Report report = page.evStop();
14        List<Issue> issues = report.getIssues();
15        assertEquals(6, issues.size());
16    } finally {
17        playwright.close();
18    }
19}

API

global initialization

Initializes the Evinced object within the project.

Example

1EvPage page = EvPageFactory.create(browser.newPage());

Please refer to configuration to see examples of using init with options.

evAnalyze(options)

Scans the current page and returns a list of accessibility issues.

Note: This method is not supported if evStart() is already running.

Please refer to configuration to see examples of using init with options.

Example

1EvPage page = EvPageFactory.create(browser.newPage());
2Report report = page.evAnalyze();

Return value

1Report

A Report object is returned containing accessibility issues. This is the recommended method for static page analysis. For more information regarding reports as well as the Report object itself, please refer to our detailed Web Reports page.

evStart(options)

Continually watches for DOM mutations and page navigation and records all accessibility issues until the evStop() method is called. This method is recommended for dynamic page flows.

Example

1EvPage page = EvPageFactory.create(browser.newPage());
2page.evStart();

Return value

1void

evStop(options)

Stops the process of issue gathering started by evStart() command.

Example

1EvPage page = EvPageFactory.create(browser.newPage());
2page.evStart();
3Report report = page.evStop();

Return value

1Report

Returns an object containing recorded accessibility issues from the point in time at which the evStart() method was instantiated. For more information regarding reports as well as the returned object itself, please refer to detailed Web Reports page.

evSaveFile(destination, issues, format)

Saves list of the issues in a file, with the specified format and location. For example, format could be ‘json’ or 'html. Please find detailed information on Web Reports page.

Example

1Report report = page.evStop();
2// OR
3Report report = page.evAnalyze();
4

5// create a JSON file named jsonReport.json
6EvincedSDK.evSaveFile("jsonReport", report, FileFormat.JSON);
7

8// create an HTML file named htmlReport.html
9EvincedSDK.evSaveFile("htmlReport", report, FileFormat.HTML);
10

11// create an SARIF file named sarifReport.html
12EvincedSDK.evSaveFile("sarifReport", report, FileFormat.SARIF);

FileFormat

Determines the file type of the report. Options are JSON, HTML and SARIF.

Return value

1Path

The Path object representing the location where the result file was stored.

Aggregated Report

The aggregated report feature allows you to have a general aggregated report for the whole run (not only for one test or suite). This report will contain all the issues found by the tests where evStart() and evStop() commands were called. It is still possible to use the evSaveFile() command in any place of your code along with this Aggregated Report feature.

Example

1Path htmlAggregatedReport = EvincedSDK.evSaveFile("evinced-html-report", FileFormat.HTML);

Licensing

To work with Playwright Java SDK you need to have Authentication token. There are to methods to provide the token: offline mode and online mode. The difference is if you use online mode the request to obtain the token will be sent to Evinced Licencing Server. You need to provide Service ID and Secret API key in such a case. They are available via the Evinced Product Hub.

Offline mode assumes that you already have token and can provide it directly. If an offline token is required please reach out to your account team or support@evinced.com.

We encourage to use environment variables to store credentials and read them in code

1# Offline mode
2export AUTH_SERVICE_ID=<serviceId>
3export AUTH_TOKEN=<token>
4

5# Online mode
6export AUTH_SERVICE_ID=<serviceId>
7export AUTH_SECRET=<secret>

Example

1using Evinced.SDK;
2

3// Offline mode
4EvincedSDK.SetOfflineCredentials(Environment.GetEnvironmentVariable("AUTH_SERVICE_ID"), Environment.GetEnvironmentVariable("AUTH_TOKEN"));
5

6// Online mode (not yet implemented)
7EvincedSDK.SetCredentials(Environment.GetEnvironmentVariable("AUTH_SERVICE_ID"), Environment.GetEnvironmentVariable("AUTH_SECRET"));

Configuration

The same configuration object can be used in global initialization, evStart() and evAnalyze() methods but with a bit different consequences. By providing some options in global initialization method you define a global configuration for all calls of evStart() or evAnalyze() methods. Please note that global configuration is not intended to be altered from tests due to it will affect all the rest tests as well as tests running in parallel threads. To alter configuration in specific tests you can provide config on the method's level, for instance with evStart() it defines local configuration for this particular session until evStop() is called. If provided in both levels, the command's level config will override the global one.

Engines configuration

Evinced uses two separate engines when scanning for accessibility issues, one is the aXe engine and the other is the Evinced engine. By default, Evinced disables the aXe Needs Review and Best Practices issues based on customer request and due to the fact they are mostly false positives. Please note this setting when comparing issue counts directly. See an example of how to enable Needs Review and Best practices issues in our toggles section.

Configuration object type

Global.config is a global configuration of type EvConfig that will be used with all commands by default.

1Global.config.setRootSelector("#some-selector");

The configurations can be passed to a specific command, such as evAnalyze() or evStart(), it will override the global configuration for a single run.

1EvConfig configuration = new EvConfig();
2configuration.setRootSelector(".block-dropdown");
3evPage.evStart(configuration);

Root Selector

Instructs engine to return issues which belong to or which are nested to element with the specified selector. By default the issues for all elements from document root will be returned.

null by default.

Example

1Global.config.setRootSelector("#some-selector");

AXE Configuration

Evinced leverages Axe open-source accessibility toolkit as part of its own accessibility detection engine. The Evinced engine is able to pinpoint far more issues than Axe alone.

For the full Axe config options, see Axe Core API.

Example

[Work in progress placeholder] 🔴 TODO 🔴

Errors Strict Mode

When set to true it switches SDK in mode when Evinced SDK errors are thrown as runtime errors and stops current test execution. Otherwise errors are printed into console except critical ones.

false by default

Example

[Work in progress placeholder] 🔴 TODO 🔴

Engine Logging

Switches logging on for engine. Log messages from the engine will be printed to console.

Levels are: debug, info, warn, error

null by default.

error is a default level on engine side.

[Work in progress placeholder] 🔴 TODO 🔴

Reports Screenshots

The Screenshots feature allows to include screenshots to the Evinced reports. When the feature is enabled, Evinced will take screenshots of the page, highlight an element with a specific accessibility issue, and include them to the report.

screenshots_feature

false by default.

Note: The Screenshots feature may affect the test run performance.

Enabling the Screenshots feature

[Work in progress placeholder] 🔴 TODO 🔴

Toggles

Toggles are feature flags which controls SDK and Engine behavior. With toggles we introduce experimental features or manage the way how accessibility issues are gathered. The list of toggles is not specified so we suggest not to rely on specific toggle name or value and use them in invetigation puprposes only.

Example enabling aXe Best Practices and Needs Review Issues

[Work in progress placeholder] 🔴 TODO 🔴

Skip Validations

Skips specific validations for the given URL pattern and the selector.

SkipValidation[] by default.

Example

[Work in progress placeholder] 🔴 TODO 🔴

How to identify ID for the specific issue type is described in tutorial section.

Knowledge Base Link overrides

This custom parameter helps to customize knowledge base links in the reports. Those links are displayed in the reports as follows:

kb_links

The knowledge base link can be overridden for every issue type ID.

Example

[Work in progress placeholder] 🔴 TODO 🔴

Where WRONG_SEMANTIC_ROLE is ID of the type of the specific issue. In this example, the corresponding issue type for this ID is Interactable Role. How to identify ID for the specific issue type is described in tutorial section. After passing the type ID and a custom knowledge base link we are set to see it in the reports.

Shadow DOM support

Example

[Work in progress placeholder] 🔴 TODO 🔴 When set to true, the accessibility tests run analysis on iframes that exist inside the page. Default is false. [Work in progress placeholder] 🔴 TODO 🔴

Global Switch

Global Switch allows to disable or enable Evinced functionality. It could be needed, for example, while working on functional tests in your local environment or for running some CI jobs that are not intended to gather information regarding accessibility issues.

When switched off

  • evStart() and evSaveFile() will be bypassed.
  • evStop() and evAnalyze() will return an empty report.
  • evValid() will never fail.

true by default.

Important! Global Switch environment variable overrides the global configuration option.

Switch on/off Evinced functionality in config

[Work in progress placeholder] 🔴 TODO 🔴

Switching off Evinced functionality with environment variable

1export EV_SWITCH_ON=false

Tutorials

You can find fully functional example projects on our GitHub

Step-by-step adding Evinced to existing test suite.

Preface: Existing test suite

1package tutorial;
2

3import com.microsoft.playwright.Browser;
4import com.microsoft.playwright.Page;
5import com.microsoft.playwright.Playwright;
6import org.junit.jupiter.api.*;
7

8import static org.junit.jupiter.api.Assertions.assertTrue;
9

10

11public class TutorialTest1 {
12    private static Playwright playwright;
13    private static Browser browser;
14    private Page page;
15

16    private interface Selectors {
17            String HOUSE_DROPDOWN = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(1) > div > div.dropdown.line";
18            String TENT_OPTION = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(1) > div > ul > li:nth-child(4)";
19            String LOCATION_DROPDOWN = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(2) > div > div.dropdown.line";
20            String CANADA_OPTION = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(2) > div > ul > li:nth-child(1)";
21            String SEARCH_BUTTON = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > a";
22            String SEARCH_RESULTS = "#gatsby-focus-wrapper > main > h1";
23    }
24

25

26    @BeforeAll
27    private static void launchBrowser() {
28        playwright = Playwright.create();
29        browser = playwright.chromium().launch();
30    }
31

32    @AfterAll
33    private static void closeBrowser() {
34        playwright.close();
35    }
36

37    @BeforeEach
38    private void setUp(){
39        page = browser.newPage();
40    }
41

42    @AfterEach
43    private void tearDown(){
44        page.close();
45    }
46

47    @Test
48    public void tutorialTest1(){
49        page.navigate("https://demo.evinced.com");
50        assertTrue(page.isVisible(Selectors.SEARCH_BUTTON));
51    }
52

53    @Test
54    public void tutorialTest2(){
55        page.navigate("https://demo.evinced.com");
56        page.click(Selectors.HOUSE_DROPDOWN);
57        page.click(Selectors.TENT_OPTION);
58        page.click(Selectors.LOCATION_DROPDOWN);
59        page.click(Selectors.CANADA_OPTION);
60        page.click(Selectors.SEARCH_BUTTON);
61        page.waitForLoadState();
62        assertTrue(page.isVisible(Selectors.SEARCH_RESULTS));
63    }
64}

Step 1: Add Evinced, use snapshot scan and partial report

1package tutorial;
2

3import com.evinced.*;
4import com.microsoft.playwright.Browser;
5import com.microsoft.playwright.Playwright;
6import org.junit.jupiter.api.*;
7

8import static org.junit.jupiter.api.Assertions.assertEquals;
9import static org.junit.jupiter.api.Assertions.assertTrue;
10

11

12public class TutorialTest2 {
13    private static Playwright playwright;
14    private static Browser browser;
15    // Change type from Page to EvPage
16    private EvPage page;
17

18    private interface Selectors {
19            String HOUSE_DROPDOWN = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(1) > div > div.dropdown.line";
20            String TENT_OPTION = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(1) > div > ul > li:nth-child(4)";
21            String LOCATION_DROPDOWN = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(2) > div > div.dropdown.line";
22            String CANADA_OPTION = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(2) > div > ul > li:nth-child(1)";
23            String SEARCH_BUTTON = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > a";
24            String SEARCH_RESULTS = "#gatsby-focus-wrapper > main > h1";
25    }
26

27

28    @BeforeAll
29    private static void launchBrowser() {
30        playwright = Playwright.create();
31        browser = playwright.chromium().launch();
32        // Add evinced licensing
33        EvincedSDK.setOfflineCredentials(System.getenv("AUTH_SERVICE_ID"), System.getenv("AUTH_TOKEN"));
34    }
35

36    @AfterAll
37    private static void closeBrowser() {
38        playwright.close();
39    }
40

41    @BeforeEach
42    private void setUp(){
43        // Wrap Playwright's page
44        // You can access the original page with page.getWrappedPage();
45        page = EvPageFactory.create(browser.newPage());
46    }
47

48    @AfterEach
49    private void tearDown(){
50        page.close();
51    }
52

53    @Test
54    public void tutorialTest1(){
55        page.navigate("https://demo.evinced.com");
56        assertTrue(page.isVisible(Selectors.SEARCH_BUTTON));
57        // Get issues snapshot on the page
58        Report report = page.evAnalyze();
59        // Save partial report for the single scan
60        EvincedSDK.evSaveFile("/tmp/ev-standalone-report.json", report, FileFormat.JSON);
61    }
62

63    @Test
64    public void tutorialTest2(){
65        page.navigate("https://demo.evinced.com");
66        page.click(Selectors.HOUSE_DROPDOWN);
67        page.click(Selectors.TENT_OPTION);
68        page.click(Selectors.LOCATION_DROPDOWN);
69        page.click(Selectors.CANADA_OPTION);
70        page.click(Selectors.SEARCH_BUTTON);
71        page.waitForLoadState();
72        assertTrue(page.isVisible(Selectors.SEARCH_RESULTS));
73        // Get issues snapshot on the result page
74        Report report = page.evAnalyze();
75        // Assert page does not contain accessibility issues
76        assertEquals(0, report.getIssues().size());
77    }
78}

Step 2: Use continuous mode and aggregated report

Let's do a step back to the original test and start modifying it again. Some steps will be similar.

1package tutorial;
2

3import com.evinced.*;
4import com.microsoft.playwright.Browser;
5import com.microsoft.playwright.Playwright;
6import org.junit.jupiter.api.*;
7

8import static org.junit.jupiter.api.Assertions.assertTrue;
9

10

11public class TutorialTest3 {
12    private static Playwright playwright;
13    private static Browser browser;
14    // Change type from Page to EvPage
15    private EvPage page;
16

17    private interface Selectors {
18            String HOUSE_DROPDOWN = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(1) > div > div.dropdown.line";
19            String TENT_OPTION = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(1) > div > ul > li:nth-child(4)";
20            String LOCATION_DROPDOWN = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(2) > div > div.dropdown.line";
21            String CANADA_OPTION = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > div:nth-child(2) > div > ul > li:nth-child(1)";
22            String SEARCH_BUTTON = "#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container > a";
23            String SEARCH_RESULTS = "#gatsby-focus-wrapper > main > h1";
24    }
25

26

27    @BeforeAll
28    private static void launchBrowser() {
29        playwright = Playwright.create();
30        browser = playwright.chromium().launch();
31        // Add evinced licensing
32        EvincedSDK.setOfflineCredentials(System.getenv("AUTH_SERVICE_ID"), System.getenv("AUTH_TOKEN"));
33    }
34

35    @AfterAll
36    private static void closeBrowser() {
37        playwright.close();
38        // Save the aggregated report for all the issues found during test run
39        EvincedSDK.evSaveFile("/tmp/ev-aggregated-report.html", FileFormat.HTML);
40    }
41

42    @BeforeEach
43    private void setUp(){
44        // Wrap Playwright's page
45        // You can access the original page with page.getWrappedPage();
46        page = EvPageFactory.create(browser.newPage());
47        // Start gathering issues
48        page.evStart();
49    }
50

51    @AfterEach
52    private void tearDown(){
53        // Stop gathering issues
54        // If you want you can work with returned set of issues gathered between evStart and evStop
55        Report report = page.evStop();
56        System.out.println("Found issues: " + report.getIssues().size());
57        page.close();
58    }
59

60    @Test
61    public void tutorialTest1(){
62        page.navigate("https://demo.evinced.com");
63        assertTrue(page.isVisible(Selectors.SEARCH_BUTTON));
64    }
65

66    @Test
67    public void tutorialTest2(){
68        page.navigate("https://demo.evinced.com");
69        page.click(Selectors.HOUSE_DROPDOWN);
70        page.click(Selectors.TENT_OPTION);
71        page.click(Selectors.LOCATION_DROPDOWN);
72        page.click(Selectors.CANADA_OPTION);
73        page.click(Selectors.SEARCH_BUTTON);
74        page.waitForLoadState();
75        assertTrue(page.isVisible(Selectors.SEARCH_RESULTS));
76    }
77}

Support

Please feel free to reach out to support@evinced.com with any questions.

FAQ

  1. Can I configure which validations to run?

Yes, see the configuration section for details on how to configure Axe validations to your needs.

  1. Can I run tests with Evinced using cloud-based services like Sauce Labs, Perfecto, or BrowserStack?

Yes, we have tested the Evinced SDK on many of these types of cloud-based services and expect no issues.