Selenium Java SDK

The Evinced Selenium Java SDK integrates with new or existing Selenium tests to automatically detect accessibility issues. With the addition of as few as 5 lines of code to your Selenium framework, you can begin to analyze all the pages and DOM changes to offer a dynamic view of how your site can become more accessible. At the conclusion of the test, a rich and comprehensive JSON or HTML report is generated to track issues in any reporting tool.

Interested in seeing this in action? Contact us to get started!

Prerequisites

  • Selenium version 3.141.59 or higher
  • Java Version 1.8 or higher
  • ChromeDriver (More coming soon!)

Get Started

Install Evinced Selenium SDK from a local ZIP archive

You can perform installation right from the standalone .jar distribution. In this case, you need to do the following:

  1. Download the .jar file.
  2. Unpack the provided selenium-sdk.zip to any desirable location
  3. Add the following dependencies entries pointing to selenium-sdk-all.jar
    1. Gradle:
      1 implementation files('/Users/<path-to-your-upacked-folder>/selenium-sdk.jar'
    2. Maven:
      1. First, install the “all“ jar into your local Maven repository by the following command:
        1 mvn install:install-file –Dfile=/Users/<path-to-unpacked-folder>/selenium-sdk.jar -DgroupId=com.evinced -DartifactId=java-selenium -Dversion=0.0.23
      2. Add the corresponding dependency into your pom.xml:
        1 <dependency>
        2 <groupId>com.evinced</groupId>
        3 <artifactId>java-selenium</artifactId>
        4 <version>0.0.26</version>
        5 </dependency>
  4. Trigger your package manager download new dependencies either via IDE or command line

ZIP archive also contains two complimentary .jar files which you can utilize:

  • selenium-sdk-javadoc.jar contains rich JavaDoc documentation which you can view in any web browser
  • selenium-sdk-sources.jar can be attached to your Evinced Selenium SDK installation in order to let you leverage JavaDoc right in your IDE. Please, refer to the corresponding documentation of your developer tools.

Your first test

Initialize the SDK object

Add the import statement to the beginning of your test file.

1import com.evinced.EvincedWebDriver;

Under the hood, the Evinced SDK utilizes several Selenium commands in order to get accessibility data. For this reason, we need to initialize EvincedWebDriver with the ChromeDriver instance in the following way:

1EvincedWebDriver driver = new EvincedWebDriver(new ChromeDriver());

Add Evinced accessibility checks (Single Run Mode)

This is a simple JUnit example of how to add a single Evinced accessibility scan to a test. Please note the inline comments that give detail on each test step.

1 @Test
2 public void evReportSimpleTest() {
3 // Initialize EvincedWebDriver which wraps a ChromeDriver instance
4 EvincedWebDriver driver = new EvincedWebDriver(new ChromeDriver());
5
6 // Navigate to the site under test
7 driver.get("https://www.google.com");
8
9 // Run analysis and get the accessibility report
10 Report report = driver.evReport();
11
12 // Assert that there are no accessibility issues
13 assertTrue(report.getIssues().size() == 0);
14 }

Add Evinced accessibility checks (Continuous Mode)

This is a simple JUnit example of how to add a continuous Evinced accessibility scan to a test. Using the evStart and evStop methods, the Evinced engine will continually scan in the background capturing all DOM changes as the test is executed. This will capture all accessibility issues as clicking on dropdowns or similar interactions reveals more of the page. The advantage of continuous mode is that no interaction with the actual test code is needed.

1@BeforeClass
2public static void setUp() throws MalformedURLException, IOException {
3 driver = new EvincedWebDriver(new ChromeDriver());
4 driver.setupCredentials(<your service account ID>, <your API key>);
5 // Start the Evinced engine scanning for accessibility issues
6 driver.evStart();
7 //... the rest of our setup code
8}
9
10@AfterClass
11public static void tearDown() throws MalformedURLException, IOException {
12 // Stop the Evinced engine
13 Report report = driver.evStop();
14 //Output the Evinced report in either JSON or HTML format
15 EvincedReporter.writeEvResultsToFile("test-results", report, EvincedReporter.FileFormat.HTML);
16 EvincedReporter.writeEvResultsToFile("test-results", report, EvincedReporter.FileFormat.JSON);
17 driver.quit();
18}
19
20@Test
21public void evReportSimpleTest() {
22 // Navigate to the site under test
23 driver.get("https://www.google.com");
24 // More test code...
25}

API

EvincedWebDriver

constructor

Default constructor

The EvincedWebDriver constructor expects an instance of ChromeWebDriver. It is possible to pass an instance of a class that inherits or extends ChromeDriver.

1ChromeDriver chromeDriver = new ChromeDriver();
2EvincedWebDriver driver = new EvincedWebDriver(chromeDriver);
Additional configuration

An optional second parameter is an EvincedConfiguration. When configurations are passed via the constructor, they are set as a default for all future actions.

In the case that other configurations are passed to a specific function, such as evReport, they will override the constructor configurations for that specific operation.

See the EvincedConfiguration options section below for more details.

1ChromeDriver chromeDriver = new ChromeDriver();
2EvincedConfiguration configuration = new EvincedConfiguration();
3EvincedWebDriver driver = new EvincedWebDriver(chromeDriver, configuration);

evReport

Scans the current page and returns a list of accessibility issues. A Report object is returned containing accessibility issues. This is the recommended method for static page analysis. Note: This method is not supported if evStart is already running.

1Report report = evincedWebDriver.evReport();

When the EvincedConfiguration object is provided to the evReport method it will be used for this run only (replacing it, not merging), and will not be saved for future runs.

1Report report = evincedWebDriver.evReport(configuration);

evStart

Continually watches for DOM mutations and page navigation and records all accessibility issues until the evStop() method is called. This method is recommended for dynamic page flows.

1evincedWebDriver.evStart();

When the EvincedConfiguration object is provided to the evStart method it will be used for this run only (replacing it, not merging), and will not be saved for future runs.

1Report report = evStart.evReport(configuration);

evStop

Returns a Report object containing recorded accessibility issues from the point in time at which the evStart method was instantiated.

1Report report = evincedWebDriver.evStop();

setLogLevel

Sets the log level for Evinced code. Default is no logging.

1evincedWebDriver.setLogLevel(Level.ERROR); // or any other level such as Level.DEBUG

EvincedReporter

Manages report file creation. Add the import to your file:

1import com.evinced.EvincedWebDriver;

writeEvResultsToFile

Writes the accessibility report to a file. Returns the Path to the created file.

Two formats are available - HTML and JSON.

1 Report report = evincedWebDriver.evStop();
2 -- or --
3 Report report = evincedWebDriver.evReport();
4
5 // create a JSON file named jsonReport.json
6 EvincedReporter.writeEvResultsToFile("jsonReport", report, EvincedReporter.FileFormat.JSON);
7
8 // create an HTML file named htmlReport.html
9 EvincedReporter.writeEvResultsToFile("htmlReport", report, EvincedReporter.FileFormat.HTML);

FileFormat

Determines the file type of the outputted report. Options are JSON and HTML.

1// create a JSON file named jsonReport.json
2EvincedReporter.writeEvResultsToFile("jsonReport", report, EvincedReporter.FileFormat.JSON);
3
4// create an HTML file named htmlReport.html
5EvincedReporter.writeEvResultsToFile("htmlReport", report, EvincedReporter.FileFormat.HTML);

EvincedConfiguration

Use the EvincedConfiguration object to pass configuration parameters to the accessibility engines. When passing the configuration to the constructor it will be the default for future evStart and evReport commands.

1ChromeDriver chromeDriver = new ChromeDriver();
2EvincedConfiguration configuration = new EvincedConfiguration();
3EvincedWebDriver driver = new EvincedWebDriver(chromeDriver, configuration);

When passing the configuration to the evStart or evReport commands it will temporarily override the constructor configurations (replacing it, not merging).

setRootSelector

Choose a single CSS selector to run the analysis, For example - run analysis on the element that holds only the menu bar. When no configuration is passed, it will scan the entire page. Default is null.

1EvincedConfiguration configuration = new EvincedConfiguration();
2configuration.setRootSelector(".some-selector"); // css selector
3EvincedWebDriver driver = new EvincedWebDriver(chromeDriver, configuration);

axeConfig

Evinced leverages Axe open-source accessibility toolkit as part of its own accessibility detection engine. The Evinced engine is able to pinpoint far more issues than Axe alone. See Axe Configuration Options

1EvincedConfiguration configuration = new EvincedConfiguration();
2AxeConfiguration axeConfig = new AxeConfiguration();
3
4// Axe's syntax to make `html-has-lang` validation disabled
5axeConfig.setRules(Collections.singletonMap("html-has-lang", Collections.singletonMap("enabled", false)));
6configuration.setAxeConfig(axeConfig);
7
8// passing axe config on constructor
9EvincedWebDriver driver = new EvincedWebDriver(chromeDriver, configuration);
10
11-- or --
12// passing axe config via evStart
13EvincedWebDriver driver = new EvincedWebDriver(chromeDriver);
14driver.evStart(configuration);

includeIframes [BETA]

When set to true, the accessibility tests run analysis on iframes that exist inside the page. Default is false.

1EvincedConfiguration configuration = new EvincedConfiguration();
2configuration.setIncludeIframes(true);
3driver.evStart(configuration);
4
5// the report will include issues that exist on a page or in iframes inside it
6Report report = driver.evStop();

setIncludeShadowDOM [BETA]

When set to true, the accessibility tests run analysis on shadow DOM elements that exist inside the page. Default is false.

1EvincedConfiguration configuration = new EvincedConfiguration();
2configuration.setIncludeShadowDOM(true);
3driver.evStart(configuration);
4
5// the report will include issues that exist on a page and its shadow DOM elements
6Report report = driver.evStop();

Tutorials

Generating a comprehensive accessibility report for your application

In this tutorial, we will enhance our existing Selenium UI test with the Evinced Selenium SDK in order to check our application for accessibility issues. In order to get started you will need the following:

  1. All of the prerequisites for the Evinced Selenium SDK should be met
  2. Evinced Selenium SDK should be added to your project

Preface - existing UI test overview

Let’s consider the following basic UI test as our starting point.

1import com.evinced.EvincedReporter;
2import com.evinced.EvincedWebDriver;
3import com.evinced.dto.results.Issue;
4import com.evinced.dto.results.Report;
5import org.junit.Assert;
6import org.junit.Test;
7import org.openqa.selenium.By;
8import org.openqa.selenium.WebElement;
9import org.openqa.selenium.chrome.ChromeDriver;
10import java.util.List;
11
12public class AccessibilityTests {
13 @Test
14 public void evReportWithAnOpenDropdownTest() {
15 ChromeDriver driver = new ChromeDriver();
16 driver.get("https://demo.evinced.com/");
17 WebElement firstDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(1) > div > div.dropdown.line"));
18 firstDropdown.click();
19 WebElement secondDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(2) > div > div.dropdown.line"));
20 secondDropdown.click();
21 driver.findElement(By.cssSelector(".react-date-picker")).click();
22 }
23}

We wrote this test for a demo travel site called TRVL that has a few known accessibility issues.

The purpose of this test is to check the functionality of the main application screen and ensure a user can successfully select their desired trip. For now, this test is only concerned with the functional testing of the app. However, with the help of the Evinced Selenium SDK, we can also check it for accessibility issues along the way. Let’s go through this process with the following step-by-step instructions.

Step #1 - Initialize the Evinced Selenium SDK

Before making any assertions against our app, we need to initialize EvincedWebDriver object. This object is used primarily as an entry point to all of the accessibility scanning features. Since we are going to use it scan throughout our test, the best place for its initialization will be our setUp method which gets executed first.

1import com.evinced.EvincedReporter;
2import com.evinced.EvincedWebDriver;
3import com.evinced.dto.results.Issue;
4import com.evinced.dto.results.Report;
5import org.junit.Assert;
6import org.junit.Test;
7import org.openqa.selenium.By;
8import org.openqa.selenium.WebElement;
9import org.openqa.selenium.chrome.ChromeDriver;
10import com.evinced.EvincedWebDriver;
11import java.util.List;
12
13public class AccessibilityTests {
14 private EvincedWebDriver driver;
15
16 @Before
17 public void setUp() {
18 driver = new EvincedWebDriver(new ChromeDriver());
19 }
20
21 @Test
22 public void evReportWithAnOpenDropdownTest() {
23 driver.get("https://demo.evinced.com/");
24 WebElement firstDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(1) > div > div.dropdown.line"));
25 firstDropdown.click();
26 WebElement secondDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(2) > div > div.dropdown.line"));
27 secondDropdown.click();
28 driver.findElement(By.cssSelector(".react-date-picker")).click();
29 }
30}

The EvincedWebDriver requires only an instance of ChromeDriver.

Step #2 - Start the Evinced engine

Now that we have initiated the Selenium driver and added the capabilities needed to scan for accessibility issues, let’s start the Evinced engine. Since we are going to use it scan throughout our test, the best place for its initialization will be our setUp method after our Selenium driver is configured.

1import com.evinced.EvincedReporter;
2import com.evinced.EvincedWebDriver;
3import com.evinced.dto.results.Issue;
4import com.evinced.dto.results.Report;
5import org.junit.Assert;
6import org.junit.Test;
7import org.openqa.selenium.By;
8import org.openqa.selenium.WebElement;
9import org.openqa.selenium.chrome.ChromeDriver;
10import com.evinced.EvincedWebDriver;
11import java.util.List;
12
13public class AccessibilityTests {
14 private EvincedWebDriver driver;
15
16 @Before
17 public void setUp() {
18 driver = new EvincedWebDriver(new ChromeDriver());
19 // Start the Evinced Engine
20 driver.evStart();
21 }
22
23 @Test
24 public void evReportWithAnOpenDropdownTest() {
25 driver.get("https://demo.evinced.com/");
26 WebElement firstDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(1) > div > div.dropdown.line"));
27 firstDropdown.click();
28 WebElement secondDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(2) > div > div.dropdown.line"));
29 secondDropdown.click();
30 driver.findElement(By.cssSelector(".react-date-picker")).click();
31 }
32}

There are additional configuration options that can be included as well. More information can be found in the API section of this documentation.

Step #3 - Stop the Evinced engine and create reports

As our test was executed we collected a lot of accessibility information. We can now perform accessibility assertions at the end of our test suite. Referring back again to our UI test the best place for this assertion will be the method that gets invoked last - tearDown. To stop the Evinced engine and generate the actual object representation of your accessibility report simply call the evStop() method. We can then output the report files in either HTML or JSON format (or both!).

1import com.evinced.EvincedReporter;
2import com.evinced.EvincedWebDriver;
3import com.evinced.dto.results.Issue;
4import com.evinced.dto.results.Report;
5import org.junit.Assert;
6import org.junit.Test;
7import org.openqa.selenium.By;
8import org.openqa.selenium.WebElement;
9import org.openqa.selenium.chrome.ChromeDriver;
10import com.evinced.EvincedWebDriver;
11import java.util.List;
12
13public class AccessibilityTests {
14 private EvincedWebDriver driver;
15
16 @Before
17 public void setUp() {
18 driver = new EvincedWebDriver(new ChromeDriver());
19 // Start the Evinced Engine
20 driver.evStart();
21 }
22
23 @After
24 public void tearDown() {
25 // Stop the Evinced Engine
26 Report report = driver.evStop();
27 // Output the Accessibility results in JSON or HTML
28 EvincedReporter.writeEvResultsToFile(testName.getMethodName(), report, EvincedReporter.FileFormat.HTML);
29 EvincedReporter.writeEvResultsToFile(testName.getMethodName(), report, EvincedReporter.FileFormat.JSON);
30 driver.quit();
31 }
32
33 @Test
34 public void evReportWithAnOpenDropdownTest() {
35 driver.get("https://demo.evinced.com/");
36 WebElement firstDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(1) > div > div.dropdown.line"));
37 firstDropdown.click();
38 WebElement secondDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(2) > div > div.dropdown.line"));
39 secondDropdown.click();
40 driver.findElement(By.cssSelector(".react-date-picker")).click();
41 }
42}

For the sake of simplicity of this tutorial let’s simply assume that our application is accessible as long as it has no accessibility issues found. Thus, if we have at least one accessibility issue detected - we want our tests to be failed. Let’s add the corresponding assertion to our tearDown method

1@After
2public void tearDown() {
3 // Stop the Evinced Engine
4 Report report = driver.evStop();
5 // Output the Accessibility results in JSON or HTML
6 EvincedReporter.writeEvResultsToFile(testName.getMethodName(), report, EvincedReporter.FileFormat.HTML);
7 EvincedReporter.writeEvResultsToFile(testName.getMethodName(), report, EvincedReporter.FileFormat.JSON);
8
9 // Optional assertion for gating purposes
10 List<Issue> issues = report.getIssues();
11 Assert.assertEquals(0, issues.size());
12
13 driver.quit();
14}

You are now set to run the test and ensure the accessibility of your application! So, go ahead and run it via your IDE or any other tooling you use for Java development.

Additional Configuration Examples

Testing accessibility in a specific state of the application

In this example, we are going to test the accessibility of a page in a specific state. This test navigates to a page, opens a dropdown, and only then runs the accessibility engines to identify issues.

1import com.evinced.EvincedReporter;
2import com.evinced.EvincedWebDriver;
3import com.evinced.dto.results.Issue;
4import com.evinced.dto.results.Report;
5import org.junit.Assert;
6import org.junit.Test;
7import org.openqa.selenium.By;
8import org.openqa.selenium.WebElement;
9import org.openqa.selenium.chrome.ChromeDriver;
10
11import java.util.List;
12
13public class AccessibilityTests {
14
15 @Test
16 public void evReportWithAnOpenDropdownTest() {
17 EvincedWebDriver driver = new EvincedWebDriver(new ChromeDriver());
18 driver.get("https://demo.evinced.com/");
19
20 // interacting with the page - opening all dropdown
21 WebElement firstDropdown = driver.findElement(By.cssSelector("div.filter-container > div:nth-child(1) > div > div.dropdown.line"));
22 firstDropdown.click();
23
24 Report report = driver.evReport();
25 List<Issue> issues = report.getIssues();
26 Assert.assertEquals(8, issues.size());
27
28 // write the report to HTML format to a file named: "test-results.html"
29 EvincedReporter.writeEvResultsToFile("test-results", report, EvincedReporter.FileFormat.HTML);
30 }
31}

Running on multiple tabs

Evinced supports performing accessibility tests that involve multiple tabs.

Running evReport on multiple tabs

The evReport command will works as expected on the primary tab. When testing on an additional browser tab, simply switch the driver from the primary Selenium “window handle” to the “window handle” of the new tab and then run evReport. This code is shown on lines 14-15 of the example below.

Example In the example below, page1.html has a link that opens in a new tab.

1// navigate to page1
2driver.get("page1.html");
3
4// get report for page1
5Report report = driver.evReport();
6List<Issue> issues = report.getIssues();
7// run some assertions
8assertEquals(1, issues.size());
9
10// perform an action that opens a new tab
11driver.findElement(new By.ById("open-tab-link")).click();
12
13// switch to new tab
14String tabWindowHandle = driver.getWindowHandles().toArray()[1].toString();
15driver.switchTo().window(tabWindowHandle);
16
17// get report for the new tab,
18// the issues for page1 won't appear here, only ones found on the new tab
19report = driver.evReport();
20issues = report.getIssues();
21assertEquals(3, issues.size());

Running evStart and evStop on multiple tabs

The evStart and evStop method scopes are limited to a single tab or Selenium “window handle”. This means that if you have more than one tab, you will need to call evStop and then switch to the window handle of the new tab. Once the “window handle” switch has occurred, run a new instance of evStart followed evStop at the appropriate time.evStop must be called on the tab that evStart command was called on, before starting evStart on a new tab.

In the example below, page1.html has a link that opens in a new tab.

1 // navigate to page1
2 driver.get("page1.html");
3
4 driver.evStart();
5
6 // perform actions on page1, clicks, etc...
7
8 // the evStart and evStop scopes are limited to a single window handle.
9 // this means that if you have more than one tab, you need to switch it's window and only then run `evStart` and `evStop`
10 // The `evStop` must be called on the tab that `evStart` command was called on, before starting `evStart` on a new tab
11 Report report = driver.evStop();
12 List<Issue> issues = report.getIssues();
13 assertEquals(1, issues.size());
14
15 // perform an action that opens a new tab
16 driver.findElement(new By.ById("open-tab-link")).click();
17
18 // move to tab
19 String tabWindowHandle = driver.getWindowHandles().toArray()[1].toString();
20 driver.switchTo().window(tabWindowHandle);
21
22 // get report for the new tab,
23 // the issues for page1 won't appear here, only ones found on the new tab
24 driver.evStart();
25 report = driver.evStop();
26 issues = report.getIssues();
27 assertEquals(3, issues.size());

Running evStart and evStop on a different TargetLocator (frame / iframe)

In case the evStart command is called on a different context, for example - after switching WebDriver’s context into an iframe, the Evinced engines will be injected into that iframe only. In order to get the results back you will need to switch to the same context evStart was called on, before calling evStop. In case you are using a website that relies deeply on iframes, we suggest using the IncludeIframes API configuration.

In the example below, page1.html has an iframe in it, with id iframe1.

1 // navigate to page1
2 driver.get("page1.html");
3
4 // switch to the iframe
5 driver.switchTo().frame(driver.findElement(By.id("iframe1")));
6
7 // start recording
8 driver.evStart();
9
10 // perform actions on iframe1 inside page1, clicks, etc...
11
12 // if you go back to the defaultContent (the iframe parent)
13 // the `evStop` method will fail
14 driver.switchTo().defaultContent();
15
16 // don't do that, it will fail
17 driver.evStop();
18
19 // but you can get the results when inside the iframe
20 // switch to the iframe
21 driver.switchTo().frame(driver.findElement(By.id("iframe1")));
22
23 // now evStop returns all the issues recorded inside the iframe
24 Report report = driver.evStop();
25 List<Issue> issues = report.getIssues();
26 assertEquals(1, issues.size());

Add accessibility testing to existing tests

It is possible to configure EvincedWebDriver to be called also used in @Before and @After methods. In the example below, evStart starts recording before the test starts, and evStop is called after each test.

1import com.evinced.dto.results.Report;
2import io.github.bonigarcia.wdm.WebDriverManager;
3import org.junit.*;
4import org.junit.rules.TestName;
5import org.openqa.selenium.chrome.ChromeDriver;
6
7public class BeforeAfterTest {
8
9 @Rule
10 public TestName testName = new TestName();
11
12 private EvincedWebDriver driver;
13
14 /**
15 * Instantiate the WebDriver
16 */
17 @BeforeClass
18 public void setupClass() {
19 WebDriverManager.chromedriver().setup();
20 driver = new EvincedWebDriver(new ChromeDriver());
21 driver.manage().window().maximize();
22 }
23
24 /**
25 * Start recording before each test
26 */
27 @Before
28 public void setUp() {
29 driver.evStart();
30 }
31
32 /**
33 * Ensure we close the WebDriver after finishing
34 */
35 @After
36 public void after() {
37 // write an HTML report to file
38 Report report = driver.evStop();
39 EvincedReporter.writeEvResultsToFile(testName.getMethodName(), report, EvincedReporter.FileFormat.HTML);
40 }
41
42 @Test
43 public void visitDemoPage() {
44 driver.get("https://demo.evinced.com");
45 }
46
47 @Test
48 public void visitGoogle() {
49 driver.get("https://www.google.com");
50 }
51}

Support

Please feel free to reach out to support@evinced.com with any questions.

FAQ

  1. Do you have support for languages other than Java?
    Yes! We have SDK versions for JavaScript as well as Cypress and WebdriverIO. We have implementations for Ruby, Python, and C# coming soon.
  2. Can I configure which validations to run?
    Yes, see the API section for details on how to configure validations to your needs.
  3. Can I run tests with Evinced using cloud-based services like Sauce Labs, Perfecto, or BrowserStack?
    Yes, we have tested the Evinced SDK on many of these types of cloud-based services and expect no issues.