Appium Java SDK

The Evinced Appium Java SDK integrates with new or existing Appium tests to automatically detect accessibility issues. With the addition of as few as 5 lines of code to your Appium framework, you can begin to analyze your entire application to understand how it can become more accessible. At the conclusion of the test, a rich and comprehensive HTML or JSON report is generated to track issues in any reporting tool.

Supported versions / frameworks

Evinced Appium SDK supports the following values for Appium’s automationName desired capability:

• XCUI SDK
• UIAutomator2
• Espresso

Older versions of automation drivers (e.g deprecated Appium iOS driver based on UIAutomation) are not supported.

Prerequisites:

• Java 8 or higher
• Appium Server ver. 1.21 or higher
• Appium Java client v7 or higher (older version may still be compatible)

Get Started

Setup

In order to use any of the Evinced Mobile SDKs you first need to create an Evinced account. Once your account is created, you will get your apiKey and a matching serviceAccountId. Pass these these tokens using the setupCredentials() method and Evinced will validate access upon test execution. If an outbound internet connection is not available in your running environment - contact us at support@evinced.com to get an offline APIKey.

Install Evinced Appium SDK from the remote registry

If you are using Gradle as your build tool follow the steps below:

1. Add Artifactory plugin to the plugins section of your build.gradle:

   1    plugins {
   2        // ... other plugins
   3        id 'com.jfrog.artifactory' version '4.24.14'
   4    }

2. To the bottom of your build.gradle file add the next configuration for Artifactory plugin:

   1artifactory {
   2    contextUrl = 'https://evinced.jfrog.io/artifactory/'
   3
   4    resolve {
   5        repository {
   6            repoKey = 'public-gradle'
   7            maven = true
   8        }
   9    }
   10}

If you are using Maven, add the Evinced package repository URL to your build file:

1<repositories>
2    <repository>
3        <id>evinced-repository</id>
4        <url>https://evinced.jfrog.io/artifactory/public-gradle</url>
5    </repository>
6</repositories>

NOTE: If you see an error com.github.appium:java-client:jar:927db52 was not found please also at the additional Jitpack repository.

1 <repositories>
2    <repository>
3        <id>jitpack</id>
4        <url>https://jitpack.io</url>
5    </repository>
6</repositories>

Adding Evinced Appium SDK as a project dependency

The SDK can be added to your project as a Maven or Gradle dependency.

When working with Maven add the following dependency to your pom.xml file:

1<dependency>
2    <groupId>com.evinced</groupId>
3    <artifactId>appium-sdk</artifactId>
4    <version>{sdk-version}</version>
5    <scope>test</scope>
6</dependency>

When working with Gradle add the following to your build.gradle dependencies section:

1dependencies {
2    // ... other project dependencies
3    testImplementation(group: 'com.evinced', name: 'appium-sdk', version: '{sdk-version}', classifier: 'all')
4}

Important notice: In case of usage of Appium Java Client 8 the {sdk-version} should include the "_java_client_v8" identifier string. The following example will support Appium Java Client 8:

1<dependency>
2    <groupId>com.evinced</groupId>
3    <artifactId>appium-sdk</artifactId>
4    <version>1.10.0_java_client_v8</version>
5    <scope>test</scope>
6</dependency>

The following example without the "_java_client_v8" identifier string will support Appium Java Client 7 and lower by default:

1<dependency>
2    <groupId>com.evinced</groupId>
3    <artifactId>appium-sdk</artifactId>
4    <version>1.10.0</version>
5    <scope>test</scope>
6</dependency>

Replace {sdk-version} with the latest SDK version that can be found here: appium sdk jfrog.

Install Evinced Appium SDK from local ZIP archive

You can also perform installation right from the standalone .jar distribution. In this case, you need to do the following:

1. Download the .jar file.
2. Unpack the provided evinced-appium-sdk.zip to any desirable location
3. Add the following dependencies entries pointing to appium-sdk-all.jar
   1. Gradle:

      1implementation files('/Users/<path-to-your-upacked-folder>/appium-sdk-all.jar')

   2. Maven:
      1. First, install the “all“ jar into your local Maven repository by the following command:

         1mvn install:install-file –Dfile=/Users/<path-to-unpacked-folder>/appium-sdk-all.jar -DgroupId=com.evinced -DartifactId=evinced-appium-sdk -Dversion={sdk-version}

      2. Add the corresponding dependency into your pom.xml:

         1 <dependency>
         2     <groupId>com.evinced</groupId>
         3     <artifactId>evinced-appium-sdk</artifactId>
         4     <version>{sdk-version}</version>
         5 </dependency>

   3. Trigger your package manager download new dependencies either via IDE or command line

ZIP archive also contains two complimentary .jar files which you can utilize:

• appium-sdk-javadoc.jar contains rich JavaDoc documentation which you can view in any web browser
• appium-sdk-sources.jar can be attached to your Evinced Appium SDK installation in order to let you leverage JavaDoc right in your IDE. Please, refer to the corresponding documentation of your developer tools

Your first test

Initialize the SDK object

Add the import statement to the beginning of your test file.

1import com.evinced.appium.sdk.core.EvincedAppiumSdk;

Under the hood, the Evinced SDK utilizes several Appium commands in order to get accessibility data from the device under test. For this reason, we need to initialize EvincedAppiumSdk with the Appium driver instance in the following way:

1public static EvincedAppiumSdk evincedSdk;
2
3@BeforeClass
4public static void setupAppiumDriver() throws MalformedURLException, IOException {
5    capabilities = new DesiredCapabilities();
6    // ...omitting the setting of capabilities for brevity
7
8    driver = new IOSDriver<IOSElement>(url, capabilities);
9
10    evincedSdk = new EvincedAppiumSdk(driver);
11}

Here we used the JUnit BeforeClass annotation to create a method that will be executed at the beginning of the test suite. This is often where the Appium Driver initialization takes place, therefore we can also use it to initialize the EvincedAppiumSdk instance as well.

Provide your license information

The Evinced Appium SDK requires authentication credentials which consist of two string values mentioned above: apiKey and serviceAccountId.

For this purpose use the method called evincedAppiumSdk.setupCredentials(serviceAccountId, apiKey) as shown:

1public static EvincedAppiumSdk evincedSdk;
2
3@BeforeClass
4public static void setupAppiumDriver() throws MalformedURLException, IOException {
5    //... the rest of our setup code
6
7    evincedSdk = new EvincedAppiumSdk(driver);
8
9    evincedSdk.setupCredentials(<your service account ID>, <your API key>);
10}

You can confirm the success of the authentication in the logs.

Add Evinced accessibility checks

1@Test
2public void testMyAppScreen() {
3    IOSElement someTableCell = (IOSElement) new WebDriverWait(driver, 30)
4            .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("TableViewCell")));
5
6    firstStationTableCell.click();
7    //... do other interactions with the app
8
9    // Run analysis and get the accessibility report
10    Report report = evincedSdk.report();
11
12    // Assert that there are no accessibility issues
13    assertTrue(report.getIssues().size() == 0);
14}

Regardless of the test result, the report method will always output JSON and HTML files. You should be able to find the files by browsing the local folder called evinced-reports . For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

Continuous mode

Continuous mode allows continual scanning of the application without the need to insert individual scan calls with in the test code. Simply specify the start and end points using the following methods evincedEngine.startAnalyze() and evincedEngine.stopAnalyze(). Then Evinced engines will automatically scan the application after each Appium command. All issues detected during the tests will be added to the HTML and JSON reports automatically generated with the stopAnalyze() method is called.

To use continuous mode, you need to replace the standard Appium driver with the EvincedAppiumAndroidDriver for Android or EvincedAppiumIOSDriver. No changes are needed to your desired capabilities or other driver setup.

1@BeforeClass
2public static void setupAppiumDriver() throws IOException {
3
4    // Usual Appium configuration
5
6    driver = new EvincedAppiumAndroidDriver<>(url, capabilities);
7    evincedAppiumSdk = new EvincedAppiumSdk(driver);
8    evincedAppiumSdk.setupCredentials(TestBuildConfig.EVINCED_SERVICE_ACCOUNT_ID, TestBuildConfig.EVINCED_API_KEY);
9}

To initialize the Evinced engines simply use the evincedAppiumSdk.startAnalyze() method and finish it use evincedAppiumSdk.stopAnalyze(). You can use it within a single test method or scan an entire class (or more) using before and after hooks.

Here is an example of using continuous mode in a single test case:

1@Test
2public void testContinuousMode() {
3    evincedAppiumSdk.startAnalyze();
4
5    // appium commands
6    // ...
7
8    evincedAppiumSdk.stopAnalyze();
9}

Here is an example of using continuous mode in for an entire class:

1@BeforeClass
2public static void setup() {
3    // ...
4    evincedAppiumSdk.startAnalyze();
5}
6
7@AfterClass
8public static void tearDown() {
9    // ...
10    evincedAppiumSdk.stopAnalyze();
11}

Optional/Advanced

If you need to use your own AppiumDriver class instead of EvincedAppiumAndroidDriver or EvincedAppiumIOSDriver as base classes, you can implement CanSetOnExecuteListener interface. It should have getOnExecuteListener and setOnExecuteListener methods and trigger onExecuteListener.executed on every execution of appium commands.

So it can be implemented the next way:

1// For iOS it can be IOSDriver instead of AndroidDriver
2public class YourOwnCustomDriver<T extends WebElement> extends AndroidDriver<T> implements CanSetOnExecuteListener {  
3  
4    private OnExecuteListener onExecuteListener = null;  
5  
6    private boolean busy = false;  
7  
8//  
9//    Your overrided constructors  
10//    ...  
11//  
12  
13    @Override  
14    public OnExecuteListener getOnExecuteListener() {  
15        return onExecuteListener;  
16    }  
17  
18    @Override  
19    public void setOnExecuteListener(OnExecuteListener onExecuteListener) {  
20        this.onExecuteListener = onExecuteListener;  
21    }  
22  
23    @Override  
24    public Response execute(String command) {  
25        return execute(command, ImmutableMap.of());  
26    }  
27  
28    @Override  
29    public Response execute(String driverCommand, Map<String, ?> parameters) {  
30        if (onExecuteListener != null && !busy) {  
31            busy = true;  
32            onExecuteListener.executed(driverCommand, parameters);  
33            busy = false;  
34        }  
35        return super.execute(driverCommand, parameters);  
36    }
37}

So you can use this driver with Evinced SDK as usual.

1    YourOwnCustomDriver<AndroidElement> driver = new YourOwnCustomDriver<>(url, capabilities);
2    EvincedAppiumSdk evincedAppiumSdk = new EvincedAppiumSdk(driver);

Issue filtering

The Evinced Appium SDK supports advanced settings that allow you to apply additional configurations when generating issue reports. Simply create an instance of the EvincedConfig object with the appropriate options.

Below is an example of using EvincedConfig and IssueFilter:

Configure filters:

1final IssueFilter severityFilter = new IssueFilter()
2                .severity(Severity.NeedsReview);
3final IssueFilter issueFilter = new IssueFilter()
4                .issueType(IssueType.TappableArea, IssueType.ColorContrast);
5
6final MobileElement element = driver.findElementById(viewId);
7final IssueFilter elementFilter = new IssueFilter().matchByElements(element1);

Global filtering:

1// Create an example filter to exclude all Needs Review issues globally
2final IssueFilter globalFilter = new IssueFilter()
3                .severity(Severity.NeedsReview);
4
5// Add the IssueFilter to the creation instance of EvincedAppiumSdk
6final EvincedConfig globalEvincedConfig = new EvincedConfig()
7                .excludeFilters(globalFilter);
8final EvincedAppiumSdk evincedAppiumSdk = new EvincedAppiumSdk(driver, new InitOptions(globalEvincedConfig));

Filtering on an individual scan (analyze() or report() call):

1// Passing multiple exclude filters for an individual scan
2final IssueFilter multipleSeverityFilter = new IssueFilter()
3                .severity(Severity.NeedsReview, Severity.BestPractice);
4
5 final IssueFilter complexFilter = new IssueFilter()
6                .matchElementById("com.example:id/myElement")
7                .issueType(IssueType.CollidingControls)
8                .severity(Severity.NeedsReview);
9
10final IssueFilter complexFilter1 = new IssueFilter()
11                .matchElementByAccessibilityId("myElementA11yID")
12                .issueType(IssueType.DuplicateName, SpecialCharacters)
13                .severity(Severity.NeedsReview, BestPractice);
14final EvincedConfig evincedConfig = new EvincedConfig()
15                .excludeFilters(multipleSeverityFilter, multipleSeverityFilter, complexFilter);
16
17// Apply the configuration for a specific screen state scan
18evincedAppiumSdk.report(false, evincedConfig);
19evincedAppiumSdk.analyze(evincedConfig);

Depending on the purpose of filtering, you can use two directions -include or exclude using the appropriate methods:

• EvincedConfig().includeFilters(filter)
• EvincedConfig().excludeFilters(filters)

List of available issue filtering options

Config file

Along with the configuration creation options within the code, it is also possible to create filters using a JSON file. To import the file use the examples below:

1 // create EvincedConfig instance from file by specify absolute or relative file path
2 final EvincedConfig evincedConfigFromFile = EvincedConfig.fromFile("/project_root_evinced_config.json");
3
4 // create EvincedConfig instance from file by specify file name, located in resources directory
5 final EvincedConfig evincedConfigFromResources = EvincedConfig.fromResourcesFile("resources_evinced_config.json");

Example File

1{
2    "excludeFilters": [
3        {
4            "viewId": "viewId",
5            "accessibilityId": "a11yId",
6            "severities": [
7                {
8                    // you can use id or name separately
9                    "id": "3c68eb41-9227-481d-8eec-9dc8956b1900",
10                    "name": "Needs Review"
11                }
12            ],
13            "issueTypes": [
14                {
15                    // you can use id or name separately
16                    "id": "e871b2e7-1b6d-47bc-96db-7f3dff4bfd5a",
17                    "name": "Duplicate Name"
18                }
19            ]
20        }
21    ]
22}

Storing screenshots in reports

By default screenshots are provided in HTML reports for each scan, but for JSON-reports they are disabled by default. You can explicitly enable or disable screenshots using the InitOptions. Screenshots in JSON reports are provided in Base64 format.

Available screenshot options:

Global screenshot config:

1final InitOptions initOptions = new InitOptions(InitOptions.ScreenshotOption.Base64);
2// setup options during the SDK initialization
3final EvincedAppiumSdk evincedApiumSdk = new EvincedAppiumSdk(driver, initOptions);
4
5// change the existing options
6final InitOptions newOptions = new InitOptions(InitOptions.ScreenshotOption.Disabled);
7evincedApiumSdk.setOptions(newOptions);

Rule validation configurations

RulesConfig allows you to change the accessibility check for different rules. It is set globally and used for all subsequent reports.

Disabling the tappable area validation

This example disables validation for tappable area:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.TAPPABLE_AREA, false))

Disabling the label capitalization validation

This example disables validation for label capitalization:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.LABEL_CAPITALIZATION, false))

Disabling the color contrast validation

This example disables validation for color contrast:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.COLOR_CONTRAST, false)))

Customize color contrast options

Fuzziness factor configuration for color contrast. For example, the WCAG norm for this ratio is 4.5:1, but if your corporate colors have a ratio of 4.45:1, then this can be specified using the following options:

1final Map<String, Object> colorContrastOptions = new HashMap<>();
2colorContrastOptions.put("contrastRatioNormalText", 4.45f);
3colorContrastOptions.put("contrastRatioLargeText", 4.45f);
4colorContrastOptions.put("contrastRatioNonText", 4.45f);
5final RuleConfig colorContrastRuleConfig = new RuleConfig(IssueType.COLOR_CONTRAST, true, colorContrastOptions);
6evincedEngine.setOptions(new InitOptions().setRulesConfig(colorContrastRuleConfig));

Exporting meaningful labels

To export a JSON report of the accessibility label of images and buttons regardless of the scan results simply add the options flag:

1final InitOptions initOptions = new InitOptions();
2initOptions.putAdditionalOption("exportMeaningfulLabels", true);
3evincedEngine.setOptions(initOptions);

You should see a new JSON file created beside other report files with a default name: Evinced_Meaningful_Labels_Test_Results_11_15_2023_09_49_51.json. The structure of this JSON file is described in our Mobile Reports page.

To change the default folder destination and the file name of the meaningful labels JSON report you can use the second additional option with an absolute folder path:

1final InitOptions initOptions = new InitOptions();
2final File meaningfulLabelsFile = new File("evinced-reports", "SpecificMeaningfulLabels.json");
3initOptions.putAdditionalOption("exportMeaningfulLabels", true);
4initOptions.putAdditionalOption("exportMeaningfulLabelsPath", meaningfulLabelsFile.getAbsolutePath());
5evincedEngine.setOptions(initOptions);

Tappable Area

Every native mobile application features a wide variety of UI elements which are interactive via tap. Consider elements like buttons, text fields, toggles and others. Each of them should be tapped in order to become focused and to allow any further interaction. The lower the tappable area of the element (an imaginary square where user should tap) the less convenient it is to operate with the element. Platform constraints describe the width and height of minimum tappable area.

tappableAreaDimensions - the minimum size that does not trigger Tappable Area validation. By default for Android this value is 48dp, for iOS it is 44px. tappableAreaDimensionsOnEdge - the same, only it applies to elements on the edge of the screen and only to Android. By default for Android this value is 32dp.

For example

1final Map<String, Object> tapAreaDimension = new HashMap<>();
2tapAreaDimension.put("tappableAreaDimensions", 70);
3tapAreaDimension.put("tappableAreaDimensionsOnEdge", 70);
4RuleConfig tapArea = new RuleConfig(IssueType.TappableArea, true, tapAreaDimension);
5InitOptions initOptions = new InitOptions().setRulesConfig(tapArea);
6evincedAppiumSdk.setOptions(initOptions);

API

EvincedAppiumSdk

public EvincedAppiumSdk(AppiumDriver<?> originalDriver)

Default constructor which accepts an AppiumDriver instance for performing accessibility snapshot collection. Can be easily instantiated in the following way:

1IOSDriver driver = new IOSDriver<IOSElement>(url, capabilities);
2EvincedAppiumSdk evincedSdk = new EvincedAppiumSdk(driver);

public EvincedAppiumSdk(AppiumDriver<?> originalDriver, EvincedAppiumSdkOptions options)

Additional constructor which allows you to configure the different aspects of a11y testing via EvincedAppiumSdkOptions object. For instance, one can set a custom path for storing reporting artifacts by using the example below:

1driver = new IOSDriver<IOSElement>(url, capabilities);
2
3EvincedAppiumSdkOptions evincedSdkOptions = new EvincedAppiumSdkOptions(); // create options object
4evincedSdkOptions.setOutputDir("my-output-dir"); // set custom output directory
5
6evincedSdk = new EvincedAppiumSdk(driver, evincedSdkOptions);

public boolean setupCredentials(String serviceAccountId, String apiKey)

A method to add credentials confirming the validity of the user’s license. It accepts the following arguments:

• serviceAccountId - Evinced service account ID
• apiKey - Evinced API key

Returns a boolean flag indicating the success of the operation - if it is false there is something wrong with either validity of the license or with the checking infrastructure. See logs for more detailed output.

public boolean setupOfflineCredentials(String serviceAccountId, String accesstoken)

A method to add credentials confirming the validity of the user’s license with an offline token. It accepts the following arguments:

• serviceAccountId - Evinced service account ID
• accesstoken - Evinced access token

Returns a boolean flag indicating the success of the operation - if it is false there is something wrong with either validity of the license or with the checking infrastructure. See logs for more detailed output.

public void startAnalyze(EvincedConfig evincedConfig)

Watch for screen state changes and record all accessibility issues until stopAnalyze() is called. It works the same way as EvincedAppiumSdk#analyze() but collects snapshot on each appium command that can change application state.

Make sure that driver you set in constructor extends EvincedAppiumAndroidDriver or EvincedAppiumIOSDriver, or implements CanSetOnExecuteListener in similar way.

Available overloads: public void startAnalyze();

public List<Report> stopAnalyze(boolean shouldAssert) Stop flow analyze and returns a report with all issues found from the last startAnalyze() call to allow users to perform any advanced assertions on their own.

If it is not needed, simple assertion may be delegated directly to the SDK by supplying shouldAssert argument set to true. In this case, the SDK will automatically throw an AssertionError if there is at least one issue found.

Available overloads: public void stopAnalyze();

public Report report(shouldAssert: boolean)

Generates accessibility report for the current state of the application. Returns a Report instance to allow users to perform any advanced assertions on their own. If it is not needed, a simple assertion may be delegated directly to the SDK by supplying shouldAssert argument set to true. In this case, the SDK will automatically throw an AssertionError if there is at least one issue found. For more information regarding the Report object , please see our detailed Mobile Reports page.

This method always writes HTML and JSON files to the configured directory.

Available overloads:

public Report report();

Example usage:

1Report report = evincedSdk.report(true); // assert is performed by EvincedAppiumSdk
2
3Report reportForMyOwnAssertion = evincedSdk.report();
4assertEquals(reportForMyOwnAssertion.hasIssues(), false); // performing assertions on your own

public void analyze()

Collects an accessibility snapshot of the current application state and puts it into internal in-memory storage. This method is supposed to be used in conjunction with EvincedAppiumSdk.assertStored() for performing actual assertions against the content of this storage.

Example usage:

1  IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
2            .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
3    // we have achieved the state of the app we would like to test. Let's take a snapshot.
4   evincedAppiumSdk.analyze();

public List<Report> reportStored(shouldAssert: boolean)

Generates a list of accessibility reports, one for each accessibility snapshot collected by EvincedAppiumSdk.analyze(). Just like EvincedAppiumSdk.report() this method may either perform simple assertion on its own (throwing an exception if there is at least one report with issues) or allow users to define their own assertions on the returned Report objects. This behavior can be configured via the shouldAssert flag.

This method always writes HTML and JSON files to the configured directory. Both of the files contain combined information about all of the application states collected. For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

Available overloads:

public List<Report> reportStored();

Example usage:

1@AfterClass
2public static void tearDown() {
3    // Option 1
4    List<Report> reportsForOwnAssertion = evincedSdk.reportStored(); // making our own assertions
5    // Option 2
6    List<Report> reports = evincedSdk.reportStored(true); // delegating assertion to the SDK
7    // Option 3
8    List<Report> reportsWithIssues = reportsForOwnAssertion.stream() // Filter out only reports which contain some issues
9      .filter(report -> report.hasIssues())
10      .collect(Collectors.toList());
11    assertEquals(reportsWithIssues.size(), 0); // Making the actual assertion
12
13    driver.quit();
14}

public void clearStored()

Removes all of the accessibility snapshots previously collected by EvincedAppiumSdk.analyze() calls.

InitOptions

1public InitOptions()

Default public constructor with no arguments.

1public InitOptions(String outputDir)
2public InitOptions(String outputDir, EvincedConfig evincedConfig)

Overloaded constructors

1public InitOptions setOutputDir(String outputDir)

Method for defining a custom output directory where a user would like to store reporting artifacts (JSON and HTML files).

1public InitOptions setEvincedConfig(EvincedConfig evincedConfig)

Method for defining a EvincedConfig object as a global setting

1public InitOptions screenshotOption(ScreenshotOption screenshotOption)

Set option to save a screenshot in a output JSON report or as a separate .png file.

1public InitOptions setRuleConfigs(List<RuleConfig> ruleConfigs)

Set rule configs

1public InitOptions setReportName(String reportName)

Set the output HTML/JSON report name prefix

EvincedConfig

An object that defines the options that will be used when performing accessibility validations and generating reports.

Default public constructors for creating EvincedConfig:

1public EvincedConfig()

Set exclude filters:

1public EvincedConfig excludeFilters(IssueFilter... excludeFilters)
2public EvincedConfig excludeFilters(List<IssueFilter> excludeFilters)

Set include filters:

1public EvincedConfig includeFilters(IssueFilter... includeFilters)
2public EvincedConfig includeFilters(List<IssueFilter> includeFilters)

Enable storing screenshots in JSON-report:

ScreenshotOption

Enum that allows enable or disable saving screenshots in JSON-reports and can be Base64, File, Both or Disabled.

IssueFilter

A filter object that allows you to exclude UI elements from availability reports by certain criteria

IssueFilter public methodes:

1public IssueFilter matchElementById(String... viewIds)
2public IssueFilter matchElementByAccessibilityId(String... accessibilityIds)
3public IssueFilter matchByElements(MobileElement... mobileElement)
4public IssueFilter matchByElements(List<MobileElement> mobileElements)
5public IssueFilter severity(List<Severity> severities)
6public IssueFilter severity(Severity... severity)
7public IssueFilter issueType(List<IssueType> issueTypes)
8public IssueFilter issueType(IssueType... issueType)

RuleConfig

Default constructor:

1public RuleConfig(IssueType name, boolean enabled)

IssueType - the IssueType on which this rule config should be applied enabled - allows to enable/disable specified IssueType. By default enabled = true

Tutorials

Generating a comprehensive a11y report for your application

In this tutorial, we will enhance our existing Appium UI test with the Evinced Appium SDK in order to check our application for accessibility issues. In order to get started you will need the following:

1. All of the prerequisites for the Evinced Appium SDK should be met
2. Evinced Appium SDK should be installed in your project

Preface - existing UI test overview

Let’s consider the following basic UI test as our starting point.

1import io.appium.java_client.MobileBy;
2import io.appium.java_client.ios.IOSDriver;
3import io.appium.java_client.ios.IOSElement;
4import io.appium.java_client.remote.MobileCapabilityType;
5
6import java.io.File;
7import java.io.IOException;
8import java.net.MalformedURLException;
9import java.net.URL;
10import org.openqa.selenium.remote.DesiredCapabilities;
11import org.junit.AfterClass;
12import org.junit.BeforeClass;
13import org.junit.Test;
14import org.openqa.selenium.support.ui.ExpectedConditions;
15import org.openqa.selenium.support.ui.WebDriverWait;
16
17public class SwiftRadioTest {
18
19    public static URL url;
20    public static DesiredCapabilities capabilities;
21    public static IOSDriver<IOSElement> driver;
22
23    @BeforeClass
24    public static void setupAppiumDriver() throws MalformedURLException, IOException {
25        final String URL_STRING = "http://127.0.0.1:4723/wd/hub";
26        url = new URL(URL_STRING);
27
28        final Class clazz = EvincedA11yValidatorTest.class;
29        final File file = new File(clazz.getResource("/SwiftRadio.app").getFile());
30
31        capabilities = new DesiredCapabilities();
32        capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "XCUITest");
33        capabilities.setCapability("useNewWDA", false);
34        capabilities.setCapability(MobileCapabilityType.NO_RESET, true);
35        capabilities.setCapability(MobileCapabilityType.DEVICE_NAME, "iPhone Simulator");
36        capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "iOS");
37        capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION, "12.0");
38        capabilities
39                .setCapability(MobileCapabilityType.APP,
40                        file.getAbsolutePath());
41
42        driver = new IOSDriver<IOSElement>(url, capabilities);
43    }
44
45    @AfterClass
46    public static void tearDown() {
47        driver.quit();
48    }
49
50    @Test public void testStationsScreen() {
51        IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
52                .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
53
54        firstStationTableCell.click();
55
56        IOSElement playButton = (IOSElement) new WebDriverWait(driver, 30)
57                .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("XCUIElementTypeButton")));
58    }
59}

We wrote this test for an application called Swift Radio. It is a small open-source app for listening to various radio stations and podcasts.

The purpose of these tests is to check the functionality of the main application screen and ensure a user can successfully navigate to their choice of the radio station. For now, this test is only concerned with the functional testing of the app. However, with the help of the Evinced Appium SDK, we can also check it for accessibility issues along the way. Let’s go through this process with the following step-by-step instructions.

Step #1 - Initialize the Evinced Appium SDK

Before making any assertions against our app, we need to initialize EvincedAppiumSdk object. This object is used primarily as an entry point to all of the accessibility scanning features. Since we are going to use it primarily within our tests, the best place for its initialization will be our setupAppiumDriver method which gets executed first.

1import io.appium.java_client.MobileBy;
2import io.appium.java_client.ios.IOSDriver;
3import io.appium.java_client.ios.IOSElement;
4import io.appium.java_client.remote.MobileCapabilityType;
5
6import java.io.File;
7import java.io.IOException;
8import java.net.MalformedURLException;
9import java.net.URL;
10import org.openqa.selenium.remote.DesiredCapabilities;
11import org.junit.AfterClass;
12import org.junit.BeforeClass;
13import org.junit.Test;
14import org.openqa.selenium.support.ui.ExpectedConditions;
15import org.openqa.selenium.support.ui.WebDriverWait;
16
17public class SwiftRadioTest {
18
19    public static URL url;
20    public static DesiredCapabilities capabilities;
21    public static IOSDriver<IOSElement> driver;
22
23    public static EvincedAppiumSdk evincedAppiumSdk;
24
25    @BeforeClass
26    public static void setupAppiumDriver() throws MalformedURLException, IOException {
27        final String URL_STRING = "http://127.0.0.1:4723/wd/hub";
28        url = new URL(URL_STRING);
29
30        final Class clazz = EvincedA11yValidatorTest.class;
31        final File file = new File(clazz.getResource("/SwiftRadio.app").getFile());
32
33        capabilities = new DesiredCapabilities();
34        capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "XCUITest");
35        capabilities.setCapability("useNewWDA", false);
36        capabilities.setCapability(MobileCapabilityType.NO_RESET, true);
37        capabilities.setCapability(MobileCapabilityType.DEVICE_NAME, "iPhone Simulator");
38        capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "iOS");
39        capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION, "12.0");
40        capabilities
41                .setCapability(MobileCapabilityType.APP,
42                        file.getAbsolutePath());
43
44        driver = new IOSDriver<IOSElement>(url, capabilities);
45        evincedAppiumSdk = new EvincedAppiumSdk(driver);
46        evincedAppiumSdk.setupCredentials(<your service account ID>, <your API key>);
47    }
48
49    @AfterClass
50    public static void tearDown() {
51        driver.quit();
52    }
53
54    @Test
55    public void testStationsScreen() {
56        IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
57                .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
58
59        firstStationTableCell.click();
60
61        IOSElement playButton = (IOSElement) new WebDriverWait(driver, 30)
62                .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("XCUIElementTypeButton")));
63    }
64}

The EvincedAppiumSdk requires only an instance of the Appium driver. There are additional configuration options that can be included as well. More information can be found in the API section of this documentation.

Step #2 - Identify which application states you want to check for a11y issues

Once the SDK is set up, let’s try to carefully consider which states of our application are actually worth checking for accessibility issues. Let’s take a look at out UI test once again:

1@Test
2public void testStationsScreen() {
3    IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
4            .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
5    // We have reached the main application screen with list of stations. It may have some accessibility issues!
6
7    firstStationTableCell.click();
8
9    IOSElement playButton = (IOSElement) new WebDriverWait(driver, 30)
10            .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("XCUIElementTypeButton")));
11    // Here we have reached yet another screen - a concrete station view. It is probably worth checking for accessibility issues as well.
12}

So far we have identified two places in our UI test where we may need to perform accessibility assertion. Each place represents some distinct application state and each of these states may contain certain accessibility issues. So, let’s put our checks into place.

Step #3 - Setup analyzing breakpoints

The only thing we need to do now is to simply add evincedAppiumSdk.analyze() calls in places within our test which we previously identified as “interesting” ones in terms of accessibility. Here is how it should look:

1@Test
2public void testStationsScreen() {
3    IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
4            .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
5    // We have reached the main application screen with list of stations. It may have some accessibility issues!
6    evincedAppiumSdk.analyze();
7
8    firstStationTableCell.click();
9
10    IOSElement playButton = (IOSElement) new WebDriverWait(driver, 30)
11            .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("XCUIElementTypeButton")));
12    // Here we have reached yet another screen - a concrete station view. It is probably worth checking for accessibility issues as well.
13    evincedAppiumSdk.analyze();
14}

The analyze method collects an accessibility snapshot of the current application state and puts it into internal in-memory storage. This snapshot contains a lot of information including application screenshot, accessibility labels and dimensions of UI elements.

Now that we have gathered the information we need we are finally ready to actually find our whether our app is accessible or not.

Step #4 - Assert your screens at the end of your test suite

As our test was executed we collected a lot of accessibility snapshots via the evincedAppiumSdk.analyze() calls. We can now perform accessibility assertions at the end of our test suite. Referring back again to our initial UI test the best place for this assertion will be the method that gets invoked last - tearDown.

1@AfterClass
2public static void tearDown() {
3    List<Report> reports = evincedAppiumSdk.reportStored();
4    driver.quit();
5}

To generate the actual object representation of your accessibility report simply call the reportStored method. This way Evinced Appium SDK will check all of the previously collected accessibility snapshots for having some accessibility issues. From this moment you are free to assert the given list of Report objects in any way you want (feel free to explore the corresponding chapter in the “API“ section).

For the sake of simplicity of this tutorial let’s simply assume that our application is accessible as long as it has no accessibility issues found. Thus, if we have at least one accessibility issue detected - we want our tests to be failed. Let’s add the corresponding assertion:

1@AfterClass
2public static void tearDown() {
3   List<Report> reports = evincedAppiumSdk.reportStored();
4   List<Report> reportsWithIssues = reports.stream() // Filter out only reports which contain some issues
5      .filter(report -> report.hasIssues())
6      .collect(Collectors.toList());
7
8   assertEquals(reportsWithIssues.size(), 0); // Making the actual assertion
9   driver.quit();
10}

You are now set to run the test and ensure the accessibility of your application! So, go ahead and run it via your IDE or any other tooling you use for Java development.

Step #5 - Read the results

At this point, you will likely see that your test has failed, which is totally fine. Swift Radio certainly has some accessibility flaws, and the fact that we have caught them simply means that Evinced Appium SDK is working!

The only thing which is missing here is the actual list of accessibility issues with would like to look at. First of all, let’s pay a bit of attention to the terminal output of our tests. You should see your tests failed due to AssertionError with the following description:

1Number of a11y issues found. Please refer to the html report located at: /Users/johndoe/dev/swift-radio/evinced-reports/evinced-report-a6a47bed-d7b5-467e-a5c2-9e6ac4c1f247.html
2java.lang.AssertionError: Number of a11y issues found. Please refer to the html report located at: /Users/johndoe/dev/swift-radio/evinced-reports/evinced-report-a6a47bed-d7b5-467e-a5c2-9e6ac4c1f247.html
3    at com.evinced.a11y.validator.appium.java.core.EvincedA11yValidator.assertTreesHaveA11yIssues(EvincedA11yValidator.java:72)
4    //... omited for brevity

According to this output, we have HTML and JSON files located right in the root of our project, inside the folder named evinced-reports. For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

Step #6 - Make your app more accessible

Congratulations! Over the course of this 5-step tutorial we have managed to integrate accessibility tests right into our existing UI tests with the help of Evinced Appium SDK and moreover, we have even found several issues in the app.

This is the perfect time for us to carefully investigate all of the issues detected (with help of the knowledge base) and put our JSON files into static storage for later use. With the history of all of the results, we can achieve a lot of exciting things including tracking our progress fixing all of the accessibility issues over time, detecting regressions, and gathering additional metrics.

Feel free to continue your journey with the Evinced Appium SDK by browsing the API section.

FAQ

1. Do you have support for languages other than Java?

Not at this moment, unfortunately. We are planning to expand our tooling coverage by writing dedicated libraries for other popular UI testing languages such as JavaScript and Python.

2. Can I configure which validations to run?

Currently, all of the available validations are run. We have not seen any drastic impact on performance caused by any particular validation. In the future, we are planning to introduce more sophisticated capabilities for filtering results by different properties (e.g. severity or problem type) or even by using your own custom predicate.

3. Where can I find the release changelogs?

If you are using JetBrains IDE (Android Studio or IntelliJ IDEA): Open ‘Project' tab (⌘1) → ‘Project’ view → 'External Libraries’ section → Gradle: com.evinced.appium-sdk:<version> → CHANGELOG.md