Appium Java SDK

The Evinced Appium Java SDK integrates with new or existing Appium tests to automatically detect accessibility issues. With the addition of as few as 5 lines of code to your Appium framework, you can begin to analyze your entire application to understand how it can become more accessible. At the conclusion of the test, a rich and comprehensive HTML or JSON report is generated to track issues in any reporting tool.

Supported versions / frameworks

Evinced Appium SDK supports the following values for Appium’s automationName desired capability:

• XCUI SDK
• UIAutomator2
• Espresso

Older versions of automation drivers (e.g deprecated Appium iOS driver based on UIAutomation) are not supported.

Prerequisites:

• Java 8 or higher
• Appium Server ver. 1.21 or higher
• Appium Java client v7 or higher (older version may still be compatible)

Get Started

Setup

In order to use any of the Evinced Mobile SDKs you first need to create an Evinced account. Once your account is created, you will get your apiKey and a matching serviceAccountId. Pass these these tokens using the setupCredentials() method and Evinced will validate access upon test execution. If an outbound internet connection is not available in your running environment - contact us at support@evinced.com to get an offline APIKey.

Install Evinced Appium SDK from the remote registry

If you are using Gradle as your build tool follow the steps below:

1. Add Artifactory plugin to the plugins section of your build.gradle:

   1    plugins {
   2        // ... other plugins
   3        id 'com.jfrog.artifactory' version '4.24.14'
   4    }

   Or if you use kotlin to configurate build.gradle.kts

   1    plugins {
   2        // ... other plugins
   3        id("com.jfrog.artifactory") version "4.24.14"
   4    }

2. To the bottom of your build.gradle file add the next configuration for Artifactory plugin:

   1artifactory {
   2    contextUrl = 'https://evinced.jfrog.io/artifactory/'
   3
   4    resolve {
   5        repository {
   6            repoKey = 'public-gradle'
   7            maven = true
   8        }
   9    }
   10}

   For build.gradle.kts:

   1artifactory {
   2    setContextUrl("https://evinced.jfrog.io/artifactory")
   3
   4    resolve {
   5        repository {
   6            setRepoKey("public-gradle")
   7            setMaven(true)
   8        }
   9    }
   10}

If you are using Maven, add the Evinced package repository URL to your build file:

1<repositories>
2    <repository>
3        <id>evinced-repository</id>
4        <url>https://evinced.jfrog.io/artifactory/public-gradle</url>
5    </repository>
6</repositories>

NOTE: If you see an error com.github.appium:java-client:jar:927db52 was not found please also at the additional Jitpack repository.

1 <repositories>
2    <repository>
3        <id>jitpack</id>
4        <url>https://jitpack.io</url>
5    </repository>
6</repositories>

Adding Evinced Appium SDK as a project dependency

The SDK can be added to your project as a Maven or Gradle dependency.

When working with Maven add the following dependency to your pom.xml file:

1<dependency>
2    <groupId>com.evinced</groupId>
3    <artifactId>appium-sdk</artifactId>
4    <version>{sdk-version}</version>
5    <scope>test</scope>
6</dependency>

When working with Gradle add the following to your build.gradle dependencies section:

1dependencies {
2    // ... other project dependencies
3    testImplementation(group: 'com.evinced', name: 'appium-sdk', version: '{sdk-version}', classifier: 'all')
4}

For build.gradle.kts:

1dependencies {
2    // ... other project dependencies
3    testImplementation("com.evinced:appium-sdk:{sdk-version}")
4}

Important notice: In case of usage of Appium Java Client 8 the {sdk-version} should include the "_java_client_v8" identifier string. The following example will support Appium Java Client 8:

1<dependency>
2    <groupId>com.evinced</groupId>
3    <artifactId>appium-sdk</artifactId>
4    <version>1.10.0_java_client_v8</version>
5    <scope>test</scope>
6</dependency>

The following example without the "_java_client_v8" identifier string will support Appium Java Client 7 and lower by default:

1<dependency>
2    <groupId>com.evinced</groupId>
3    <artifactId>appium-sdk</artifactId>
4    <version>1.10.0</version>
5    <scope>test</scope>
6</dependency>

Replace {sdk-version} with the latest SDK version that can be found here: appium sdk jfrog.

Install Evinced Appium SDK from local ZIP archive

You can also perform installation right from the standalone .jar distribution. In this case, you need to do the following:

1. Download the .jar file.
2. Unpack the provided evinced-appium-sdk.zip to any desirable location
3. Add the following dependencies entries pointing to appium-sdk-all.jar
   1. Gradle:

      1implementation files('/Users/<path-to-your-upacked-folder>/appium-sdk-all.jar')

   2. Maven:
      1. First, install the “all“ jar into your local Maven repository by the following command:

         1mvn install:install-file –Dfile=/Users/<path-to-unpacked-folder>/appium-sdk-all.jar -DgroupId=com.evinced -DartifactId=evinced-appium-sdk -Dversion={sdk-version}

      2. Add the corresponding dependency into your pom.xml:

         1 <dependency>
         2     <groupId>com.evinced</groupId>
         3     <artifactId>evinced-appium-sdk</artifactId>
         4     <version>{sdk-version}</version>
         5 </dependency>

   3. Trigger your package manager download new dependencies either via IDE or command line

ZIP archive also contains two complimentary .jar files which you can utilize:

• appium-sdk-javadoc.jar contains rich JavaDoc documentation which you can view in any web browser
• appium-sdk-sources.jar can be attached to your Evinced Appium SDK installation in order to let you leverage JavaDoc right in your IDE. Please, refer to the corresponding documentation of your developer tools

Your first test

Initialize the SDK object

Add the import statement to the beginning of your test file.

1import com.evinced.appium.sdk.core.EvincedAppiumSdk;

Under the hood, the Evinced SDK utilizes several Appium commands in order to get accessibility data from the device under test. For this reason, we need to initialize EvincedAppiumSdk with the Appium driver instance in the following way:

1public static EvincedAppiumSdk evincedSdk;
2
3@BeforeClass
4public static void setupAppiumDriver() throws MalformedURLException, IOException {
5    capabilities = new DesiredCapabilities();
6    // ...omitting the setting of capabilities for brevity
7
8    driver = new IOSDriver<IOSElement>(url, capabilities);
9
10    evincedSdk = new EvincedAppiumSdk(driver);
11}

Here we used the JUnit BeforeClass annotation to create a method that will be executed at the beginning of the test suite. This is often where the Appium Driver initialization takes place, therefore we can also use it to initialize the EvincedAppiumSdk instance as well.

Provide your license information

The Evinced Appium SDK requires authentication credentials which consist of two string values mentioned above: apiKey and serviceAccountId.

For this purpose use the method called evincedAppiumSdk.setupCredentials(serviceAccountId, apiKey) as shown:

1public static EvincedAppiumSdk evincedSdk;
2
3@BeforeClass
4public static void setupAppiumDriver() throws MalformedURLException, IOException {
5    //... the rest of our setup code
6
7    evincedSdk = new EvincedAppiumSdk(driver);
8
9    evincedSdk.setupCredentials(<your service account ID>, <your API key>);
10}

You can confirm the success of the authentication in the logs.

Add Evinced accessibility checks

1@Test
2public void testMyAppScreen() {
3    IOSElement someTableCell = (IOSElement) new WebDriverWait(driver, 30)
4            .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("TableViewCell")));
5
6    firstStationTableCell.click();
7    //... do other interactions with the app
8
9    // Run analysis and get the accessibility report
10    Report report = evincedSdk.report();
11
12    // Assert that there are no accessibility issues
13    assertTrue(report.getIssues().size() == 0);
14}

Regardless of the test result, the report method will always output JSON and HTML files. You should be able to find the files by browsing the local folder called evinced-reports . For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

If you only need Report objects during test runtime, and you don't need JSON and HTML report files saved, you can disable creating report files with disableAllReportsOutputFiles additional option.

1    final InitOptions initOptions = new InitOptions()
2            .putAdditionalOption("disableAllReportsOutputFiles", true);
3    evincedAppiumSdk.setOptions(initOptions);

Continuous mode

Continuous mode allows continual scanning of the application without the need to insert individual scan calls with in the test code. Simply specify the start and end points using the following methods evincedEngine.startAnalyze() and evincedEngine.stopAnalyze(). Then Evinced engines will automatically scan the application after each Appium command. All issues detected during the tests will be added to the HTML and JSON reports automatically generated with the stopAnalyze() method is called.

To use continuous mode, you need to replace the standard Appium driver with the EvincedAppiumAndroidDriver for Android or EvincedAppiumIOSDriver. No changes are needed to your desired capabilities or other driver setup.

1@BeforeClass
2public static void setupAppiumDriver() throws IOException {
3
4    // Usual Appium configuration
5
6    driver = new EvincedAppiumAndroidDriver<>(url, capabilities);
7    evincedAppiumSdk = new EvincedAppiumSdk(driver);
8    evincedAppiumSdk.setupCredentials(TestBuildConfig.EVINCED_SERVICE_ACCOUNT_ID, TestBuildConfig.EVINCED_API_KEY);
9}

To initialize the Evinced engines simply use the evincedAppiumSdk.startAnalyze() method and finish it use evincedAppiumSdk.stopAnalyze(). You can use it within a single test method or scan an entire class (or more) using before and after hooks.

Here is an example of using continuous mode in a single test case:

1@Test
2public void testContinuousMode() {
3    evincedAppiumSdk.startAnalyze();
4
5    // appium commands
6    // ...
7
8    evincedAppiumSdk.stopAnalyze();
9}

Here is an example of using continuous mode in for an entire class:

1@BeforeClass
2public static void setup() {
3    // ...
4    evincedAppiumSdk.startAnalyze();
5}
6
7@AfterClass
8public static void tearDown() {
9    // ...
10    evincedAppiumSdk.stopAnalyze();
11}

Optional/Advanced

If you need to use your own AppiumDriver class instead of EvincedAppiumAndroidDriver or EvincedAppiumIOSDriver as base classes, you can implement CanSetOnExecuteListener interface. It should have getOnExecuteListener and setOnExecuteListener methods and trigger onExecuteListener.executed on every execution of appium commands.

So it can be implemented the next way:

1// For iOS it can be IOSDriver instead of AndroidDriver
2public class YourOwnCustomDriver<T extends WebElement> extends AndroidDriver<T> implements CanSetOnExecuteListener {
3
4    private OnExecuteListener onExecuteListener = null;
5
6    private boolean busy = false;
7
8//
9//    Your overridden constructors
10//    ...
11//
12
13    @Override
14    public OnExecuteListener getOnExecuteListener() {
15        return onExecuteListener;
16    }
17
18    @Override
19    public void setOnExecuteListener(OnExecuteListener onExecuteListener) {
20        this.onExecuteListener = onExecuteListener;
21    }
22
23    @Override
24    public Response execute(String command) {
25        return execute(command, ImmutableMap.of());
26    }
27
28    @Override
29    public Response execute(String driverCommand, Map<String, ?> parameters) {
30        if (onExecuteListener != null && !busy) {
31            busy = true;
32            onExecuteListener.executed(driverCommand, parameters);
33            busy = false;
34        }
35        return super.execute(driverCommand, parameters);
36    }
37}

So you can use this driver with Evinced SDK as usual.

1    YourOwnCustomDriver<AndroidElement> driver = new YourOwnCustomDriver<>(url, capabilities);
2    EvincedAppiumSdk evincedAppiumSdk = new EvincedAppiumSdk(driver);

Issue filtering

The Evinced Appium SDK supports advanced settings that allow you to apply additional configurations when generating issue reports. Simply create an instance of the EvincedConfig object with the appropriate options.

Below is an example of using EvincedConfig and IssueFilter:

Configure filters:

1final IssueFilter severityFilter = new IssueFilter()
2                .severity(Severity.NeedsReview);
3final IssueFilter issueFilter = new IssueFilter()
4                .issueType(IssueType.TappableArea, IssueType.ColorContrast);
5
6final MobileElement element = driver.findElementById(viewId);
7final IssueFilter elementFilter = new IssueFilter().matchByElements(element1);

Global filtering:

1// Create an example filter to exclude all Needs Review issues globally
2final IssueFilter globalFilter = new IssueFilter()
3                .severity(Severity.NeedsReview);
4
5// Add the IssueFilter to the creation instance of EvincedAppiumSdk
6final EvincedConfig globalEvincedConfig = new EvincedConfig()
7                .excludeFilters(globalFilter);
8final EvincedAppiumSdk evincedAppiumSdk = new EvincedAppiumSdk(driver, new InitOptions(globalEvincedConfig));

Filtering on an individual scan (analyze() or report() call):

1// Passing multiple exclude filters for an individual scan
2final IssueFilter multipleSeverityFilter = new IssueFilter()
3                .severity(Severity.NeedsReview, Severity.BestPractice);
4
5 final IssueFilter complexFilter = new IssueFilter()
6                .matchElementById("com.example:id/myElement")
7                .issueType(IssueType.CollidingControls)
8                .severity(Severity.NeedsReview);
9
10final IssueFilter complexFilter1 = new IssueFilter()
11                .matchElementByAccessibilityId("myElementA11yID")
12                .issueType(IssueType.DuplicateName, SpecialCharacters)
13                .severity(Severity.NeedsReview, BestPractice);
14final EvincedConfig evincedConfig = new EvincedConfig()
15                .excludeFilters(multipleSeverityFilter, multipleSeverityFilter, complexFilter);
16
17// Apply the configuration for a specific screen state scan
18evincedAppiumSdk.report(false, evincedConfig);
19evincedAppiumSdk.analyze(evincedConfig);

Depending on the purpose of filtering, you can use two directions -include or exclude using the appropriate methods:

• EvincedConfig().includeFilters(filter)
• EvincedConfig().excludeFilters(filters)

List of available issue filtering options

Config file

Along with the configuration creation options within the code, it is also possible to create filters using a JSON file. To import the file use the examples below:

1 // create EvincedConfig instance from file by specify absolute or relative file path
2 final EvincedConfig evincedConfigFromFile = EvincedConfig.fromFile("/project_root_evinced_config.json");
3
4 // create EvincedConfig instance from file by specify file name, located in resources directory
5 final EvincedConfig evincedConfigFromResources = EvincedConfig.fromResourcesFile("resources_evinced_config.json");

Example File

1{
2    "excludeFilters": [
3        {
4            "viewId": "viewId",
5            "accessibilityId": "a11yId",
6            "severities": [
7                {
8                    // you can use id or name separately
9                    "id": "3c68eb41-9227-481d-8eec-9dc8956b1900",
10                    "name": "Needs Review"
11                }
12            ],
13            "issueTypes": [
14                {
15                    // you can use id or name separately
16                    "id": "e871b2e7-1b6d-47bc-96db-7f3dff4bfd5a",
17                    "name": "Duplicate Name"
18                }
19            ]
20        }
21    ]
22}

Storing screenshots in reports

By default screenshots are provided in HTML reports for each scan, but for JSON-reports they are disabled by default. You can explicitly enable or disable screenshots using the InitOptions. Screenshots in JSON reports are provided in Base64 format.

Available screenshot options:

Global screenshot config:

1final InitOptions initOptions = new InitOptions(InitOptions.ScreenshotOption.Base64);
2// setup options during the SDK initialization
3final EvincedAppiumSdk evincedApiumSdk = new EvincedAppiumSdk(driver, initOptions);
4
5// change the existing options
6final InitOptions newOptions = new InitOptions(InitOptions.ScreenshotOption.Disabled);
7evincedApiumSdk.setOptions(newOptions);

Rule validation configurations

RulesConfig allows you to change the accessibility check for different rules. It is set globally and used for all subsequent reports.

Disabling the tappable area validation

This example disables validation for tappable area:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.TAPPABLE_AREA, false))

Disabling the label capitalization validation

This example disables validation for label capitalization:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.LABEL_CAPITALIZATION, false))

Disabling the color contrast validation

This example disables validation for color contrast:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.COLOR_CONTRAST, false)))

Customize color contrast options

Fuzziness factor configuration for color contrast. For example, the WCAG norm for this ratio is 4.5:1, but if your corporate colors have a ratio of 4.45:1, then this can be specified using the following options:

1final Map<String, Object> colorContrastOptions = new HashMap<>();
2colorContrastOptions.put("contrastRatioNormalText", 4.45f);
3colorContrastOptions.put("contrastRatioLargeText", 4.45f);
4colorContrastOptions.put("contrastRatioNonText", 4.45f);
5final RuleConfig colorContrastRuleConfig = new RuleConfig(IssueType.COLOR_CONTRAST, true, colorContrastOptions);
6evincedEngine.setOptions(new InitOptions().setRulesConfig(colorContrastRuleConfig));

Exporting meaningful labels

To export a JSON report of the accessibility label of images and buttons regardless of the scan results simply add the options flag:

1final InitOptions initOptions = new InitOptions();
2initOptions.putAdditionalOption("exportMeaningfulLabels", true);
3evincedEngine.setOptions(initOptions);

You should see a new JSON file created beside other report files with a default name: Evinced_Meaningful_Labels_Test_Results_11_15_2023_09_49_51.json. The structure of this JSON file is described in our Mobile Reports page.

To change the default folder destination and the file name of the meaningful labels JSON report you can use the second additional option with an absolute folder path:

1final InitOptions initOptions = new InitOptions();
2final File meaningfulLabelsFile = new File("evinced-reports", "SpecificMeaningfulLabels.json");
3initOptions.putAdditionalOption("exportMeaningfulLabels", true);
4initOptions.putAdditionalOption("exportMeaningfulLabelsPath", meaningfulLabelsFile.getAbsolutePath());
5evincedEngine.setOptions(initOptions);

Exporting elements with no issues

To export a JSON report of all scanned elements that had no issues found, simply add the additional options flag:

1final InitOptions initOptions = new InitOptions();
2initOptions.putAdditionalOption("exportNoIssuesElements", true);
3evincedEngine.setOptions(initOptions);

You should see a new JSON file created beside other report files with a default name: Evinced_Accessibility_Test_Results_No_Issues_Elements_07_25_2024_09_48_55.json.

The structure of this JSON file is described in our Mobile Reports page.

To change the default folder destination and the file name of the elements with no issues JSON report you can use another additional option with an absolute folder path:

1final InitOptions initOptions = new InitOptions();
2final File noIssuesElementsFile = new File("evinced-reports", "SpecificNoIssuesElements.json");
3initOptions.putAdditionalOption("exportNoIssuesElements", true);
4initOptions.putAdditionalOption("exportNoIssuesElementsPath", noIssuesElementsFile.getAbsolutePath());
5evincedEngine.setOptions(initOptions);

You can also obtain elements with no issues JSON report on test runtime using subreports:

1final InitOptions initOptions = new InitOptions();
2initOptions.putAdditionalOption("exportNoIssuesElements", true);
3evincedEngine.setOptions(initOptions);
4Report report = evincedAppiumSdk.report();
5NoIssuesElementsReport noIssueElementsReport = report.getSubReports().getNoIssueElements();
6List<NoIssuesElement> noIssuesElements = noIssueElementsReport.getItems();

Tappable Area

Every native mobile application features a wide variety of UI elements which are interactive via tap. Consider elements like buttons, text fields, toggles and others. Each of them should be tapped in order to become focused and to allow any further interaction. The lower the tappable area of the element (an imaginary square where user should tap) the less convenient it is to operate with the element. Platform constraints describe the width and height of minimum tappable area.

tappableAreaDimensions - the minimum size that does not trigger Tappable Area validation. By default for Android this value is 48dp, for iOS it is 44px.

For example

1final Map<String, Object> tapAreaDimension = new HashMap<>();
2tapAreaDimension.put("tappableAreaDimensions", 70);
3RuleConfig tapArea = new RuleConfig(IssueType.TappableArea, true, tapAreaDimension);
4InitOptions initOptions = new InitOptions().setRulesConfig(tapArea);
5evincedAppiumSdk.setOptions(initOptions);

Uploading reports to Evinced Platform

[Evinced Platform] allows you to seamlessly collect, organize, and visualize Evinced accessibility reports in one place. In this section, we will guide you through the key functionalities of the upload methods from Evinced Appium Java SDK, which was introduced in version 1.20.0, to the Evinced Platform. This upload method is fully compatible with the previous versions of the Evinced Espresso SDK API, and is disabled by default.

Initialize report uploading to the Evinced Platform

In order to start using [Evinced Platform] feature you need to include the PlatformConfig in the InitOptions:

1@BeforeClass
2public static void setup() {
3    ...
4    final InitOptions options = new InitOptions(
5            new InitOptions.PlatformConfig(
6                InitOptions.UploadOption.ENABLED_BY_DEFAULT
7            )
8        );
9    evincedAppiumSdk.setOptions(options);
10}

In this case, all of the reports from a test class will be uploaded to the [Evinced Platform] where you can view the accessibility issues.

Selective uploading

You can configure uploading only a specific set of reports:

1@Test
2public void test1() {
3    ...
4    final Report report1 = evincedAppiumSdk.report();
5    ...
6    final Report report2 = evincedAppiumSdk.report(PlatformUpload.ENABLED);
7}

In this case, only the second report will be uploaded to [Evinced Platform].

The same behavior applies to internally stored reports reportStored() after the analyze() calls and to the stopAnalyze() in the continuous mode:

1@Test
2public void test1() {
3    ...
4    evincedEngine.analyze();
5    List<Report> reportList = evincedAppiumSdk.reportStored(PlatformUpload.ENABLED);
6}

Or:

1@Test
2public void test1() {
3    ...
4    evincedEngine.startAnalyze();
5    // test body
6    List<Report> reportList = evincedAppiumSdk.stopAnalyze(PlatformUpload.ENABLED);
7}

Notice: that in these examples we didn't use InitOptions.PlatformConfig. It's not required to use this config argument to enable the selective report uploading. Please find more information about continuous mode here if needed.

Disable report uploading to Platform

In order to disable the upload method for all reports without changing arguments for each report(), reportStored() or stopAnalyze() methods you can use the following InitOption:

1@BeforeClass
2public static void setup() {
3    ...
4    final InitOptions options = new InitOptions(
5            new InitOptions.PlatformConfig(
6                // Setting the forceDisableReportsUpload flag to true, disable all reports uploading.
7                true
8            )
9        );
10    evincedAppiumSdk.setOptions(options);
11}
12
13@Test
14public void test1() {
15    ...
16    final Report report1 = evincedAppiumSdk.report();
17    ...
18    final Report report2 = evincedAppiumSdk.report(PlatformUpload.ENABLED);
19}

Labeling uploaded reports

To be able to distinguish between different reports on [Evinced Platform], the following pre-defined labels will be collected and added programmatically.

Customizing labels before uploading reports

To distinguish between reports from different parts of your applications or tests you can also add custom labels to the uploaded reports.

1@Test
2public void test1() {
3    ...
4    Map<String, String> labels1 = new HashMap<>();
5    labels1.put("MyCustomLabelKey1", "MyCustomLabelValue1");
6    final Report report1 = evincedAppiumSdk.report(PlatformUpload.ENABLED, labels1);
7    ...
8    Map<String, String> labels2 = new HashMap<>();
9    labels2.put("MyCustomLabelKey2", "MyCustomLabelValue2");
10    final Report report2 = evincedAppiumSdk.report(PlatformUpload.ENABLED, labels2);
11}

The custom key "MyCustomLabelKey1" and the custom value "MyCustomLabelValue1" will be attached to the first report. The custom key "MyCustomLabelKey2" and the custom value "MyCustomLabelValue2" will be attached to the second report.

Customizing labels before uploading reports for the entire test class

In order to label all reports inside your test class with a common key/value, use the addTestCaseMetadata() method:

1@BeforeClass
2public static void setup() {
3    ...
4    evincedAppiumSdk.addTestCaseMetadata("MyCustomCommonLabelKey", "MyCustomCommonLabelValue");
5    ...
6}
7
8@Test
9public void test1() {
10    ...
11    Map<String, String> labels1 = new HashMap<>();
12    labels1.put("MyCustomLabelKey1", "MyCustomLabelValue1");
13    final Report report1 = evincedAppiumSdk.report(PlatformUpload.ENABLED, labels1);
14    ...
15    Map<String, String> labels2 = new HashMap<>();
16    labels2.put("MyCustomLabelKey2", "MyCustomLabelValue2");
17    final Report report2 = evincedAppiumSdk.report(PlatformUpload.ENABLED, labels2);
18}

In this case, all reports in the test class scope will have "MyCustomCommonLabelKey"/"MyCustomCommonLabelValue". In addition, "MyCustomLabelKey1"/"MyCustomLabelValue1" will be added only to the first report and "MyCustomLabelKey2"/"MyCustomLabelValue2" will be added only to the second report.

Test assertion

If you want to interrupt the test execution in case the report is not uploaded, it's possible to set the failTestOnUploadError argument of the InitOptions.PlatformConfig:

1@BeforeClass
2public static void setup() {
3    ...
4    final InitOptions options = new InitOptions(
5            new InitOptions.PlatformConfig(
6                InitOptions.UploadOption.DISABLED_BY_DEFAULT,
7                // Enabled the failTestOnUploadError flag
8                true
9            )
10        );
11    evincedAppiumSdk.setOptions(options);
12}
13
14@Test
15public void test1() {
16    ...
17    Map<String, String> labels1 = new HashMap();
18    labels1.put("MyCustomLabelKey1", "MyCustomLabelKey1");
19    final Report report1 = evincedAppiumSdk.report(PlatformUpload.ENABLED, labels1);
20    // The test will fail in case of any uploading error, it can be an internal connection or routing problems
21    ...
22}

In the case of upload errors you should see an exception with a detailed error message that can be found in the logcat window to help understand the cause of the error and possible solutions.

List of available Platform upload options

forceDisableReportsUpload

forceDisableReportsUpload = true - the upload method is completely disabled. All report uploading will be disabled regardless of the upload argument:

1@BeforeClass
2public static void setup() {
3    ...
4    final InitOptions options = new InitOptions(
5            new InitOptions.PlatformConfig(
6                    // forceDisableReportsUpload = true
7                    true
8            )
9    );
10    ...
11}
12
13@Test
14public void test1() {
15    ...
16    // This report won't be uploaded to Evinced Platform, because of forceDisableReportsUpload = true
17    Report report1 = evincedAppiumSdk.reportStored(PlatformUpload.ENABLED);
18    ...
19    // These reports won't be uploaded to Evinced Platform, because of forceDisableReportsUpload = true
20    List<Report> reportList2 = evincedAppiumSdk.reportStored(PlatformUpload.ENABLED);
21    ...
22    // These reports won't be uploaded to Evinced Platform, because of forceDisableReportsUpload = true
23    List<Report> reportList3 = evincedAppiumSdk.stopAnalyze(PlatformUpload.ENABLED);
24    ...
25}

forceDisableReportsUpload = false is a default value of the upload to Platform configuration;

uploadOption

InitOptions.UploadOption.ENABLED_BY_DEFAULT - All reports will be uploaded, except for the reports with the upload argument equals to PlatformUpload.DISABLED:

1@BeforeClass
2public static void setup() {
3    ...
4    final InitOptions options = new InitOptions(
5            new InitOptions.PlatformConfig(
6                // uploadOption = InitOptions.UploadOption.ENABLED_BY_DEFAULT,
7                InitOptions.UploadOption.ENABLED_BY_DEFAULT
8            )
9        );
10    evincedAppiumSdk.setOptions(options);
11    ...
12}
13
14@Test
15public void test1() {
16    ...
17    // This report won't be uploaded to Evinced Platform
18    Report report1 = evincedAppiumSdk.reportStored(PlatformUpload.DISABLED);
19    ...
20    // These reports won't be uploaded to Evinced Platform
21    List<Report> reportList2 = evincedAppiumSdk.reportStored(PlatformUpload.DISABLED);
22    ...
23    // These reports won't be uploaded to Evinced Platform
24    List<Report> reportList3 = evincedAppiumSdk.stopAnalyze(PlatformUpload.DISABLED);
25    ...
26}

InitOptions.UploadOption.DISABLED_BY_DEFAULT - Only reports that are marked with upload argument equal to PlatformUpload.ENABLED will be uploaded:

1@BeforeClass
2public static void setup() {
3    ...
4    final InitOptions options = new InitOptions(
5            new InitOptions.PlatformConfig(
6                // uploadOption = InitOptions.UploadOption.DISABLED_BY_DEFAULT,
7                InitOptions.UploadOption.DISABLED_BY_DEFAULT
8            )
9        );
10    evincedAppiumSdk.setOptions(options);
11    ...
12}
13
14@Test
15public void test1() {
16    ...
17    // This report will be uploaded to Evinced Platform
18    Report report1 = evincedAppiumSdk.reportStored(PlatformUpload.ENABLED);
19    ...
20    // These reports will be uploaded to Evinced Platform
21    List<Report> reportList2 = evincedAppiumSdk.reportStored(PlatformUpload.ENABLED);
22    ...
23    // These reports will be uploaded to Evinced Platform
24    List<Report> reportList3 = evincedAppiumSdk.stopAnalyze(PlatformUpload.ENABLED);
25    ...
26}

failTestOnUploadError

failTestOnUploadError = true - Stop the test execution in case of an upload error;

failTestOnUploadError = false - Continues the test execution no matter of any upload errors. Default value.

The state table of the configurations is as following:

List of available Platform upload arguments

report(PlatformUpload.ENABLED) - Upload the report or reports. Will have an effect when the InitOptions.PlatformConfig is InitOptions.UploadOption.ENABLED_BY_DEFAULT or InitOptions.UploadOption.DISABLED_BY_DEFAULT. Can be used as an argument is report(PlatformUpload.ENABLED), stopAnalyze(PlatformUpload.ENABLED) or reportStored(PlatformUpload.ENABLED) methods;

report(PlatformUpload.DISABLED) - Don't upload the report or reports. Will have an effect when the InitOptions.PlatformConfig is InitOptions.UploadOption.ENABLED_BY_DEFAULT or InitOptions.UploadOption.DISABLED_BY_DEFAULT Can be used as an argument is report(PlatformUpload.DISABLED), stopAnalyze(PlatformUpload.DISABLED) or reportStored(PlatformUpload.DISABLED) methods.

Integration Android Mobile Kit feature

Android/iOS Mobile Kit adds new types of validations for your target application.

Add the import statement to the beginning of your test file.

1import com.evinced.appium.sdk.core.models.MobileKitConfig;

Setup Mobile Kit config.

1@BeforeClass
2public static void setupAppiumDriver() throws MalformedURLException, IOException {
3    capabilities = new DesiredCapabilities();
4    // ...omitting the setting of capabilities for brevity
5
6    driver = new EvincedAppiumAndroidDriver<>(url, capabilities);
7    InitOptions initOptions = new InitOptions(InitOptions.ScreenshotOption.Both)
8    // Setting mobileKitEnabled = true
9    .setMobileKitConfig(new MobileKitConfig(true));
10
11    evincedAppiumSdk = new EvincedAppiumSdk(driver, initOptions);
12    evincedAppiumSdk.setupCredentials(<your service account ID>, <your API key>);
13}

Note: Make sure that your target application has the MobileKit dependency as described here: Android Mobile Kit Documentation.

In case you need to get a certain validation and your test relies on this validation. You can stop the test execution in case of Mobile Kit connection issues using the failOnMobileKitConnectionError flag.

1@BeforeClass
2public static void setupAppiumDriver() throws MalformedURLException, IOException {
3    capabilities = new DesiredCapabilities();
4    // ...omitting the setting of capabilities for brevity
5
6    driver = new EvincedAppiumAndroidDriver<>(url, capabilities);
7    InitOptions initOptions = new InitOptions(InitOptions.ScreenshotOption.Both)
8    // Setting mobileKitEnabled = true
9    // And failMobileKitConnectionError = true
10    .setMobileKitConfig(new MobileKitConfig(true, true));
11    evincedAppiumSdk = new EvincedAppiumSdk(driver, initOptions);
12    evincedAppiumSdk.setupCredentials(<your service account ID>, <your API key>);
13}

If you can't use the provided EvincedAppiumAndroidDriver or EvincedAppiumIOSDriver in your setup. Please, don't forget to obtain a driver back using evincedAppiumSdk.getDriver(). Due to the internal SDK logic, the driver will be recreated in case of Mobile Kit usage.

1@BeforeClass
2public static void setupAppiumDriver() throws MalformedURLException, IOException {
3   capabilities = new DesiredCapabilities();
4   // ...omitting the setting of capabilities for brevity
5
6   driver = new YourCustomDriver<>(url, capabilities);
7   InitOptions initOptions = new InitOptions(InitOptions.ScreenshotOption.Both)
8   // Setting mobileKitEnabled = true
9   .setMobileKitConfig(new MobileKitConfig(true));
10
11   evincedAppiumSdk = new EvincedAppiumSdk(driver, initOptions);
12   evincedAppiumSdk.setupCredentials(<your service account ID>, <your API key>);
13
14   driver = (AndroidDriver<AndroidElement>) evincedAppiumSdk.getDriver();
15}

Note: Make sure that your target application has the Mobile Kit dependency.

Analyze prefetched appium data

Alternatively, you can store appium data ahead of time and then check it with EvincedAppiumAnalyzer.

To use it, firstly you need to save driver.getPageSource() and driver.getScreenshotAs(OutputType.FILE) from the appium driver and optionally save it somewhere.

1String pageSourceXml = driver.getPageSource();
2File screenshotFile = driver.getScreenshotAs(OutputType.FILE);

Then in your test you need to create an EvincedAppiumAnalyzer class instead of EvincedAppiumSdk. You will need to use separate builder-methods for Android and for iOS. Also, it's needed to setup credentials as well.

1// for Android
2evincedAppiumAnalyzer = EvincedAppiumAnalyzer.buildForAndroid(/*displayDensity*/ 440L, /*realDisplaySize*/ "1080x2220");
3
4// for iOS
5evincedAppiumAnalyzer = EvincedAppiumAnalyzer.buildForIOS();
6
7evincedAppiumAnalyzer.setupCredentials(YOUR_SERVICE_ACCOUNT_ID, YOUR_API_KEY);

Note: For android tests, make sure to use the same displayDensity and realDisplaySize that were used to fetch appium data.

Then you can use method evincedAppiumAnalyzer.analyze() to check your data.

1@Test
2public void testAnalyze() {
3    Report report = evincedAppiumAnalyzer.analyze(getPageSourceString, screenshotFile);
4    assertNotNull(report);
5}

Alternatively, you can firstly store data with evincedAppiumAnalyzer.store() method and then analyze it with evincedAppiumAnalyzer.analyze().

1@Test
2public void testStoreAnalyze() {
3    evincedAppiumAnalyzer.store(getPageSourceString1, screenshotFile1);
4    evincedAppiumAnalyzer.store(getPageSourceString2, screenshotFile2);
5    evincedAppiumAnalyzer.store(getPageSourceString3, screenshotFile3);
6
7    List<Report> reports = evincedAppiumAnalyzer.analyze();
8    assertEquals(3, reports.size());
9}

Additionally, you can add some of your metadata to the final report for each usecase:

1evincedAppiumAnalyzer.store(getPageSourceString, screenshotFile, Collections.singletonMap("someKey1", "someValue1"));

1Report report = evincedAppiumAnalyzer.analyze(getPageSourceString, screenshotFile, Collections.singletonMap("someKey", "someValue"));

API

EvincedAppiumSdk

public EvincedAppiumSdk(AppiumDriver<?> originalDriver)

Default constructor which accepts an AppiumDriver instance for performing accessibility snapshot collection. Can be easily instantiated in the following way:

1IOSDriver driver = new IOSDriver<IOSElement>(url, capabilities);
2EvincedAppiumSdk evincedSdk = new EvincedAppiumSdk(driver);

public EvincedAppiumSdk(AppiumDriver<?> originalDriver, EvincedAppiumSdkOptions options)

Additional constructor which allows you to configure the different aspects of a11y testing via EvincedAppiumSdkOptions object. For instance, one can set a custom path for storing reporting artifacts by using the example below:

1driver = new IOSDriver<IOSElement>(url, capabilities);
2
3EvincedAppiumSdkOptions evincedSdkOptions = new EvincedAppiumSdkOptions(); // create options object
4evincedSdkOptions.setOutputDir("my-output-dir"); // set custom output directory
5
6evincedSdk = new EvincedAppiumSdk(driver, evincedSdkOptions);

public boolean setupCredentials(String serviceAccountId, String apiKey)

A method to add credentials confirming the validity of the user’s license. It accepts the following arguments:

• serviceAccountId - Evinced service account ID
• apiKey - Evinced API key

Returns a boolean flag indicating the success of the operation - if it is false there is something wrong with either validity of the license or with the checking infrastructure. See logs for more detailed output.

public boolean setupOfflineCredentials(String serviceAccountId, String accesstoken)

A method to add credentials confirming the validity of the user’s license with an offline token. It accepts the following arguments:

• serviceAccountId - Evinced service account ID
• accesstoken - Evinced access token

Returns a boolean flag indicating the success of the operation - if it is false there is something wrong with either validity of the license or with the checking infrastructure. See logs for more detailed output.

public void startAnalyze(EvincedConfig evincedConfig)

Watch for screen state changes and record all accessibility issues until stopAnalyze() is called. It works the same way as EvincedAppiumSdk#analyze() but collects snapshot on each appium command that can change application state.

Make sure that driver you set in constructor extends EvincedAppiumAndroidDriver or EvincedAppiumIOSDriver, or implements CanSetOnExecuteListener in similar way.

Available overloads: public void startAnalyze();

public List<Report> stopAnalyze(boolean shouldAssert) Stop flow analyze and returns a report with all issues found from the last startAnalyze() call to allow users to perform any advanced assertions on their own.

If it is not needed, simple assertion may be delegated directly to the SDK by supplying shouldAssert argument set to true. In this case, the SDK will automatically throw an AssertionError if there is at least one issue found.

Available overloads: public void stopAnalyze();

public Report report(shouldAssert: boolean)

Generates accessibility report for the current state of the application. Returns a Report instance to allow users to perform any advanced assertions on their own. If it is not needed, a simple assertion may be delegated directly to the SDK by supplying shouldAssert argument set to true. In this case, the SDK will automatically throw an AssertionError if there is at least one issue found. For more information regarding the Report object , please see our detailed Mobile Reports page.

This method always writes HTML and JSON files to the configured directory.

Available overloads:

public Report report();

Example usage:

1Report report = evincedSdk.report(true); // assert is performed by EvincedAppiumSdk
2
3Report reportForMyOwnAssertion = evincedSdk.report();
4assertEquals(reportForMyOwnAssertion.hasIssues(), false); // performing assertions on your own

public void analyze()

Collects an accessibility snapshot of the current application state and puts it into internal in-memory storage. This method is supposed to be used in conjunction with EvincedAppiumSdk.assertStored() for performing actual assertions against the content of this storage.

Example usage:

1  IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
2            .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
3    // we have achieved the state of the app we would like to test. Let's take a snapshot.
4   evincedAppiumSdk.analyze();

public List<Report> reportStored(shouldAssert: boolean)

Generates a list of accessibility reports, one for each accessibility snapshot collected by EvincedAppiumSdk.analyze(). Just like EvincedAppiumSdk.report() this method may either perform simple assertion on its own (throwing an exception if there is at least one report with issues) or allow users to define their own assertions on the returned Report objects. This behavior can be configured via the shouldAssert flag.

This method always writes HTML and JSON files to the configured directory. Both of the files contain combined information about all of the application states collected. For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

Available overloads:

public List<Report> reportStored();

Example usage:

1@AfterClass
2public static void tearDown() {
3    // Option 1
4    List<Report> reportsForOwnAssertion = evincedSdk.reportStored(); // making our own assertions
5    // Option 2
6    List<Report> reports = evincedSdk.reportStored(true); // delegating assertion to the SDK
7    // Option 3
8    List<Report> reportsWithIssues = reportsForOwnAssertion.stream() // Filter out only reports which contain some issues
9      .filter(report -> report.hasIssues())
10      .collect(Collectors.toList());
11    assertEquals(reportsWithIssues.size(), 0); // Making the actual assertion
12
13    driver.quit();
14}

public void clearStored()

Removes all of the accessibility snapshots previously collected by EvincedAppiumSdk.analyze() calls.

InitOptions

1public InitOptions()

Default public constructor with no arguments.

1public InitOptions(String outputDir)
2public InitOptions(String outputDir, EvincedConfig evincedConfig)

Overloaded constructors

1public InitOptions setOutputDir(String outputDir)

Method for defining a custom output directory where a user would like to store reporting artifacts (JSON and HTML files).

1public InitOptions setEvincedConfig(EvincedConfig evincedConfig)

Method for defining a EvincedConfig object as a global setting

1public InitOptions screenshotOption(ScreenshotOption screenshotOption)

Set option to save a screenshot in a output JSON report or as a separate .jpeg file.

1public InitOptions setRuleConfigs(List<RuleConfig> ruleConfigs)

Set rule configs

1public InitOptions setReportName(String reportName)

Set the output HTML/JSON report name prefix

1    public InitOptions setMobileKitConfig(MobileKitConfig mobileKitConfig)

Sets Mobile Kit feature configurations

EvincedConfig

An object that defines the options that will be used when performing accessibility validations and generating reports.

Default public constructors for creating EvincedConfig:

1public EvincedConfig()

Set exclude filters:

1public EvincedConfig excludeFilters(IssueFilter... excludeFilters)
2public EvincedConfig excludeFilters(List<IssueFilter> excludeFilters)

Set include filters:

1public EvincedConfig includeFilters(IssueFilter... includeFilters)
2public EvincedConfig includeFilters(List<IssueFilter> includeFilters)

Enable storing screenshots in JSON-report:

ScreenshotOption

Enum that allows enable or disable saving screenshots in JSON-reports and can be Base64, File, Both or Disabled.

IssueFilter

A filter object that allows you to exclude UI elements from availability reports by certain criteria

IssueFilter public methodes:

1public IssueFilter matchElementById(String... viewIds)
2public IssueFilter matchElementByAccessibilityId(String... accessibilityIds)
3public IssueFilter matchByElements(MobileElement... mobileElement)
4public IssueFilter matchByElements(List<MobileElement> mobileElements)
5public IssueFilter severity(List<Severity> severities)
6public IssueFilter severity(Severity... severity)
7public IssueFilter issueType(List<IssueType> issueTypes)
8public IssueFilter issueType(IssueType... issueType)

RuleConfig

Default constructor:

1public RuleConfig(IssueType name, boolean enabled)

IssueType - the IssueType on which this rule config should be applied enabled - allows to enable/disable specified IssueType. By default enabled = true

Tutorials

Generating a comprehensive a11y report for your application

In this tutorial, we will enhance our existing Appium UI test with the Evinced Appium SDK in order to check our application for accessibility issues. In order to get started you will need the following:

1. All of the prerequisites for the Evinced Appium SDK should be met
2. Evinced Appium SDK should be installed in your project

Preface - existing UI test overview

Let’s consider the following basic UI test as our starting point.

1import io.appium.java_client.MobileBy;
2import io.appium.java_client.ios.IOSDriver;
3import io.appium.java_client.ios.IOSElement;
4import io.appium.java_client.remote.MobileCapabilityType;
5
6import java.io.File;
7import java.io.IOException;
8import java.net.MalformedURLException;
9import java.net.URL;
10import org.openqa.selenium.remote.DesiredCapabilities;
11import org.junit.AfterClass;
12import org.junit.BeforeClass;
13import org.junit.Test;
14import org.openqa.selenium.support.ui.ExpectedConditions;
15import org.openqa.selenium.support.ui.WebDriverWait;
16
17public class SwiftRadioTest {
18
19    public static URL url;
20    public static DesiredCapabilities capabilities;
21    public static IOSDriver<IOSElement> driver;
22
23    @BeforeClass
24    public static void setupAppiumDriver() throws MalformedURLException, IOException {
25        final String URL_STRING = "http://127.0.0.1:4723/wd/hub";
26        url = new URL(URL_STRING);
27
28        final Class clazz = EvincedA11yValidatorTest.class;
29        final File file = new File(clazz.getResource("/SwiftRadio.app").getFile());
30
31        capabilities = new DesiredCapabilities();
32        capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "XCUITest");
33        capabilities.setCapability("useNewWDA", false);
34        capabilities.setCapability(MobileCapabilityType.NO_RESET, true);
35        capabilities.setCapability(MobileCapabilityType.DEVICE_NAME, "iPhone Simulator");
36        capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "iOS");
37        capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION, "12.0");
38        capabilities
39                .setCapability(MobileCapabilityType.APP,
40                        file.getAbsolutePath());
41
42        driver = new IOSDriver<IOSElement>(url, capabilities);
43    }
44
45    @AfterClass
46    public static void tearDown() {
47        driver.quit();
48    }
49
50    @Test public void testStationsScreen() {
51        IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
52                .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
53
54        firstStationTableCell.click();
55
56        IOSElement playButton = (IOSElement) new WebDriverWait(driver, 30)
57                .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("XCUIElementTypeButton")));
58    }
59}

We wrote this test for an application called Swift Radio. It is a small open-source app for listening to various radio stations and podcasts.

The purpose of these tests is to check the functionality of the main application screen and ensure a user can successfully navigate to their choice of the radio station. For now, this test is only concerned with the functional testing of the app. However, with the help of the Evinced Appium SDK, we can also check it for accessibility issues along the way. Let’s go through this process with the following step-by-step instructions.

Step #1 - Initialize the Evinced Appium SDK

Before making any assertions against our app, we need to initialize EvincedAppiumSdk object. This object is used primarily as an entry point to all of the accessibility scanning features. Since we are going to use it primarily within our tests, the best place for its initialization will be our setupAppiumDriver method which gets executed first.

1import io.appium.java_client.MobileBy;
2import io.appium.java_client.ios.IOSDriver;
3import io.appium.java_client.ios.IOSElement;
4import io.appium.java_client.remote.MobileCapabilityType;
5
6import java.io.File;
7import java.io.IOException;
8import java.net.MalformedURLException;
9import java.net.URL;
10import org.openqa.selenium.remote.DesiredCapabilities;
11import org.junit.AfterClass;
12import org.junit.BeforeClass;
13import org.junit.Test;
14import org.openqa.selenium.support.ui.ExpectedConditions;
15import org.openqa.selenium.support.ui.WebDriverWait;
16
17public class SwiftRadioTest {
18
19    public static URL url;
20    public static DesiredCapabilities capabilities;
21    public static IOSDriver<IOSElement> driver;
22
23    public static EvincedAppiumSdk evincedAppiumSdk;
24
25    @BeforeClass
26    public static void setupAppiumDriver() throws MalformedURLException, IOException {
27        final String URL_STRING = "http://127.0.0.1:4723/wd/hub";
28        url = new URL(URL_STRING);
29
30        final Class clazz = EvincedA11yValidatorTest.class;
31        final File file = new File(clazz.getResource("/SwiftRadio.app").getFile());
32
33        capabilities = new DesiredCapabilities();
34        capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "XCUITest");
35        capabilities.setCapability("useNewWDA", false);
36        capabilities.setCapability(MobileCapabilityType.NO_RESET, true);
37        capabilities.setCapability(MobileCapabilityType.DEVICE_NAME, "iPhone Simulator");
38        capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "iOS");
39        capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION, "12.0");
40        capabilities
41                .setCapability(MobileCapabilityType.APP,
42                        file.getAbsolutePath());
43
44        driver = new IOSDriver<IOSElement>(url, capabilities);
45        evincedAppiumSdk = new EvincedAppiumSdk(driver);
46        evincedAppiumSdk.setupCredentials(<your service account ID>, <your API key>);
47    }
48
49    @AfterClass
50    public static void tearDown() {
51        driver.quit();
52    }
53
54    @Test
55    public void testStationsScreen() {
56        IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
57                .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
58
59        firstStationTableCell.click();
60
61        IOSElement playButton = (IOSElement) new WebDriverWait(driver, 30)
62                .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("XCUIElementTypeButton")));
63    }
64}