Appium Java SDK

The Evinced Appium Java SDK integrates with new or existing Appium tests to automatically detect accessibility issues. With the addition of as few as 5 lines of code to your Appium framework, you can begin to analyze your entire application to understand how it can become more accessible. At the conclusion of the test, a rich and comprehensive HTML or JSON report is generated to track issues in any reporting tool.

Supported versions / frameworks

Evinced Appium SDK supports the following values for Appium’s automationName desired capability:

  • XCUI SDK
  • UIAutomator2
  • Espresso

Older versions of automation drivers (e.g deprecated Appium iOS driver based on UIAutomation) are not supported.

Prerequisites:

  • Java 8 or higher
  • Appium Server ver. 1.21 or higher
  • Appium Java client v7 or higher (older version may still be compatible)

Get Started

Setup

In order to use any of the Evinced Mobile SDKs you first need to create an Evinced account. Once your account is created, you will get your apiKey and a matching serviceAccountId. Pass these these tokens using the setupCredentials() method and Evinced will validate access upon test execution. If an outbound internet connection is not available in your running environment - contact us at support@evinced.com to get an offline APIKey.

Install Evinced Appium SDK from the remote registry

If you are using Gradle as your build tool follow the steps below:

  1. Add Artifactory plugin to the plugins section of your build.gradle:

    1 plugins {
    2 // ... other plugins
    3 id 'com.jfrog.artifactory' version '4.24.14'
    4 }

    Or if you use kotlin to configurate build.gradle.kts

    1 plugins {
    2 // ... other plugins
    3 id("com.jfrog.artifactory") version "4.24.14"
    4 }
  2. To the bottom of your build.gradle file add the next configuration for Artifactory plugin:

    1artifactory {
    2 contextUrl = 'https://evinced.jfrog.io/artifactory/'
    3
    4 resolve {
    5 repository {
    6 repoKey = 'public-gradle'
    7 maven = true
    8 }
    9 }
    10}

    For build.gradle.kts:

    1artifactory {
    2 setContextUrl("https://evinced.jfrog.io/artifactory")
    3
    4 resolve {
    5 repository {
    6 setRepoKey("public-gradle")
    7 setMaven(true)
    8 }
    9 }
    10}

If you are using Maven, add the Evinced package repository URL to your build file:

1<repositories>
2 <repository>
3 <id>evinced-repository</id>
4 <url>https://evinced.jfrog.io/artifactory/public-gradle</url>
5 </repository>
6</repositories>

NOTE: If you see an error com.github.appium:java-client:jar:927db52 was not found please also at the additional Jitpack repository.

1 <repositories>
2 <repository>
3 <id>jitpack</id>
4 <url>https://jitpack.io</url>
5 </repository>
6</repositories>
Adding Evinced Appium SDK as a project dependency

The SDK can be added to your project as a Maven or Gradle dependency.

When working with Maven add the following dependency to your pom.xml file:

1<dependency>
2 <groupId>com.evinced</groupId>
3 <artifactId>appium-sdk</artifactId>
4 <version>{sdk-version}</version>
5 <scope>test</scope>
6</dependency>

When working with Gradle add the following to your build.gradle dependencies section:

1dependencies {
2 // ... other project dependencies
3 testImplementation(group: 'com.evinced', name: 'appium-sdk', version: '{sdk-version}', classifier: 'all')
4}

For build.gradle.kts:

1dependencies {
2 // ... other project dependencies
3 testImplementation("com.evinced:appium-sdk:{sdk-version}")
4}

Important notice: In case of usage of Appium Java Client 8 the {sdk-version} should include the "_java_client_v8" identifier string. The following example will support Appium Java Client 8:

1<dependency>
2 <groupId>com.evinced</groupId>
3 <artifactId>appium-sdk</artifactId>
4 <version>1.10.0_java_client_v8</version>
5 <scope>test</scope>
6</dependency>

The following example without the "_java_client_v8" identifier string will support Appium Java Client 7 and lower by default:

1<dependency>
2 <groupId>com.evinced</groupId>
3 <artifactId>appium-sdk</artifactId>
4 <version>1.10.0</version>
5 <scope>test</scope>
6</dependency>

Replace {sdk-version} with the latest SDK version that can be found here: appium sdk jfrog.

Install Evinced Appium SDK from local ZIP archive

You can also perform installation right from the standalone .jar distribution. In this case, you need to do the following:

  1. Download the .jar file.
  2. Unpack the provided evinced-appium-sdk.zip to any desirable location
  3. Add the following dependencies entries pointing to appium-sdk-all.jar
    1. Gradle:
      1implementation files('/Users/<path-to-your-upacked-folder>/appium-sdk-all.jar')
    2. Maven:
      1. First, install the “all“ jar into your local Maven repository by the following command:
        1mvn install:install-file –Dfile=/Users/<path-to-unpacked-folder>/appium-sdk-all.jar -DgroupId=com.evinced -DartifactId=evinced-appium-sdk -Dversion={sdk-version}
      2. Add the corresponding dependency into your pom.xml:
        1 <dependency>
        2 <groupId>com.evinced</groupId>
        3 <artifactId>evinced-appium-sdk</artifactId>
        4 <version>{sdk-version}</version>
        5 </dependency>
    3. Trigger your package manager download new dependencies either via IDE or command line

ZIP archive also contains two complimentary .jar files which you can utilize:

  • appium-sdk-javadoc.jar contains rich JavaDoc documentation which you can view in any web browser
  • appium-sdk-sources.jar can be attached to your Evinced Appium SDK installation in order to let you leverage JavaDoc right in your IDE. Please, refer to the corresponding documentation of your developer tools

Your first test

Initialize the SDK object

Add the import statement to the beginning of your test file.

1import com.evinced.appium.sdk.core.EvincedAppiumSdk;

Under the hood, the Evinced SDK utilizes several Appium commands in order to get accessibility data from the device under test. For this reason, we need to initialize EvincedAppiumSdk with the Appium driver instance in the following way:

1public static EvincedAppiumSdk evincedSdk;
2
3@BeforeClass
4public static void setupAppiumDriver() throws MalformedURLException, IOException {
5 capabilities = new DesiredCapabilities();
6 // ...omitting the setting of capabilities for brevity
7
8 driver = new IOSDriver<IOSElement>(url, capabilities);
9
10 evincedSdk = new EvincedAppiumSdk(driver);
11}

Here we used the JUnit BeforeClass annotation to create a method that will be executed at the beginning of the test suite. This is often where the Appium Driver initialization takes place, therefore we can also use it to initialize the EvincedAppiumSdk instance as well.

Provide your license information

The Evinced Appium SDK requires authentication credentials which consist of two string values mentioned above: apiKey and serviceAccountId.

For this purpose use the method called evincedAppiumSdk.setupCredentials(serviceAccountId, apiKey) as shown:

1public static EvincedAppiumSdk evincedSdk;
2
3@BeforeClass
4public static void setupAppiumDriver() throws MalformedURLException, IOException {
5 //... the rest of our setup code
6
7 evincedSdk = new EvincedAppiumSdk(driver);
8
9 evincedSdk.setupCredentials(<your service account ID>, <your API key>);
10}

You can confirm the success of the authentication in the logs.

Add Evinced accessibility checks

1@Test
2public void testMyAppScreen() {
3 IOSElement someTableCell = (IOSElement) new WebDriverWait(driver, 30)
4 .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("TableViewCell")));
5
6 firstStationTableCell.click();
7 //... do other interactions with the app
8
9 // Run analysis and get the accessibility report
10 Report report = evincedSdk.report();
11
12 // Assert that there are no accessibility issues
13 assertTrue(report.getIssues().size() == 0);
14}

Regardless of the test result, the report method will always output JSON and HTML files. You should be able to find the files by browsing the local folder called evinced-reports . For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

If you only need Report objects during test runtime, and you don't need JSON and HTML report files saved, you can disable creating report files with disableAllReportsOutputFiles additional option.

1 final InitOptions initOptions = new InitOptions()
2 .putAdditionalOption("disableAllReportsOutputFiles", true);
3 evincedAppiumSdk.setOptions(initOptions);

Continuous mode

Continuous mode allows continual scanning of the application without the need to insert individual scan calls with in the test code. Simply specify the start and end points using the following methods evincedEngine.startAnalyze() and evincedEngine.stopAnalyze(). Then Evinced engines will automatically scan the application after each Appium command. All issues detected during the tests will be added to the HTML and JSON reports automatically generated with the stopAnalyze() method is called.

To use continuous mode, you need to replace the standard Appium driver with the EvincedAppiumAndroidDriver for Android or EvincedAppiumIOSDriver. No changes are needed to your desired capabilities or other driver setup.

1@BeforeClass
2public static void setupAppiumDriver() throws IOException {
3
4 // Usual Appium configuration
5
6 driver = new EvincedAppiumAndroidDriver<>(url, capabilities);
7 evincedAppiumSdk = new EvincedAppiumSdk(driver);
8 evincedAppiumSdk.setupCredentials(TestBuildConfig.EVINCED_SERVICE_ACCOUNT_ID, TestBuildConfig.EVINCED_API_KEY);
9}

To initialize the Evinced engines simply use the evincedAppiumSdk.startAnalyze() method and finish it use evincedAppiumSdk.stopAnalyze(). You can use it within a single test method or scan an entire class (or more) using before and after hooks.

Here is an example of using continuous mode in a single test case:

1@Test
2public void testContinuousMode() {
3 evincedAppiumSdk.startAnalyze();
4
5 // appium commands
6 // ...
7
8 evincedAppiumSdk.stopAnalyze();
9}

Here is an example of using continuous mode in for an entire class:

1@BeforeClass
2public static void setup() {
3 // ...
4 evincedAppiumSdk.startAnalyze();
5}
6
7@AfterClass
8public static void tearDown() {
9 // ...
10 evincedAppiumSdk.stopAnalyze();
11}

Optional/Advanced

If you need to use your own AppiumDriver class instead of EvincedAppiumAndroidDriver or EvincedAppiumIOSDriver as base classes, you can implement CanSetOnExecuteListener interface. It should have getOnExecuteListener and setOnExecuteListener methods and trigger onExecuteListener.executed on every execution of appium commands.

So it can be implemented the next way:

1// For iOS it can be IOSDriver instead of AndroidDriver
2public class YourOwnCustomDriver<T extends WebElement> extends AndroidDriver<T> implements CanSetOnExecuteListener {
3
4 private OnExecuteListener onExecuteListener = null;
5
6 private boolean busy = false;
7
8//
9// Your overridden constructors
10// ...
11//
12
13 @Override
14 public OnExecuteListener getOnExecuteListener() {
15 return onExecuteListener;
16 }
17
18 @Override
19 public void setOnExecuteListener(OnExecuteListener onExecuteListener) {
20 this.onExecuteListener = onExecuteListener;
21 }
22
23 @Override
24 public Response execute(String command) {
25 return execute(command, ImmutableMap.of());
26 }
27
28 @Override
29 public Response execute(String driverCommand, Map<String, ?> parameters) {
30 if (onExecuteListener != null && !busy) {
31 busy = true;
32 onExecuteListener.executed(driverCommand, parameters);
33 busy = false;
34 }
35 return super.execute(driverCommand, parameters);
36 }
37}

So you can use this driver with Evinced SDK as usual.

1 YourOwnCustomDriver<AndroidElement> driver = new YourOwnCustomDriver<>(url, capabilities);
2 EvincedAppiumSdk evincedAppiumSdk = new EvincedAppiumSdk(driver);

Issue filtering

The Evinced Appium SDK supports advanced settings that allow you to apply additional configurations when generating issue reports. Simply create an instance of the EvincedConfig object with the appropriate options.

Below is an example of using EvincedConfig and IssueFilter:

Configure filters:

1final IssueFilter severityFilter = new IssueFilter()
2 .severity(Severity.NeedsReview);
3final IssueFilter issueFilter = new IssueFilter()
4 .issueType(IssueType.TappableArea, IssueType.ColorContrast);
5
6final MobileElement element = driver.findElementById(viewId);
7final IssueFilter elementFilter = new IssueFilter().matchByElements(element1);

Global filtering:

1// Create an example filter to exclude all Needs Review issues globally
2final IssueFilter globalFilter = new IssueFilter()
3 .severity(Severity.NeedsReview);
4
5// Add the IssueFilter to the creation instance of EvincedAppiumSdk
6final EvincedConfig globalEvincedConfig = new EvincedConfig()
7 .excludeFilters(globalFilter);
8final EvincedAppiumSdk evincedAppiumSdk = new EvincedAppiumSdk(driver, new InitOptions(globalEvincedConfig));

Filtering on an individual scan (analyze() or report() call):

1// Passing multiple exclude filters for an individual scan
2final IssueFilter multipleSeverityFilter = new IssueFilter()
3 .severity(Severity.NeedsReview, Severity.BestPractice);
4
5 final IssueFilter complexFilter = new IssueFilter()
6 .matchElementById("com.example:id/myElement")
7 .issueType(IssueType.CollidingControls)
8 .severity(Severity.NeedsReview);
9
10final IssueFilter complexFilter1 = new IssueFilter()
11 .matchElementByAccessibilityId("myElementA11yID")
12 .issueType(IssueType.DuplicateName, SpecialCharacters)
13 .severity(Severity.NeedsReview, BestPractice);
14final EvincedConfig evincedConfig = new EvincedConfig()
15 .excludeFilters(multipleSeverityFilter, multipleSeverityFilter, complexFilter);
16
17// Apply the configuration for a specific screen state scan
18evincedAppiumSdk.report(false, evincedConfig);
19evincedAppiumSdk.analyze(evincedConfig);

Depending on the purpose of filtering, you can use two directions -include or exclude using the appropriate methods:

  • EvincedConfig().includeFilters(filter)
  • EvincedConfig().excludeFilters(filters)

List of available issue filtering options

OptionMethodDescription
viewIdpublic IssueFilter matchElementById(String viewId)Android only, native UI element identifier: resource-id
accessibilityIdpublic IssueFilter matchElementByAccessibilityId(String a11yId)iOS only, for XCUITest it is the element's accessibility-id attribute.
mobileElementpublic IssueFilter matchByElements(MobileElement... mobileElement)
public IssueFilter matchByElements(List<MobileElement> mobileElements)
Native Appium element
severitypublic IssueFilter
severity(Severity... severity) public IssueFilter severity(List<Severity> severities)
Filter by Severity - not apply on UI elements the accessibility validations with specific Severity types.
issueTypepublic IssueFilter issueType(IssueType... issueType)
public IssueFilter issueType(List<IssueType> issueTypes)
Filter by IssueType - not apply on UI elements the accessibility validations with specific IssueType.
recursivepublic IssueFilter severity(List<Severity> severities)
public IssueFilter setRecursive(boolean recursive) Determines whether or not to include child elements in the current filter. By default - true

Config file

Along with the configuration creation options within the code, it is also possible to create filters using a JSON file. To import the file use the examples below:

1 // create EvincedConfig instance from file by specify absolute or relative file path
2 final EvincedConfig evincedConfigFromFile = EvincedConfig.fromFile("/project_root_evinced_config.json");
3
4 // create EvincedConfig instance from file by specify file name, located in resources directory
5 final EvincedConfig evincedConfigFromResources = EvincedConfig.fromResourcesFile("resources_evinced_config.json");

Example File

1{
2 "excludeFilters": [
3 {
4 "viewId": "viewId",
5 "accessibilityId": "a11yId",
6 "severities": [
7 {
8 // you can use id or name separately
9 "id": "3c68eb41-9227-481d-8eec-9dc8956b1900",
10 "name": "Needs Review"
11 }
12 ],
13 "issueTypes": [
14 {
15 // you can use id or name separately
16 "id": "e871b2e7-1b6d-47bc-96db-7f3dff4bfd5a",
17 "name": "Duplicate Name"
18 }
19 ]
20 }
21 ]
22}

Storing screenshots in reports

By default screenshots are provided in HTML reports for each scan, but for JSON-reports they are disabled by default. You can explicitly enable or disable screenshots using the InitOptions. Screenshots in JSON reports are provided in Base64 format.

Available screenshot options:

OptionDescription
InitOptions.ScreenshotOption.DisabledThe screenshot is available only in the HTML report. Disabled by default.
InitOptions.ScreenshotOption.Base64Add a screenshot to the json report and the Report object in Base64. Available using the screenshotBase64 field
InitOptions.ScreenshotOption.FileSave a screenshot as separately .jpeg file. the name of the screenshot is the same as the id of the corresponding report.
InitOptions.ScreenshotOption.BothSave the screenshot as a .jpeg file and also as Base64 in the json report. See options above

Global screenshot config:

1final InitOptions initOptions = new InitOptions(InitOptions.ScreenshotOption.Base64);
2// setup options during the SDK initialization
3final EvincedAppiumSdk evincedApiumSdk = new EvincedAppiumSdk(driver, initOptions);
4
5// change the existing options
6final InitOptions newOptions = new InitOptions(InitOptions.ScreenshotOption.Disabled);
7evincedApiumSdk.setOptions(newOptions);

Rule validation configurations

RulesConfig allows you to change the accessibility check for different rules. It is set globally and used for all subsequent reports.

Disabling the tappable area validation

This example disables validation for tappable area:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.TAPPABLE_AREA, false))
Disabling the label capitalization validation

This example disables validation for label capitalization:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.LABEL_CAPITALIZATION, false))
Disabling the color contrast validation

This example disables validation for color contrast:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.COLOR_CONTRAST, false)))
Customize color contrast options

Fuzziness factor configuration for color contrast. For example, the WCAG norm for this ratio is 4.5:1, but if your corporate colors have a ratio of 4.45:1, then this can be specified using the following options:

1final Map<String, Object> colorContrastOptions = new HashMap<>();
2colorContrastOptions.put("contrastRatioNormalText", 4.45f);
3colorContrastOptions.put("contrastRatioLargeText", 4.45f);
4colorContrastOptions.put("contrastRatioNonText", 4.45f);
5final RuleConfig colorContrastRuleConfig = new RuleConfig(IssueType.COLOR_CONTRAST, true, colorContrastOptions);
6evincedEngine.setOptions(new InitOptions().setRulesConfig(colorContrastRuleConfig));

Exporting meaningful labels

To export a JSON report of the accessibility label of images and buttons regardless of the scan results simply add the options flag:

1final InitOptions initOptions = new InitOptions();
2initOptions.putAdditionalOption("exportMeaningfulLabels", true);
3evincedEngine.setOptions(initOptions);

You should see a new JSON file created beside other report files with a default name: Evinced_Meaningful_Labels_Test_Results_11_15_2023_09_49_51.json. The structure of this JSON file is described in our Mobile Reports page.

To change the default folder destination and the file name of the meaningful labels JSON report you can use the second additional option with an absolute folder path:

1final InitOptions initOptions = new InitOptions();
2final File meaningfulLabelsFile = new File("evinced-reports", "SpecificMeaningfulLabels.json");
3initOptions.putAdditionalOption("exportMeaningfulLabels", true);
4initOptions.putAdditionalOption("exportMeaningfulLabelsPath", meaningfulLabelsFile.getAbsolutePath());
5evincedEngine.setOptions(initOptions);

Exporting elements with no issues

To export a JSON report of all scanned elements that had no issues found, simply add the additional options flag:

1final InitOptions initOptions = new InitOptions();
2initOptions.putAdditionalOption("exportNoIssuesElements", true);
3evincedEngine.setOptions(initOptions);

You should see a new JSON file created beside other report files with a default name: Evinced_Accessibility_Test_Results_No_Issues_Elements_07_25_2024_09_48_55.json.

The structure of this JSON file is described in our Mobile Reports page.

To change the default folder destination and the file name of the elements with no issues JSON report you can use another additional option with an absolute folder path:

1final InitOptions initOptions = new InitOptions();
2final File noIssuesElementsFile = new File("evinced-reports", "SpecificNoIssuesElements.json");
3initOptions.putAdditionalOption("exportNoIssuesElements", true);
4initOptions.putAdditionalOption("exportNoIssuesElementsPath", noIssuesElementsFile.getAbsolutePath());
5evincedEngine.setOptions(initOptions);

You can also obtain elements with no issues JSON report on test runtime using subreports:

1final InitOptions initOptions = new InitOptions();
2initOptions.putAdditionalOption("exportNoIssuesElements", true);
3evincedEngine.setOptions(initOptions);
4Report report = evincedAppiumSdk.report();
5NoIssuesElementsReport noIssueElementsReport = report.getSubReports().getNoIssueElements();
6List<NoIssuesElement> noIssuesElements = noIssueElementsReport.getItems();

Tappable Area

Every native mobile application features a wide variety of UI elements which are interactive via tap. Consider elements like buttons, text fields, toggles and others. Each of them should be tapped in order to become focused and to allow any further interaction. The lower the tappable area of the element (an imaginary square where user should tap) the less convenient it is to operate with the element. Platform constraints describe the width and height of minimum tappable area.

tappableAreaDimensions - the minimum size that does not trigger Tappable Area validation. By default for Android this value is 48dp, for iOS it is 44px.

For example

1final Map<String, Object> tapAreaDimension = new HashMap<>();
2tapAreaDimension.put("tappableAreaDimensions", 70);
3RuleConfig tapArea = new RuleConfig(IssueType.TappableArea, true, tapAreaDimension);
4InitOptions initOptions = new InitOptions().setRulesConfig(tapArea);
5evincedAppiumSdk.setOptions(initOptions);

Uploading reports to Evinced Platform

[Evinced Platform] allows you to seamlessly collect, organize, and visualize Evinced accessibility reports in one place. In this section, we will guide you through the key functionalities of the upload methods from Evinced Appium Java SDK, which was introduced in version 1.20.0, to the Evinced Platform. This upload method is fully compatible with the previous versions of the Evinced Espresso SDK API, and is disabled by default.

Initialize report uploading to the Evinced Platform

In order to start using [Evinced Platform] feature you need to include the PlatformConfig in the InitOptions:

1@BeforeClass
2public static void setup() {
3 ...
4 final InitOptions options = new InitOptions(
5 new InitOptions.PlatformConfig(
6 InitOptions.UploadOption.ENABLED_BY_DEFAULT
7 )
8 );
9 evincedAppiumSdk.setOptions(options);
10}

In this case, all of the reports from a test class will be uploaded to the [Evinced Platform] where you can view the accessibility issues.

Selective uploading

You can configure uploading only a specific set of reports:

1@Test
2public void test1() {
3 ...
4 final Report report1 = evincedAppiumSdk.report();
5 ...
6 final Report report2 = evincedAppiumSdk.report(PlatformUpload.ENABLED);
7}

In this case, only the second report will be uploaded to [Evinced Platform].

The same behavior applies to internally stored reports reportStored() after the analyze() calls and to the stopAnalyze() in the continuous mode:

1@Test
2public void test1() {
3 ...
4 evincedEngine.analyze();
5 List<Report> reportList = evincedAppiumSdk.reportStored(PlatformUpload.ENABLED);
6}

Or:

1@Test
2public void test1() {
3 ...
4 evincedEngine.startAnalyze();
5 // test body
6 List<Report> reportList = evincedAppiumSdk.stopAnalyze(PlatformUpload.ENABLED);
7}

Notice: that in these examples we didn't use InitOptions.PlatformConfig. It's not required to use this config argument to enable the selective report uploading. Please find more information about continuous mode here if needed.

Disable report uploading to Platform

In order to disable the upload method for all reports without changing arguments for each report(), reportStored() or stopAnalyze() methods you can use the following InitOption:

1@BeforeClass
2public static void setup() {
3 ...
4 final InitOptions options = new InitOptions(
5 new InitOptions.PlatformConfig(
6 // Setting the forceDisableReportsUpload flag to true, disable all reports uploading.
7 true
8 )
9 );
10 evincedAppiumSdk.setOptions(options);
11}
12
13@Test
14public void test1() {
15 ...
16 final Report report1 = evincedAppiumSdk.report();
17 ...
18 final Report report2 = evincedAppiumSdk.report(PlatformUpload.ENABLED);
19}

Labeling uploaded reports

To be able to distinguish between different reports on [Evinced Platform], the following pre-defined labels will be collected and added programmatically.

LabelsDescription
appNameThe display name of the target application.
appVersionFor example 1.1.3.
appBuildNumberFor example 56.
deviceManufacturerFor example - Google.
deviceModelFor example - sdk_gphone_x86_64.
deviceNameFor example - generic_x86_64_arm64.
testMethodNameThe method name of the actual test.
testCaseNameThe class name of the actual test.
deviceTypeCan be emulator or physical.
isInAppSdkConnectedShould be true in case of accessibility validation using Evinced InApp SDK inside a target application.
isInAppSdkInstalledShould be true in case of the presence of Evinced InApp SDK in the target application.
osNameShould always be Android.
osVersionThe version of the target Android.
SDKBuildNumberShould be equal to 1.20.0 or higher.
SDKBuildTypeShould always be Espresso SDK.

Customizing labels before uploading reports

To distinguish between reports from different parts of your applications or tests you can also add custom labels to the uploaded reports.

1@Test
2public void test1() {
3 ...
4 Map<String, String> labels1 = new HashMap<>();
5 labels1.put("MyCustomLabelKey1", "MyCustomLabelValue1");
6 final Report report1 = evincedAppiumSdk.report(PlatformUpload.ENABLED, labels1);
7 ...
8 Map<String, String> labels2 = new HashMap<>();
9 labels2.put("MyCustomLabelKey2", "MyCustomLabelValue2");
10 final Report report2 = evincedAppiumSdk.report(PlatformUpload.ENABLED, labels2);
11}

The custom key "MyCustomLabelKey1" and the custom value "MyCustomLabelValue1" will be attached to the first report. The custom key "MyCustomLabelKey2" and the custom value "MyCustomLabelValue2" will be attached to the second report.

Customizing labels before uploading reports for the entire test class

In order to label all reports inside your test class with a common key/value, use the addTestCaseMetadata() method:

1@BeforeClass
2public static void setup() {
3 ...
4 evincedAppiumSdk.addTestCaseMetadata("MyCustomCommonLabelKey", "MyCustomCommonLabelValue");
5 ...
6}
7
8@Test
9public void test1() {
10 ...
11 Map<String, String> labels1 = new HashMap<>();
12 labels1.put("MyCustomLabelKey1", "MyCustomLabelValue1");
13 final Report report1 = evincedAppiumSdk.report(PlatformUpload.ENABLED, labels1);
14 ...
15 Map<String, String> labels2 = new HashMap<>();
16 labels2.put("MyCustomLabelKey2", "MyCustomLabelValue2");
17 final Report report2 = evincedAppiumSdk.report(PlatformUpload.ENABLED, labels2);
18}

In this case, all reports in the test class scope will have "MyCustomCommonLabelKey"/"MyCustomCommonLabelValue". In addition, "MyCustomLabelKey1"/"MyCustomLabelValue1" will be added only to the first report and "MyCustomLabelKey2"/"MyCustomLabelValue2" will be added only to the second report.

Test assertion

If you want to interrupt the test execution in case the report is not uploaded, it's possible to set the failTestOnUploadError argument of the InitOptions.PlatformConfig:

1@BeforeClass
2public static void setup() {
3 ...
4 final InitOptions options = new InitOptions(
5 new InitOptions.PlatformConfig(
6 InitOptions.UploadOption.DISABLED_BY_DEFAULT,
7 // Enabled the failTestOnUploadError flag
8 true
9 )
10 );
11 evincedAppiumSdk.setOptions(options);
12}
13
14@Test
15public void test1() {
16 ...
17 Map<String, String> labels1 = new HashMap();
18 labels1.put("MyCustomLabelKey1", "MyCustomLabelKey1");
19 final Report report1 = evincedAppiumSdk.report(PlatformUpload.ENABLED, labels1);
20 // The test will fail in case of any uploading error, it can be an internal connection or routing problems
21 ...
22}

In the case of upload errors you should see an exception with a detailed error message that can be found in the logcat window to help understand the cause of the error and possible solutions.

List of available Platform upload options

forceDisableReportsUpload

forceDisableReportsUpload = true - the upload method is completely disabled. All report uploading will be disabled regardless of the upload argument:

1@BeforeClass
2public static void setup() {
3 ...
4 final InitOptions options = new InitOptions(
5 new InitOptions.PlatformConfig(
6 // forceDisableReportsUpload = true
7 true
8 )
9 );
10 ...
11}
12
13@Test
14public void test1() {
15 ...
16 // This report won't be uploaded to Evinced Platform, because of forceDisableReportsUpload = true
17 Report report1 = evincedAppiumSdk.reportStored(PlatformUpload.ENABLED);
18 ...
19 // These reports won't be uploaded to Evinced Platform, because of forceDisableReportsUpload = true
20 List<Report> reportList2 = evincedAppiumSdk.reportStored(PlatformUpload.ENABLED);
21 ...
22 // These reports won't be uploaded to Evinced Platform, because of forceDisableReportsUpload = true
23 List<Report> reportList3 = evincedAppiumSdk.stopAnalyze(PlatformUpload.ENABLED);
24 ...
25}

forceDisableReportsUpload = false is a default value of the upload to Platform configuration;

uploadOption

InitOptions.UploadOption.ENABLED_BY_DEFAULT - All reports will be uploaded, except for the reports with the upload argument equals to PlatformUpload.DISABLED:

1@BeforeClass
2public static void setup() {
3 ...
4 final InitOptions options = new InitOptions(
5 new InitOptions.PlatformConfig(
6 // uploadOption = InitOptions.UploadOption.ENABLED_BY_DEFAULT,
7 InitOptions.UploadOption.ENABLED_BY_DEFAULT
8 )
9 );
10 evincedAppiumSdk.setOptions(options);
11 ...
12}
13
14@Test
15public void test1() {
16 ...
17 // This report won't be uploaded to Evinced Platform
18 Report report1 = evincedAppiumSdk.reportStored(PlatformUpload.DISABLED);
19 ...
20 // These reports won't be uploaded to Evinced Platform
21 List<Report> reportList2 = evincedAppiumSdk.reportStored(PlatformUpload.DISABLED);
22 ...
23 // These reports won't be uploaded to Evinced Platform
24 List<Report> reportList3 = evincedAppiumSdk.stopAnalyze(PlatformUpload.DISABLED);
25 ...
26}

InitOptions.UploadOption.DISABLED_BY_DEFAULT - Only reports that are marked with upload argument equal to PlatformUpload.ENABLED will be uploaded:

1@BeforeClass
2public static void setup() {
3 ...
4 final InitOptions options = new InitOptions(
5 new InitOptions.PlatformConfig(
6 // uploadOption = InitOptions.UploadOption.DISABLED_BY_DEFAULT,
7 InitOptions.UploadOption.DISABLED_BY_DEFAULT
8 )
9 );
10 evincedAppiumSdk.setOptions(options);
11 ...
12}
13
14@Test
15public void test1() {
16 ...
17 // This report will be uploaded to Evinced Platform
18 Report report1 = evincedAppiumSdk.reportStored(PlatformUpload.ENABLED);
19 ...
20 // These reports will be uploaded to Evinced Platform
21 List<Report> reportList2 = evincedAppiumSdk.reportStored(PlatformUpload.ENABLED);
22 ...
23 // These reports will be uploaded to Evinced Platform
24 List<Report> reportList3 = evincedAppiumSdk.stopAnalyze(PlatformUpload.ENABLED);
25 ...
26}

failTestOnUploadError

failTestOnUploadError = true - Stop the test execution in case of an upload error;

failTestOnUploadError = false - Continues the test execution no matter of any upload errors. Default value.

The state table of the configurations is as following:

forceDisable ReportsUploaduploadOptionfailTestOn UploadErroruploadDescription
falseDISABLED_BY_DEFAULTfalsenullReport won't be uploaded.
falseDISABLED_BY_DEFAULTfalseENABLEDReport will be uploaded.
falseDISABLED_BY_DEFAULTfalseDISABLEDReport won't be uploaded.
falseENABLED_BY_DEFAULTfalsenullReport will be uploaded.
falseENABLED_BY_DEFAULTfalseENABLEDReport will be uploaded.
falseENABLED_BY_DEFAULTfalseDISABLEDReport won't be uploaded.
trueDISABLED_BY_DEFAULTfalsenullReport won't be uploaded.
trueDISABLED_BY_DEFAULTfalseENABLEDReport won't be uploaded.
trueDISABLED_BY_DEFAULTfalseDISABLEReport won't be uploaded.
trueENABLED_BY_DEFAULTfalsenullReport won't be uploaded.
trueENABLED_BY_DEFAULTfalseENABLEDReport won't be uploaded.
trueENABLED_BY_DEFAULTfalseDISABLEReport won't be uploaded.
falseDISABLED_BY_DEFAULTtruenullReport won't be uploaded.
falseDISABLED_BY_DEFAULTtrueENABLEDReport will be uploaded. Test execution will be interrupted in case of an upload error.
falseDISABLED_BY_DEFAULTtrueDISABLEReport won't be uploaded.
falseENABLED_BY_DEFAULTtruenullReport will be uploaded. Test execution will be interrupted in case of an upload error.
falseENABLED_BY_DEFAULTtrueENABLEDReport will be uploaded. Test execution will be interrupted in case of an upload error.
falseENABLED_BY_DEFAULTtrueDISABLEReport won't be uploaded.

List of available Platform upload arguments

report(PlatformUpload.ENABLED) - Upload the report or reports. Will have an effect when the InitOptions.PlatformConfig is InitOptions.UploadOption.ENABLED_BY_DEFAULT or InitOptions.UploadOption.DISABLED_BY_DEFAULT. Can be used as an argument is report(PlatformUpload.ENABLED), stopAnalyze(PlatformUpload.ENABLED) or reportStored(PlatformUpload.ENABLED) methods;

report(PlatformUpload.DISABLED) - Don't upload the report or reports. Will have an effect when the InitOptions.PlatformConfig is InitOptions.UploadOption.ENABLED_BY_DEFAULT or InitOptions.UploadOption.DISABLED_BY_DEFAULT Can be used as an argument is report(PlatformUpload.DISABLED), stopAnalyze(PlatformUpload.DISABLED) or reportStored(PlatformUpload.DISABLED) methods.

Integration Android Mobile Kit feature

Android/iOS Mobile Kit adds new types of validations for your target application.

Add the import statement to the beginning of your test file.

1import com.evinced.appium.sdk.core.models.MobileKitConfig;

Setup Mobile Kit config.

1@BeforeClass
2public static void setupAppiumDriver() throws MalformedURLException, IOException {
3 capabilities = new DesiredCapabilities();
4 // ...omitting the setting of capabilities for brevity
5
6 driver = new EvincedAppiumAndroidDriver<>(url, capabilities);
7 InitOptions initOptions = new InitOptions(InitOptions.ScreenshotOption.Both)
8 // Setting mobileKitEnabled = true
9 .setMobileKitConfig(new MobileKitConfig(true));
10
11 evincedAppiumSdk = new EvincedAppiumSdk(driver, initOptions);
12 evincedAppiumSdk.setupCredentials(<your service account ID>, <your API key>);
13}

Note: Make sure that your target application has the MobileKit dependency as described here: Android Mobile Kit Documentation.

In case you need to get a certain validation and your test relies on this validation. You can stop the test execution in case of Mobile Kit connection issues using the failOnMobileKitConnectionError flag.

1@BeforeClass
2public static void setupAppiumDriver() throws MalformedURLException, IOException {
3 capabilities = new DesiredCapabilities();
4 // ...omitting the setting of capabilities for brevity
5
6 driver = new EvincedAppiumAndroidDriver<>(url, capabilities);
7 InitOptions initOptions = new InitOptions(InitOptions.ScreenshotOption.Both)
8 // Setting mobileKitEnabled = true
9 // And failMobileKitConnectionError = true
10 .setMobileKitConfig(new MobileKitConfig(true, true));
11 evincedAppiumSdk = new EvincedAppiumSdk(driver, initOptions);
12 evincedAppiumSdk.setupCredentials(<your service account ID>, <your API key>);
13}

If you can't use the provided EvincedAppiumAndroidDriver or EvincedAppiumIOSDriver in your setup. Please, don't forget to obtain a driver back using evincedAppiumSdk.getDriver(). Due to the internal SDK logic, the driver will be recreated in case of Mobile Kit usage.

1@BeforeClass
2public static void setupAppiumDriver() throws MalformedURLException, IOException {
3 capabilities = new DesiredCapabilities();
4 // ...omitting the setting of capabilities for brevity
5
6 driver = new YourCustomDriver<>(url, capabilities);
7 InitOptions initOptions = new InitOptions(InitOptions.ScreenshotOption.Both)
8 // Setting mobileKitEnabled = true
9 .setMobileKitConfig(new MobileKitConfig(true));
10
11 evincedAppiumSdk = new EvincedAppiumSdk(driver, initOptions);
12 evincedAppiumSdk.setupCredentials(<your service account ID>, <your API key>);
13
14 driver = (AndroidDriver<AndroidElement>) evincedAppiumSdk.getDriver();
15}

Note: Make sure that your target application has the Mobile Kit dependency.

API

EvincedAppiumSdk

public EvincedAppiumSdk(AppiumDriver<?> originalDriver)

Default constructor which accepts an AppiumDriver instance for performing accessibility snapshot collection. Can be easily instantiated in the following way:

1IOSDriver driver = new IOSDriver<IOSElement>(url, capabilities);
2EvincedAppiumSdk evincedSdk = new EvincedAppiumSdk(driver);

public EvincedAppiumSdk(AppiumDriver<?> originalDriver, EvincedAppiumSdkOptions options)

Additional constructor which allows you to configure the different aspects of a11y testing via EvincedAppiumSdkOptions object. For instance, one can set a custom path for storing reporting artifacts by using the example below:

1driver = new IOSDriver<IOSElement>(url, capabilities);
2
3EvincedAppiumSdkOptions evincedSdkOptions = new EvincedAppiumSdkOptions(); // create options object
4evincedSdkOptions.setOutputDir("my-output-dir"); // set custom output directory
5
6evincedSdk = new EvincedAppiumSdk(driver, evincedSdkOptions);

public boolean setupCredentials(String serviceAccountId, String apiKey)

A method to add credentials confirming the validity of the user’s license. It accepts the following arguments:

  • serviceAccountId - Evinced service account ID
  • apiKey - Evinced API key

Returns a boolean flag indicating the success of the operation - if it is false there is something wrong with either validity of the license or with the checking infrastructure. See logs for more detailed output.

public boolean setupOfflineCredentials(String serviceAccountId, String accesstoken)

A method to add credentials confirming the validity of the user’s license with an offline token. It accepts the following arguments:

  • serviceAccountId - Evinced service account ID
  • accesstoken - Evinced access token

Returns a boolean flag indicating the success of the operation - if it is false there is something wrong with either validity of the license or with the checking infrastructure. See logs for more detailed output.

public void startAnalyze(EvincedConfig evincedConfig)

Watch for screen state changes and record all accessibility issues until stopAnalyze() is called. It works the same way as EvincedAppiumSdk#analyze() but collects snapshot on each appium command that can change application state.

Make sure that driver you set in constructor extends EvincedAppiumAndroidDriver or EvincedAppiumIOSDriver, or implements CanSetOnExecuteListener in similar way.

Available overloads: public void startAnalyze();

public List<Report> stopAnalyze(boolean shouldAssert) Stop flow analyze and returns a report with all issues found from the last startAnalyze() call to allow users to perform any advanced assertions on their own.

If it is not needed, simple assertion may be delegated directly to the SDK by supplying shouldAssert argument set to true. In this case, the SDK will automatically throw an AssertionError if there is at least one issue found.

Available overloads: public void stopAnalyze();

public Report report(shouldAssert: boolean)

Generates accessibility report for the current state of the application. Returns a Report instance to allow users to perform any advanced assertions on their own. If it is not needed, a simple assertion may be delegated directly to the SDK by supplying shouldAssert argument set to true. In this case, the SDK will automatically throw an AssertionError if there is at least one issue found. For more information regarding the Report object , please see our detailed Mobile Reports page.

This method always writes HTML and JSON files to the configured directory.

Available overloads:

public Report report();

Example usage:

1Report report = evincedSdk.report(true); // assert is performed by EvincedAppiumSdk
2
3Report reportForMyOwnAssertion = evincedSdk.report();
4assertEquals(reportForMyOwnAssertion.hasIssues(), false); // performing assertions on your own

public void analyze()

Collects an accessibility snapshot of the current application state and puts it into internal in-memory storage. This method is supposed to be used in conjunction with EvincedAppiumSdk.assertStored() for performing actual assertions against the content of this storage.

Example usage:

1 IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
2 .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
3 // we have achieved the state of the app we would like to test. Let's take a snapshot.
4 evincedAppiumSdk.analyze();

public List<Report> reportStored(shouldAssert: boolean)

Generates a list of accessibility reports, one for each accessibility snapshot collected by EvincedAppiumSdk.analyze(). Just like EvincedAppiumSdk.report() this method may either perform simple assertion on its own (throwing an exception if there is at least one report with issues) or allow users to define their own assertions on the returned Report objects. This behavior can be configured via the shouldAssert flag.

This method always writes HTML and JSON files to the configured directory. Both of the files contain combined information about all of the application states collected. For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

Available overloads:

public List<Report> reportStored();

Example usage:

1@AfterClass
2public static void tearDown() {
3 // Option 1
4 List<Report> reportsForOwnAssertion = evincedSdk.reportStored(); // making our own assertions
5 // Option 2
6 List<Report> reports = evincedSdk.reportStored(true); // delegating assertion to the SDK
7 // Option 3
8 List<Report> reportsWithIssues = reportsForOwnAssertion.stream() // Filter out only reports which contain some issues
9 .filter(report -> report.hasIssues())
10 .collect(Collectors.toList());
11 assertEquals(reportsWithIssues.size(), 0); // Making the actual assertion
12
13 driver.quit();
14}

public void clearStored()

Removes all of the accessibility snapshots previously collected by EvincedAppiumSdk.analyze() calls.

InitOptions

1public InitOptions()

Default public constructor with no arguments.

1public InitOptions(String outputDir)
2public InitOptions(String outputDir, EvincedConfig evincedConfig)

Overloaded constructors

1public InitOptions setOutputDir(String outputDir)

Method for defining a custom output directory where a user would like to store reporting artifacts (JSON and HTML files).

1public InitOptions setEvincedConfig(EvincedConfig evincedConfig)

Method for defining a EvincedConfig object as a global setting

1public InitOptions screenshotOption(ScreenshotOption screenshotOption)

Set option to save a screenshot in a output JSON report or as a separate .jpeg file.

1public InitOptions setRuleConfigs(List<RuleConfig> ruleConfigs)

Set rule configs

1public InitOptions setReportName(String reportName)

Set the output HTML/JSON report name prefix

1 public InitOptions setMobileKitConfig(MobileKitConfig mobileKitConfig)

Sets Mobile Kit feature configurations

EvincedConfig

An object that defines the options that will be used when performing accessibility validations and generating reports.

Default public constructors for creating EvincedConfig:

1public EvincedConfig()

Set exclude filters:

1public EvincedConfig excludeFilters(IssueFilter... excludeFilters)
2public EvincedConfig excludeFilters(List<IssueFilter> excludeFilters)

Set include filters:

1public EvincedConfig includeFilters(IssueFilter... includeFilters)
2public EvincedConfig includeFilters(List<IssueFilter> includeFilters)

Enable storing screenshots in JSON-report:

ScreenshotOption

Enum that allows enable or disable saving screenshots in JSON-reports and can be Base64, File, Both or Disabled.

IssueFilter

A filter object that allows you to exclude UI elements from availability reports by certain criteria

IssueFilter public methodes:

1public IssueFilter matchElementById(String... viewIds)
2public IssueFilter matchElementByAccessibilityId(String... accessibilityIds)
3public IssueFilter matchByElements(MobileElement... mobileElement)
4public IssueFilter matchByElements(List<MobileElement> mobileElements)
5public IssueFilter severity(List<Severity> severities)
6public IssueFilter severity(Severity... severity)
7public IssueFilter issueType(List<IssueType> issueTypes)
8public IssueFilter issueType(IssueType... issueType)

RuleConfig

Default constructor:

1public RuleConfig(IssueType name, boolean enabled)

IssueType - the IssueType on which this rule config should be applied enabled - allows to enable/disable specified IssueType. By default enabled = true

Tutorials

Generating a comprehensive a11y report for your application

In this tutorial, we will enhance our existing Appium UI test with the Evinced Appium SDK in order to check our application for accessibility issues. In order to get started you will need the following:

  1. All of the prerequisites for the Evinced Appium SDK should be met
  2. Evinced Appium SDK should be installed in your project

Preface - existing UI test overview

Let’s consider the following basic UI test as our starting point.

1import io.appium.java_client.MobileBy;
2import io.appium.java_client.ios.IOSDriver;
3import io.appium.java_client.ios.IOSElement;
4import io.appium.java_client.remote.MobileCapabilityType;
5
6import java.io.File;
7import java.io.IOException;
8import java.net.MalformedURLException;
9import java.net.URL;
10import org.openqa.selenium.remote.DesiredCapabilities;
11import org.junit.AfterClass;
12import org.junit.BeforeClass;
13import org.junit.Test;
14import org.openqa.selenium.support.ui.ExpectedConditions;
15import org.openqa.selenium.support.ui.WebDriverWait;
16
17public class SwiftRadioTest {
18
19 public static URL url;
20 public static DesiredCapabilities capabilities;
21 public static IOSDriver<IOSElement> driver;
22
23 @BeforeClass
24 public static void setupAppiumDriver() throws MalformedURLException, IOException {
25 final String URL_STRING = "http://127.0.0.1:4723/wd/hub";
26 url = new URL(URL_STRING);
27
28 final Class clazz = EvincedA11yValidatorTest.class;
29 final File file = new File(clazz.getResource("/SwiftRadio.app").getFile());
30
31 capabilities = new DesiredCapabilities();
32 capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "XCUITest");
33 capabilities.setCapability("useNewWDA", false);
34 capabilities.setCapability(MobileCapabilityType.NO_RESET, true);
35 capabilities.setCapability(MobileCapabilityType.DEVICE_NAME, "iPhone Simulator");
36 capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "iOS");
37 capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION, "12.0");
38 capabilities
39 .setCapability(MobileCapabilityType.APP,
40 file.getAbsolutePath());
41
42 driver = new IOSDriver<IOSElement>(url, capabilities);
43 }
44
45 @AfterClass
46 public static void tearDown() {
47 driver.quit();
48 }
49
50 @Test public void testStationsScreen() {
51 IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
52 .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
53
54 firstStationTableCell.click();
55
56 IOSElement playButton = (IOSElement) new WebDriverWait(driver, 30)
57 .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("XCUIElementTypeButton")));
58 }
59}

We wrote this test for an application called Swift Radio. It is a small open-source app for listening to various radio stations and podcasts.

The purpose of these tests is to check the functionality of the main application screen and ensure a user can successfully navigate to their choice of the radio station. For now, this test is only concerned with the functional testing of the app. However, with the help of the Evinced Appium SDK, we can also check it for accessibility issues along the way. Let’s go through this process with the following step-by-step instructions.

Step #1 - Initialize the Evinced Appium SDK

Before making any assertions against our app, we need to initialize EvincedAppiumSdk object. This object is used primarily as an entry point to all of the accessibility scanning features. Since we are going to use it primarily within our tests, the best place for its initialization will be our setupAppiumDriver method which gets executed first.

1import io.appium.java_client.MobileBy;
2import io.appium.java_client.ios.IOSDriver;
3import io.appium.java_client.ios.IOSElement;
4import io.appium.java_client.remote.MobileCapabilityType;
5
6import java.io.File;
7import java.io.IOException;
8import java.net.MalformedURLException;
9import java.net.URL;
10import org.openqa.selenium.remote.DesiredCapabilities;
11import org.junit.AfterClass;
12import org.junit.BeforeClass;
13import org.junit.Test;
14import org.openqa.selenium.support.ui.ExpectedConditions;
15import org.openqa.selenium.support.ui.WebDriverWait;
16
17public class SwiftRadioTest {
18
19 public static URL url;
20 public static DesiredCapabilities capabilities;
21 public static IOSDriver<IOSElement> driver;
22
23 public static EvincedAppiumSdk evincedAppiumSdk;
24
25 @BeforeClass
26 public static void setupAppiumDriver() throws MalformedURLException, IOException {
27 final String URL_STRING = "http://127.0.0.1:4723/wd/hub";
28 url = new URL(URL_STRING);
29
30 final Class clazz = EvincedA11yValidatorTest.class;
31 final File file = new File(clazz.getResource("/SwiftRadio.app").getFile());
32
33 capabilities = new DesiredCapabilities();
34 capabilities.setCapability(MobileCapabilityType.AUTOMATION_NAME, "XCUITest");
35 capabilities.setCapability("useNewWDA", false);
36 capabilities.setCapability(MobileCapabilityType.NO_RESET, true);
37 capabilities.setCapability(MobileCapabilityType.DEVICE_NAME, "iPhone Simulator");
38 capabilities.setCapability(MobileCapabilityType.PLATFORM_NAME, "iOS");
39 capabilities.setCapability(MobileCapabilityType.PLATFORM_VERSION, "12.0");
40 capabilities
41 .setCapability(MobileCapabilityType.APP,
42 file.getAbsolutePath());
43
44 driver = new IOSDriver<IOSElement>(url, capabilities);
45 evincedAppiumSdk = new EvincedAppiumSdk(driver);
46 evincedAppiumSdk.setupCredentials(<your service account ID>, <your API key>);
47 }
48
49 @AfterClass
50 public static void tearDown() {
51 driver.quit();
52 }
53
54 @Test
55 public void testStationsScreen() {
56 IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
57 .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
58
59 firstStationTableCell.click();
60
61 IOSElement playButton = (IOSElement) new WebDriverWait(driver, 30)
62 .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("XCUIElementTypeButton")));
63 }
64}

The EvincedAppiumSdk requires only an instance of the Appium driver. There are additional configuration options that can be included as well. More information can be found in the API section of this documentation.

Step #2 - Identify which application states you want to check for a11y issues

Once the SDK is set up, let’s try to carefully consider which states of our application are actually worth checking for accessibility issues. Let’s take a look at out UI test once again:

1@Test
2public void testStationsScreen() {
3 IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
4 .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
5 // We have reached the main application screen with list of stations. It may have some accessibility issues!
6
7 firstStationTableCell.click();
8
9 IOSElement playButton = (IOSElement) new WebDriverWait(driver, 30)
10 .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("XCUIElementTypeButton")));
11 // Here we have reached yet another screen - a concrete station view. It is probably worth checking for accessibility issues as well.
12}

So far we have identified two places in our UI test where we may need to perform accessibility assertion. Each place represents some distinct application state and each of these states may contain certain accessibility issues. So, let’s put our checks into place.

Step #3 - Setup analyzing breakpoints

The only thing we need to do now is to simply add evincedAppiumSdk.analyze() calls in places within our test which we previously identified as “interesting” ones in terms of accessibility. Here is how it should look:

1@Test
2public void testStationsScreen() {
3 IOSElement firstStationTableCell = (IOSElement) new WebDriverWait(driver, 30)
4 .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("StationTableViewCell")));
5 // We have reached the main application screen with list of stations. It may have some accessibility issues!
6 evincedAppiumSdk.analyze();
7
8 firstStationTableCell.click();
9
10 IOSElement playButton = (IOSElement) new WebDriverWait(driver, 30)
11 .until(ExpectedConditions.visibilityOfElementLocated(MobileBy.className("XCUIElementTypeButton")));
12 // Here we have reached yet another screen - a concrete station view. It is probably worth checking for accessibility issues as well.
13 evincedAppiumSdk.analyze();
14}

The analyze method collects an accessibility snapshot of the current application state and puts it into internal in-memory storage. This snapshot contains a lot of information including application screenshot, accessibility labels and dimensions of UI elements.

Now that we have gathered the information we need we are finally ready to actually find our whether our app is accessible or not.

Step #4 - Assert your screens at the end of your test suite

As our test was executed we collected a lot of accessibility snapshots via the evincedAppiumSdk.analyze() calls. We can now perform accessibility assertions at the end of our test suite. Referring back again to our initial UI test the best place for this assertion will be the method that gets invoked last - tearDown.

1@AfterClass
2public static void tearDown() {
3 List<Report> reports = evincedAppiumSdk.reportStored();
4 driver.quit();
5}

To generate the actual object representation of your accessibility report simply call the reportStored method. This way Evinced Appium SDK will check all of the previously collected accessibility snapshots for having some accessibility issues. From this moment you are free to assert the given list of Report objects in any way you want (feel free to explore the corresponding chapter in the “API“ section).

For the sake of simplicity of this tutorial let’s simply assume that our application is accessible as long as it has no accessibility issues found. Thus, if we have at least one accessibility issue detected - we want our tests to be failed. Let’s add the corresponding assertion:

1@AfterClass
2public static void tearDown() {
3 List<Report> reports = evincedAppiumSdk.reportStored();
4 List<Report> reportsWithIssues = reports.stream() // Filter out only reports which contain some issues
5 .filter(report -> report.hasIssues())
6 .collect(Collectors.toList());
7
8 assertEquals(reportsWithIssues.size(), 0); // Making the actual assertion
9 driver.quit();
10}

You are now set to run the test and ensure the accessibility of your application! So, go ahead and run it via your IDE or any other tooling you use for Java development.

Step #5 - Read the results

At this point, you will likely see that your test has failed, which is totally fine. Swift Radio certainly has some accessibility flaws, and the fact that we have caught them simply means that Evinced Appium SDK is working!

The only thing which is missing here is the actual list of accessibility issues with would like to look at. First of all, let’s pay a bit of attention to the terminal output of our tests. You should see your tests failed due to AssertionError with the following description:

1Number of a11y issues found. Please refer to the html report located at: /Users/johndoe/dev/swift-radio/evinced-reports/evinced-report-a6a47bed-d7b5-467e-a5c2-9e6ac4c1f247.html
2java.lang.AssertionError: Number of a11y issues found. Please refer to the html report located at: /Users/johndoe/dev/swift-radio/evinced-reports/evinced-report-a6a47bed-d7b5-467e-a5c2-9e6ac4c1f247.html
3 at com.evinced.a11y.validator.appium.java.core.EvincedA11yValidator.assertTreesHaveA11yIssues(EvincedA11yValidator.java:72)
4 //... omited for brevity

According to this output, we have HTML and JSON files located right in the root of our project, inside the folder named evinced-reports. For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

Step #6 - Make your app more accessible

Congratulations! Over the course of this 5-step tutorial we have managed to integrate accessibility tests right into our existing UI tests with the help of Evinced Appium SDK and moreover, we have even found several issues in the app.

This is the perfect time for us to carefully investigate all of the issues detected (with help of the knowledge base) and put our JSON files into static storage for later use. With the history of all of the results, we can achieve a lot of exciting things including tracking our progress fixing all of the accessibility issues over time, detecting regressions, and gathering additional metrics.

Feel free to continue your journey with the Evinced Appium SDK by browsing the API section.

FAQ

1. Do you have support for languages other than Java?

Not at this moment, unfortunately. We are planning to expand our tooling coverage by writing dedicated libraries for other popular UI testing languages such as JavaScript and Python.

2. Can I configure which validations to run?

Currently, all of the available validations are run. We have not seen any drastic impact on performance caused by any particular validation. In the future, we are planning to introduce more sophisticated capabilities for filtering results by different properties (e.g. severity or problem type) or even by using your own custom predicate.

3. Where can I find the release changelogs?

If you are using JetBrains IDE (Android Studio or IntelliJ IDEA): Open ‘Project' tab (⌘1) → ‘Project’ view → 'External Libraries’ section → Gradle: com.evinced.appium-sdk:<version>CHANGELOG.md