Espresso / UIAutomator SDK

The Evinced Espresso / UIAutomator SDK integrates with new or existing UI tests to automatically detect accessibility issues. With the addition of a few lines of code you can analyze your entire application to understand how it can become more accessible. At the conclusion of the tests actionable HTML and JSON reports are generated to track issues in any reporting tool.

Supported platforms / frameworks

The Espresso / UIAutomator SDK supports Android UI tests on emulators or real devices with a minimum Android API level of 21 (Android 5.0).

Get Started

Setup

1. In order to use any of the Evinced Mobile SDKs you first need to create an Evinced account. Once your account is created, you will get your apiKey and a matching serviceAccountId. Initialize the SDK with these tokens it will validate access with Evinced servers when you run your tests. If an outbound internet connection is not available in your running environment - contact us at support@evinced.com to get an offline APIKey.

2. Add the SDK as a dependency using one of the following options:

   1. Add to Gradle
      Add the Evinced package repository URL to your root build.gradle at the end of repositories:

      1allprojects {
      2     repositories {
      3         //...
      4         maven { url 'https://evinced.jfrog.io/artifactory/public-gradle' }
      5     }
      6 }

      Note: If you use Gradle 7.0 or higher, you should add it instread to the settings.gradle file

      Add the following to your module build.gradle file:

      1androidTestImplementation "com.evinced:espresso-sdk:{sdk-version}"

   2. Add to Maven:
      Add the Evinced package repository URL to your build file:

      1<repositories>
      2    <repository>
      3        <id>evinced-repository</id>
      4        <url>https://evinced.jfrog.io/artifactory/public-gradle</url>
      5    </repository>
      6</repositories>

      Add the dependency:

      1<dependency>
      2    <groupId>com.evinced</groupId>
      3    <artifactId>espresso-sdk</artifactId>
      4    <version>{sdk-version}</version>
      5    <type>aar</type>
      6</dependency>

   3. Download .aar library from Evinced package repository: Espresso/UIAutomator SDK and add it manually for your project.

3. Configure the SDK with Evinced authentication credentials
   Customizing the JUnit Runner (Recommended)

   1. Online setup by using AccountId and ApiKey

      1public class CustomJUnitRunner extends AndroidJUnitRunner {
      2     @Override
      3     public void callApplicationOnCreate(Application app) {
      4         super.callApplicationOnCreate(app);
      5         EvincedEngine.setupCredentials(accountId, apiKey);
      6     }
      7 }
      8 ```

   2. Offline setup by using AccountId and AccessToken

      1public class YourCustomJUnitRunner extends AndroidJUnitRunner {
      2    @Override
      3    public void callApplicationOnCreate(Application app) {
      4        super.callApplicationOnCreate(app);
      5        EvincedEngine.setupOfflineCredentials(accountId, accessToken)
      6    }
      7}

      Replace the existing AndroidJunitRunner in `build.gradle:

      1defaultConfig {
      2    //...
      3
      4    // testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
      5    testInstrumentationRunner 'com.xx.yy.CustomJUnitRunner'
      6}

Replace {sdk-version} with the latest SDK version that can be found here: [espresso sdk jfrog]. After creating the custom AndroidJUnitRunner you may need to sync Gradle/Maven with the project.

Authenticating within a test class

Add the setupCredentials call to your @BeforeClass method:

1@BeforeClass
2public static void setup() {
3    // Init instance of EvincedEngine in @BeforeClass method
4    evincedEngine = EvincedEngine.getInstance(InstrumentationRegistry.getInstrumentation());
5    evincedEngine.setupCredentials(accountId, apiKey);
6    // Other setup code...
7}

Your first test

Initialize the SDK instance

Init the EvincedEngine instance in each test class inside the @BeforeClass (@BeforeAll equivalent for JUnit5 runner) method:

1// import statement
2import com.evinced.test.EvincedEngine;
3
4@RunWith(AndroidJUnit4.class)
5public class UITests1 {
6    // Declare EvincedEngine instance
7    private static EvincedEngine evincedEngine;
8    // public static method annotated @BeforeClass executed only once before running all tests in the current class
9    @BeforeClass
10    public static void setup() {
11        // Init instance of EvincedEngine in @BeforeClass method
12        evincedEngine = EvincedEngine.getInstance(InstrumentationRegistry.getInstrumentation());
13        // Other setup code...
14    }
15}

Add accessibility scans to your test

Call the EvincedEngine.analyze() method at every point at which an accessibility scan would capture new areas of the application.

1@RunWith(AndroidJUnit4.class)
2public class UITests {
3    //...
4    @Test
5    public void test1() {
6        // test code...
7        // Call EvincedEngine.analyze() every time the information on the screens changes
8        // or when you want to make accessibility check
9        evincedEngine.analyze();
10        // test code...
11        evincedEngine.analyze();
12    }
13    //...
14    @Test
15    public void testN() {
16        // test code...
17        evincedEngine.analyze();
18        // test code...
19    }
20  }

Run validations and generate reports

The @AfterClass method executes only after all tests in the current class have completed. This makes it the perfect place to run our validations on data collected and stored by the EvincedEngine.analyze() calls and generate HTML and JSON reports in the emulator/device file structure.

1@RunWith(AndroidJUnit4.class)
2public class UITests {
3  //...
4  // @AfterClass method executed only once after running all thes tests in the current class are complete
5  @AfterClass
6  public static void tearDown() {
7      // Run accessibility validations for all collected data stored by the EvincedEngine.analyze()
8      // calls and generate HTML and JSON reports in the emulator/device file structure
9      evincedEngine.reportStored();
10      // Other tearDown code...
11  }
12}

Report

EvincedEngine.reportStored() or EvincedEngine.report() will generate HTML and JSON reports stored in a specific folder in the file structure of the emulator or device on which the tests were executed. The file path to the reports can be found by searching the logs using the key evinced_report_path.

Report path example:
D/evinced_report_path:/storage/self/primary/Evinced_A11Y_Test_Results_08.10.2021_11.57

Run tests locally

If the tests are being run locally reports can be downloaded by using AndroidStudio -> Device File Explorer or the following adb command:

adb pull <reports path> <local path>

Run tests on CI

If the tests are being executed using a CI tool the pipeline script below can be used to pass the Evinced reports to the CI tool reporting structure:

1// add this step after the tests run command
2- run:
3    name: get evinced reports
4    when: always
5    command: |
6      adb logcat -d > logfile.txt
7      REPORT_PATH="$(cat logfile.txt | grep evinced_report_path | awk -F " :" '{ print $NF }' | tail -n 1 | grep -o '\/.*[0-9]' )"
8      echo ${REPORT_PATH}
9      adb pull ${REPORT_PATH} evinced_report
10      zip -r evinced_report.zip evinced_report

Note: The script uses a list of logs that are cached within the emulator storage. If the number of logs is large, it is possible to lose the evinced_report_path key. In this case, please increase the buffer size with the following command adb logcat -G <size> M (default size = 2 Mb)

1//... other steps
2- android/start-emulator-and-run-tests:
3          pre-run-tests-steps:
4            - run: adb logcat -G 16M // increase log buffer size to 16 Mb
5          test-command: ./gradlew :app:connectedFreeDebugAndroidTest //tests run command

Note: If you use for test emulators with API 21 or 22, make sure that you create them with sdcard storage by adding -c 1000M option (you can read details in avdmanager documentation). You can add it in pipeline script with additional-avd-args: -c 1000M line in android/start-emulator-and-run-tests step.

Also if you run tests on emulators with API 23+, you need add next adb options to the app build.gradle. You can check for environmental variables to enable/disable these options if you run different emulators.

1android {
2  //...
3  adbOptions {
4    installOptions '-g', '-r'
5  }
6}

For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

Permissions

The following permissions are required for the Espresso SDK:

android.permission.WRITE_EXTERNAL_STORAGE (for storing HTML / JSON reports in the device file structures) and android.permission.INTERNET (if you use online setup by using EvincedEngine.setupCredentials (accountId, apiKey)). Instrumentation tests use the tested app permissions, so they need to be added to the main manifest file.

If you do not have the required permissions, you can use the following instructions to add permissions to debug build and avoid adding them to the production version.

​​Set the permissions in src/debug/AndroidManifest.xml. If the file doesn't exist, create it.

By default, AndroidTest uses debug as a BuildType, so if you define your testing permissions there, then the manifest merging process will add these permissions to your UI test build.

Below is an example of the AndroidManifest.xml file with the added permissions. Note that You don't need to include the package attribute because it will be inherited from lower priority level manifest.

1<?xml version="1.0" encoding="utf-8"?>
2<manifest xmlns:android="http://schemas.android.com/apk/res/android">
3    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
4    <uses-permission android:name="android.permission.INTERNET"/>
5
6    <!--required requestLegacyExternalStorage if you run your tests on Android 29-->
7    <application
8        android:requestLegacyExternalStorage="true"/>
9</manifest>

Continuous mode

Continuous mode allows continual scanning of the application without the need to insert individual scan calls with in the test code. Simply specify the start and end points using the following methods EvincedEngine.startAnalyze() and EvincedEngine.stopAnalyze(). Then Evinced engines will automatically scan the application after each Espresso command. All issues detected during the tests will be added to the HTML and JSON reports automatically generated with the stopAnalyze() method is called.

Track scan all changes that occur in a particular test:

1@Test
2public void yourTest() {
3    evincedEngine.startAnalyze();
4    // test body
5    evincedEngine.stopAnalyze();
6}

or scan all changes that occur in the test class:

1@BeforeClass
2public static void setup() {
3    evincedEngine.startAnalyze();
4}
5
6@AfterClass
7public static void tearDown() {
8    evincedEngine.stopAnalyze();
9}

Note: for stable operation in continuous scanning mode, please make sure that all animations on the device (emulator) are turned off.

Issue filtering

The Espresso SDK supports advanced settings that allow you to apply additional configurations when generating issue reports. Simply create an instance of the EvincedConfig object with the appropriate options. Below is an example of using EvincedConfig and IssueFilter:

Global filtering:

1// Create an example filter to exclude all Needs Review issues globally
2final IssueFilter globalFilter = new IssueFilter()
3                .severity(Severity.NeedsReview);
4
5// Add the IssueFilter to the creation instance of EvincedAppiumSdk
6final EvincedConfig globalEvincedConfig = new EvincedConfig()
7                .excludeFilters(globalFilter);
8final EvincedEngine evincedEngine = EvincedEngine.getInstance(InstrumentationRegistry.getInstrumentation(), new InitOptions(globalEvincedConfig));

Filtering on an individual scan (analyze() or report() call):

1// Passing multiple exclude filters for an individual scan
2final IssueFilter multipleSeverityFilter = new IssueFilter()
3                .severity(Severity.NeedsReview, Severity.BestPractice);
4
5 final IssueFilter complexFilter = new IssueFilter()
6                .matchElementById("com.example:id/myElement")
7                .issueType(IssueType.CollidingControls)
8                .severity(Severity.NeedsReview);
9
10final IssueFilter complexFilter1 = new IssueFilter()
11                .issueType(IssueType.DuplicateName, SpecialCharacters)
12                .severity(Severity.NeedsReview, BestPractice);
13final EvincedConfig evincedConfig = new EvincedConfig()
14                .excludeFilters(multipleSeverityFilter, multipleSeverityFilter, complexFilter);
15
16// Apply the configuration for a specific screen state scan
17evincedEngine.report(false, evincedConfig);
18evincedEngine.analyze(evincedConfig);

Depending on the purpose of filtering, you can use two directions -include or exclude using the appropriate methods:

• EvincedConfig().includeFilters(filter)
• EvincedConfig().excludeFilters(filters)

List of available issue filtering options

Filter by UIElement

Evinced Espresso SDK supports Espresso, UIAutomator and JetpackCompose test frameworks. Each of them provides its own API for finding and identifying UI elements. Identified elements can be used to build filters to include/exclude certain interface elements to perform validations on them. Building this type of filter is available in several steps:

For Espresso:

1. Add espresso extensions module to your module build.gradle file:
   androidTestImplementation 'com.evinced:espresso-ext:{version}'
2. Find the element using Espresso native capabilities:

   1final ViewInteraction viewInteraction = onView(withId(R.id.yourViewId));

3. Build Espresso SDK UIElement object:

   1final List<UIElement> uiElements = EvincedEspressoUtils.buildUIElements(viewInteraction);

4. Create filter object:

   1final IssueFilter issueFilter = new IssueFilter().matchByUIElements(uiElements);

For UIAutomator:

1. Add UIAutomator extensions module to your module build.gradle file:
   androidTestImplementation 'com.evinced:uiautomator-ext:{version}'
2. Find the element using Espresso native capabilities:

   1final UiObject uiObject = uiDevice.findObject(new UiSelector().resourceId("yourViewId"));

3. Build Espresso SDK UIElement object:

   1final List<UIElement> uiElements = EvincedUIAutomatorUtils.buildUIElements(uiObject);

4. Create filter object:

   1final IssueFilter issueFilter = new IssueFilter().matchByUIElements(uiElements);

For JetpackCompose: :

1. Add UIAutomator extensions module to your module build.gradle file:
   androidTestImplementation 'com.evinced:uiautomator-ext:{version}'
2. Find the element using Espresso native capabilities:

   1val node: SemanticsNodeInteraction = composeTestRule.onNodeWithText("Hello world!")

3. Build Espresso SDK UIElement object:

   1val uiElements = EvincedComposeUtils.buildUIElements(node)

4. Create filter object:

   1IssueFilter().matchByUIElements(uiElements)

Config file

Along with the configuration creation options within the code, it is also possible to create filters using a JSON file located in assets directory of your tests. To import the file use the examples below:

1final EvincedConfig evincedConfigFromFile = EvincedConfig.fromAssetsFile(InstrumentationRegistry.getInstrumentation(), "evinced_config.json");

1{
2    "excludeFilters": [
3        {
4            "viewId": "viewId",
5            "severities": [
6                {
7                    // you can use id or name separately
8                    "id": "3c68eb41-9227-481d-8eec-9dc8956b1900",
9                    "name": "Needs Review"
10                }
11            ],
12            "issueTypes": [
13                {
14                    // you can use id or name separately
15                    "id": "e871b2e7-1b6d-47bc-96db-7f3dff4bfd5a",
16                    "name": "Duplicate Name"
17                }
18            ]
19        }
20    ]
21}

Storing screenshots in reports

By default screenshots are provided in HTML reports for each scan, but for JSON-reports they are disabled by default. You can explicitly enable or disable screenshots using the InitOptions. Screenshots in JSON reports are provided in Base64 format.

Available screenshot options:

Global screenshot config:

1final InitOptions initOptions = new InitOptions(InitOptions.ScreenshotOption.Base64);
2// setup options during the SDK initialization
3final EvincedEngine evincedEngine = EvincedEngine.getInstance(InstrumentationRegistry.getInstrumentation(), initOptions);
4
5// change the existing options
6final InitOptions newOptions = new InitOptions(InitOptions.ScreenshotOption.Disabled);
7evincedEngine.setOptions(newOptions);

Rule validation configurations

RulesConfig allows you to change the accessibility check for different rules. It is set globally and used for all subsequent reports.

Disabling the tappable area validation

This example disables validation for tappable area:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.TAPPABLE_AREA, false))

Disabling the label capitalization validation

This example disables validation for label capitalization:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.LABEL_CAPITALIZATION, false))

Disabling the color contrast validation

This example disables validation for color contrast:

1evincedEngine.setOptions(new InitOptions(new RuleConfig(IssueType.COLOR_CONTRAST, false)))

Customize color contrast options

Fuzziness factor configuration for color contrast. For example, the WCAG norm for this ratio is 4.5:1, but if your corporate colors have a ratio of 4.45:1, then this can be specified using the following options:

1final Map<String, Object> colorContrastOptions = new HashMap<>();
2colorContrastOptions.put("contrastRatioNormalText", 4.45f);
3colorContrastOptions.put("contrastRatioLargeText", 4.45f);
4colorContrastOptions.put("contrastRatioNonText", 4.45f);
5final RuleConfig colorContrastRuleConfig = new RuleConfig(IssueType.COLOR_CONTRAST, true, colorContrastOptions);
6evincedEngine.setOptions(new InitOptions(colorContrastRuleConfig));

API

EvincedEngine.getInstance(InstrumentationRegistry.getInstrumentation())

Init the EvincedEngine instance - should be called in annotated methods @BeforeClass (@BeforeAll for JUnit5).

Example usage:

1@BeforeClass
2public static void setupEvinced() {
3    evincedEngine = EvincedEngine.getInstance(InstrumentationRegistry.getInstrumentation());
4}

void EvincedEngine.analyze()

Should be called each time you want to scan a specific screen state for accessibility violations. This method collects the current view hierarchy tree to be later analyzed when the reportStored() method is called. No return value.

Example usage:

1@Test
2public void testSwitchTheme() {
3    clickPreference(R.string.user_interface_label);
4    evincedEngine.analyze();
5}

Report EvincedEngine.report()

Report EvincedEngine.report(boolean assertIssues)

Scan the current page state for accessibility violations. A list of Report objects will be returned and used to automatically generate HTML and JSON reports in the emulator/device file structure. If assertIssues = true an assertion error will be thrown if any accessibility issues are found.

Example usage:

1@Test
2public void testSwitchTheme() {
3    clickPreference(R.string.user_interface_label);
4    evincedEngine.analyze();
5    // This will throw an assertion error if any accessibility issues are found
6    evincedEngine.report(true)
7}

List<Report> EvincedEngine.reportStored()

List<Report> EvincedEngine.reportStored(boolean assertIssues)

Run the accessibility validations using the data collected by the EvincedEngine.analyze() method calls. A list of Report objects will be returned and used to automatically generate HTML and JSON reports in the emulator/device file structure. If assertIssues = true an assertion error will be thrown if any accessibility issues are found. For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

Example usage:

1@AfterClass
2public static void collectA11yData() {
3    evincedEngine.reportStored();
4}

void EvincedEngine.clearStored()

Clear all stored data collected by EvincedEngine.analyze(). Called automatically after EvincedEngine.reportStored() method.

Example usage:

1@Test
2public void testSwitchTheme() {
3    clickPreference(R.string.user_interface_label);
4    evincedEngine.analyze();
5    // This will throw an assertion error if any accessibility issues are found
6    evincedEngine.report(true)
7    evincedEngine.clearStored();
8    // More test code...
9}

void EvincedEngine.startAnalyze()

Starts the continuous scan mode - tracks the changes that take place on the screen and under the hood and calls analyze() every time a change occurs. If for some reason a screen change is not tracked, you can add them manually by calling EvincedEngine.analyze()

1@Test
2public void yourTest() {
3    evincedEngine.startAnalyze();
4}

Or

1@BeforeClass
2public static void setup() {
3    evincedEngine.startAnalyze();
4}

List<Report> EvincedEngine.stopAnalyze()

List<Report> EvincedEngine.stopAnalyze(boolean assertIssues)()

Stops the continuous scan mode and calls reportStored() under the hood(which will generate HTML and JSON reports). Should be called after EvincedEngine.startAnalyze() to finish continuous scan mode. If assertIssues = true an assertion error will be thrown if any accessibility issues are found.

1@Test
2public void yourTest() {
3   evincedEngine.stopAnalyze();
4}

Or

1@AfterClass
2public static void tearDown() {
3    evincedEngine.stopAnalyze();
4}

InitOptions

Helper class contains a global SDK options

Default constructor:
public InitOptions()

Method for defining a EvincedConfig object as a global setting:
public InitOptions setEvincedConfig(EvincedConfig evincedConfig)

Set option to save a screenshot in a output JSON report or as a separate .png file:
public InitOptions setScreenshotOption(ScreenshotOption screenshotOption)

Set rule configs:
public InitOptions setRuleConfigs(List<RuleConfig> ruleConfigs)

Method for defining a custom output directory where a user would like to store reporting artifacts (JSON and HTML files):
public InitOptions setOutputDir(String outputDir)

Set output HTML/JSON report name prefix public InitOptions setReportName(String reportName)

EvincedConfig

An object that defines the options that will be used when performing accessibility validations and generating reports.

Default public constructors for creating EvincedConfig :

public EvincedConfig()

Set exclude filters:

public EvincedConfig excludeFilters(IssueFilter... excludeFilters)

public EvincedConfig excludeFilters(List<IssueFilter> excludeFilters)

IssueFilter

A filter object that allows you to exclude UI elements from availability reports by certain criteria

IssueFilter public methodes:

public IssueFilter matchElementById(String viewId)

public IssueFilter severity(List<Severity> severities)

public IssueFilter severity(Severity... severity)

public IssueFilter issueType(List<IssueType> issueTypes)

public IssueFilter issueType(IssueType... issueType)

RuleConfig

Default constructor:
public RuleConfig(IssueType name, boolean enabled, Map<String, Any> options)

IssueType - the IssueType on which this rule config should be applied
enabled - allows to enable/disable specified IssueType. By default enabled = true options - additional a11y validation rule options

Kotlin

While the syntax of the Evinced SDK remains the same, the Kotlin implementation varies slightly to the Java examples above. Below is an example Kotlin implementation.

1package com.example.news
2
3import androidx.compose.ui.test.*
4import androidx.compose.ui.test.junit4.createComposeRule
5import androidx.test.core.app.ApplicationProvider
6import androidx.test.ext.junit.runners.AndroidJUnit4
7import androidx.test.platform.app.InstrumentationRegistry
8import com.evinced.test.EvincedEngine
9import org.junit.*
10import org.junit.runner.RunWith
11
12@RunWith(AndroidJUnit4::class)
13class NewsTests {
14    private val evincedEngine: EvincedEngine = EvincedEngine.getInstance(InstrumentationRegistry.getInstrumentation())
15
16    @get:Rule
17    val composeTestRule = createComposeRule()
18
19    @Before
20    fun setUp() {
21        // Using targetContext as the Context of the instrumentation code
22        composeTestRule.launchJetNewsApp(ApplicationProvider.getApplicationContext())
23        EvincedEngine.setupCredentials("<Service Account ID>", "<API Key>")
24    }
25
26    @After
27    fun tearDown() {
28        evincedEngine.reportStored()
29    }
30
31    @Test
32    fun app_opensInterests() {
33        evincedEngine.analyze()
34        composeTestRule.onNodeWithContentDescription(
35            label = "Open navigation drawer",
36            useUnmergedTree = true
37        ).performClick()
38        evincedEngine.analyze()
39        composeTestRule.onNodeWithText("Interests").performClick()
40        evincedEngine.analyze()
41        composeTestRule.onNodeWithText("Topics").assertExists()
42    }
43}

Tutorials

Generating a comprehensive accessibility report for your Android application

In this tutorial, we will enhance our existing Espresso UI test with the Evinced Espresso SDK in order to check our application for accessibility issues. In order to get started you will need the following:

1. All of the prerequisites for Evinced Espresso SDK should be met
2. Evinced Espresso SDK should be installed in your project

Preface - Existing UI test overview

Let’s consider the following basic UI test as our starting point.

1@RunWith(AndroidJUnit4.class)
2public class PreferencesTest {
3
4    @Test
5    public void testSwitchTheme() {
6        final int theme = UserPreferences.getTheme();
7        int otherTheme;
8        if (theme == core.R.style.Theme_Light) {
9            otherTheme = R.string.pref_theme_title_dark;
10        } else {
11            otherTheme = R.string.pref_theme_title_light;
12        }
13        clickPreference(R.string.user_interface_label);
14        clickPreference(R.string.pref_set_theme_title);
15        onView(withText(otherTheme)).perform(click());
16        Awaitility.await().atMost(1000, MILLISECONDS)
17                .until(() -> UserPreferences.getTheme() != theme);
18    }
19}

The purpose of this test is to check that the application theme can be changed successfully. For now, this test is only concerned with the functional testing of the app. However, with the help of the Espresso / UIAutomator SDK, we can also check it for accessibility issues along the way. Let’s go through this process with the following step-by-step instructions.

Step #1 - Initialize the Evinced Espresso SDK

Before making any assertions against our app, we need to initialize EvincedEngine object. This object is used primarily as an entry point to all of the accessibility scanning features. Since we are going to use it primarily within our test, the best place for its initialization will be our @BeforeClass method which gets executed first.

1@RunWith(AndroidJUnit4.class)
2public class PreferencesTest {
3
4    @BeforeClass
5    public static void setupEvinced() {
6        evincedEngine = EvincedEngine.getInstance(InstrumentationRegistry.getInstrumentation());
7    }
8
9    @Test
10    public void testSwitchTheme() {
11        final int theme = UserPreferences.getTheme();
12        int otherTheme;
13        if (theme == core.R.style.Theme_Light) {
14            otherTheme = R.string.pref_theme_title_dark;
15        } else {
16            otherTheme = R.string.pref_theme_title_light;
17        }
18        clickPreference(R.string.user_interface_label);
19        clickPreference(R.string.pref_set_theme_title);
20        onView(withText(otherTheme)).perform(click());
21        Awaitility.await().atMost(1000, MILLISECONDS)
22                .until(() -> UserPreferences.getTheme() != theme);
23    }
24}

The only setup needed for the EvincedEngine is to pass it the instrumentation.

Step #2 - Identify which application states you want to check for accessibility issues

Once the SDK is set up, let’s try to carefully consider which states of our application are actually worth checking for accessibility issues. Let’s take a look at our UI test once again:

1@Test
2public void testSwitchTheme() {
3    final int theme = UserPreferences.getTheme();
4    int otherTheme;
5    // This step gets the current application theme and determines what the theme will be changed to
6    if (theme == core.R.style.Theme_Light) {
7        otherTheme = R.string.pref_theme_title_dark;
8    } else {
9        otherTheme = R.string.pref_theme_title_light;
10    }
11    // Before we change the theme lets check the current theme for accessibility issues!
12    clickPreference(R.string.user_interface_label);
13    clickPreference(R.string.pref_set_theme_title);
14    onView(withText(otherTheme)).perform(click());
15    Awaitility.await().atMost(1000, MILLISECONDS)
16            .until(() -> UserPreferences.getTheme() != theme);
17    //Now that we have confirmed the theme has changed let's check the new theme for accessibility issues!
18}

We have identified two places in our UI test where we would like to perform accessibility scans. Each place represents some distinct application state and each of these states may contain certain accessibility issues. So, let’s put our checks into place.

Step #3 - Setup accessibility analyzing points

The only thing we need to do now is to simply add evincedEngine.analyze(); calls to the places in the test we identified above. Here is what it should look like:

1@Test
2public void testSwitchTheme() {
3    final int theme = UserPreferences.getTheme();
4    int otherTheme;
5    // This step gets the current application theme and determines what the theme will be changed to
6    if (theme == core.R.style.Theme_Light) {
7        otherTheme = R.string.pref_theme_title_dark;
8    } else {
9        otherTheme = R.string.pref_theme_title_light;
10    }
11    // Before we change the theme lets check the current theme for accessibility issues!
12    evincedEngine.analyze();
13
14    clickPreference(R.string.user_interface_label);
15    clickPreference(R.string.pref_set_theme_title);
16    onView(withText(otherTheme)).perform(click());
17    Awaitility.await().atMost(1000, MILLISECONDS)
18            .until(() -> UserPreferences.getTheme() != theme);
19
20    //Now that we have confirmed the theme has changed let's check the new theme for accessibility issues!
21    evincedEngine.analyze();
22}

The analyze method collects an accessibility snapshot of the current application state and puts it into internal in-memory storage. This snapshot contains a lot of information including an application screenshot, accessibility labels, and dimensions of UI elements.

Now that we have gathered the information we need we are finally ready to actually find our whether our app is accessible or not.

Step #4 - Assert your screens at the end of your test suite

As our test was executed we collected a lot of accessibility snapshots via the evincedEngine.analyze() calls. We can now perform accessibility assertions at the end of our test suite. Referring back again to our initial UI test the best place for this assertion will be the method that gets invoked last - @AfterClass.

1@AfterClass
2public static void collectA11yData() {
3    evincedEngine.reportStored();
4}

To generate the actual object representation of your accessibility report simply call the reportStored method. The Evinced Espresso SDK will check all of the previously collected accessibility snapshots for accessibility issues. From this moment you are free to assert against the given list of Report objects in any way, you would like (feel free to explore the corresponding chapter in the “API“ section).

As an example, let’s simply assume that our application is accessible as long as it has no accessibility issues found. Thus, if we have at least one accessibility issue detected - we want our tests to fail. Let’s add the corresponding assertion:

1@AfterClass
2public static void collectA11yData() {
3    evincedEngine.reportStored(true);
4}

You are now set to run the test and ensure the accessibility of your application!

Step #5 - Read the results

For more information regarding the HTML and JSON reports as well as the Report object itself, please see our detailed Mobile Reports page.

FAQ

1. What is the minimum Android API version supported?

The minimum supported Android API level is 21 (Android 5.0).

2. Is there any difference in the number of accessibility validations Evinced will find when testing on an emulator vs. a real physical device?

The Evinced Espresso SDK works equally well on real devices and emulators. The number of validations doesn't depend on the device type (emulator or real device).

3. Can the Engine be used when testing on remote or cloud-based devices?

It may depend slightly on the functionality of the cloud-based device provider, but there are no inherent reasons why the Espresso / UIAutomator SDK wouldn’t work well on any device capable of running Espresso / UIAutomator tests.

4. Do I need to add anything (SDK etc.) to my app during the build process to test for accessibility violations?

The Espresso / UIAutomator SDK is integrated only in UI tests, which run as a separate application. No change or addition is needed to the build process of the application under test.

5. Can the reports be exported directly to the project directory as opposed to remaining on the device?

Unfortunately not. Because the tests are performed as a separate application, it only has access only to the emulator or device file structure. However, you can automate this process of extracting the file by using the example scripts above in your CI tool or as a Gradle task.

6. I got the following error while integrating the Espresso SDK.

1Execution failed for task ':app:processDebugAndroidTestManifest'.
2> Manifest merger failed : Attribute application@extractNativeLibs value=(false) from manifestMerger6360702180557287033.xml:36:9-42
3    is also present at [com.evinced:espresso-sdk:<version>]

Why it happened and how to fix it? The error occurred while merging the AndroidManifest of your tests and the Espresso SDK AndroidManifest. To fix it in your androidTest AndroidManifest.xml add:

1<application
2    tools:replace="android:extractNativeLibs"
3    android:extractNativeLibs="true"
4    ...>

7. Where can I find the release changelogs?

If you are using JetBrains IDE (Android Studio or IntelliJ IDEA): Open ‘Project' tab (⌘1) → ‘Project’ view → 'External Libraries’ section → Gradle: com.evinced.espresso-sdk:<version> → CHANGELOG.md

[espresso sdk jfrog] https://evinced.jfrog.io/ui/repos/tree/General/public-gradle/com/evinced/espresso-sdk