Appium Python SDK
The Evinced Appium Python SDK integrates with new or existing Appium tests to automatically detect accessibility issues. With the addition of as few as 5 lines of code to your Appium framework, you can begin to analyze your entire application to understand how it can become more accessible. At the conclusion of the test, a rich and comprehensive HTML or JSON report is generated to track issues in any reporting tool.
Supported versions / frameworks
Evinced Appium Python SDK supports the following values for Appium’s automationName desired capability:
Older versions of automation drivers (e.g. deprecated Appium iOS driver based on UIAutomation) are not supported.
- Python 3.7 or higher.
In order to use any of the Evinced Mobile SDKs you first need to create an Evinced account. Once your account is created, you will get your API key and a matching service account ID. Pass these these credentials by calling
LicenseManager.setup_credentials(service_id, api_key) and Evinced will validate access upon test execution. If an outbound internet connection is unavailable in your running environment - contact us at firstname.lastname@example.org to get an offline token.
Install Evinced Appium SDK from the remote registry
The most basic way to install the latest package would be by pulling it from jfrog repository:
1pip install evinced-appium-sdk --extra-index-url https://evinced.jfrog.io/artifactory/api/pypi/public-python/simple/
No preliminary authentication is required.
Optional: Setting up the credentials
An additional way to provide credentials to the validator without hardcoding them into your tests would be setting up some env variables.
1set EVINCED_SERVICE_ID=uuid-type-string-here2set EVINCED_API_KEY=short-string-here3set EVINCED_TOKEN=longest-string-here <- if you were given offline access
MacOS and Linux
1export EVINCED_SERVICE_ID=uuid-type-string-here2export EVINCED_API_KEY=short-string-here3export EVINCED_TOKEN=longest-string-here <- if you were given offline access
Initialize the SDK
1import os2import pytest3from appium.options.android import UiAutomator2Options4from appium import webdriver5from evinced_appium_sdk import * # This import will provide all necessary structures for writing your solution67# Setup Evinced license8LicenseManager.setup_credentials(service_account_id, api_key)910# Prepare the target options11options = UiAutomator2Options()12options.platform_name = 'Android'13options.device_name = 'API_30'14options.app = os.path.abspath(os.path.pardir) + '/resources/app-debug.apk'15driver = webdriver.Remote("http://localhost:4723/wd/hub", options=options)1617# Create Evinced Appium default runner18with EvincedAppiumDefaultRunner(driver, None) as runner:19 report = runner.report()
Add Evinced accessibility checks
1...2# Run analysis and get the accessibility report3report = runner.report()4# Assert that there are no accessibility issues5assert report.total == 0
Regardless of the test result, the
report method will always output JSON and HTML files. You should be able to find the files by browsing the local folder called
evinced-reports. It's also possible to configure this path. Use
1with EvincedAppiumDefaultRunner(driver, None) as runner:2 runner.set_output_directory(Path("/Users/username/output"))3 runner.report()
For more information regarding the HTML and JSON reports as well as the
report object itself, please see our detailed Mobile Reports page.
Continuous mode allows continual scanning of the application without the need to insert individual scan calls with in the test code. Simply substitute
EvincedAppiumContinuesRunner and instead of calling
report_stored you need to start a
analyzation session using
runner.stop_analyze(). Evinced validator will automatically scan the application upon using appium commands
tap. All issues detected during the tests will be added to the HTML and JSON reports automatically generated with the
runner.stop_analyze() method is called.
Here is an example of using continuous mode in a single test case:
1...2driver = get_driver()34with EvincedAppiumContinuesRunner(driver, None) as runner:5 # Run analysis and get the accessibility report6 runner.start_analyze()7 # This click method will produce a dedicated report8 runner.driver.find_element(By.ID, "SomeButton").click()9 # ...10 reports = runner.stop_analyze()
Please, notice that we are using the instance of driver from the runner object
runner.driver and not the original driver, they are specifically designed to capture reports on screens upon making an actions.
Note: It's important to note that calling stop_analyze immediately after making an action could result in a race condition where appium might make a scan mid-animation after the click. This could produce inconsistent results. Currently, there's no way to prevent this from happening in the python ecosystem without a significant cost to performance. We would recommend adding a quick block process before the animation has finished. Here's an example:
1with EvincedAppiumContinuesRunner(driver, None) as runner:2 runner.start_analyze()3 runner.driver.find_element(By.ID, "SomeButton").click() # <- animation starts4 time.sleep(0.25) # <- block thread to prevent appium from scanning too early5 reports = runner.stop_analyze() # <- will produce a final report correctly
Evinced Appium SDK provides a configuration for end-users to fit the reports to their liking.
By default, screenshots are provided in HTML reports for each scan, but for JSON-reports, they are disabled by default. You can explicitly enable or disable screenshots using the
InitOptions. Screenshots in JSON reports are provided in Base64 format.
Available screenshot options:
|ScreenshotOption.disabled||The screenshot is available only in the HTML report. Disabled by default.|
|ScreenshotOption.base64||Add a screenshot to the JSON report and the Report object in Base64. Available using the screenshotBase64 field|
It is very simple to filter returned report data. There are two different filtering options: include filters and exclude filters. Include filters will filter out all the other issue types that were not defined by the filter and exclude filter will remove the issues as defined. In combination the include filter will be prioritized, meaning that if we filter out Best Practice issues from the report and include a particular button, then the report will contain only this button, but it will not have Best Practices issues.
To get the most up-to-date information of possible ways to filter out the issue the end user is encouraged to take a look at source code filtering structures.
Filtering based on Severity level
The validator will produce reports containing issues with folowing severity levels:
- Needs Review
- Best Practice
Filtering based on Issue Type
As of now, the validator will produce reports with the following Issue types:
- Accessible Name
- Color Contrast
- Interactable Role
- Accessibility Not Enabled
- Tappable Area
- Type In Label
- Label Capitalization
- Special Characters
- Sentence Like Label
- Duplicate Name
- Colliding Controls
- State In Name
- Conflicting State In Name
- Invalid Labeling Attribute
- Alternative Text
- Instruction In Name
- Focus Trap
- Primary Context Has Title
1# In this example we will filter in all needs review2report_filter_include = ReportFilter([Severity.needs_review])3ev_config = EvincedConfig(include_filters=[report_filter_include])4init_options = InitOptions(evinced_config=ev_config)56with EvincedAppiumDefaultRunner(driver, init_options) as runner:7 runner.report() # <- this will produce reports with isssues that only have NeedsReview severity level
1# In this example we will filter out all critical and serious2report_filter_exclude = ReportFilter(severity_filters=[Severity.critical, Severity.serious])3ev_config = EvincedConfig(exclude_filters=[report_filter_exclude])4init_options = InitOptions(evinced_config=ev_config)56with EvincedAppiumDefaultRunner(driver, init_options) as runner:7 runner.report() # <- the report will contain only critical serious
1# In this example we will filter out all critical and serious and other issues2# That is not Color Contrast issue type3report_filter_exclude = ReportFilter(severity_filters=[Severity.critical, Severity.serious])4report_filter_include = ReportFilter(issue_type_filters=[IssueType.color_contrast])5ev_config = EvincedConfig(exclude_filters=[report_filter_exclude])6init_options = InitOptions(evinced_config=ev_config)78with EvincedAppiumDefaultRunner(driver, init_options) as runner:9 runner.report()
RulesConfig is designed to provide configuration for some of the aspects of reporting. Most notably ColorContrast and TappableArea.
TappableArea controls the threshold for the raising of TappableArea violation. More information can be found here tappable area. For even more information please consult with WCAG guidelines.
1rule_options = TappableArea(dimensions=15.0, dimensions_on_edge=10.0)2rules_config = [RulesConfig(A11yRule.TappableAreaIsTooSmall, False, rule_options.to_dict())]3options = InitOptions(rules_config=rules_config)
ColorContrast also controls the threshold for raising color contrast issues, for different text sizes and also controls if OCR is enabled. More information is found here
1rule_options = TappableArea(disable_ocr=True, ratio_large_text=35.0)2rules_config = [RulesConfig(A11yRule.ColorContrast, False, rule_options.to_dict())]3options = InitOptions(rules_config=rules_config)