WebDriver Ruby SDK

The Evinced WebDriver Ruby SDK integrates with new or existing Selenium WebDriver tests to automatically detect accessibility issues. By adding a few lines of code to your Selenium WebDriver project, you can begin analyzing all the web pages and DOM changes to provide a dynamic view of how your site can become more accessible. As a result of the test, a rich and comprehensive report is generated to easily track issues to resolution.

Interested in seeing this in action? Contact us to get started!

Prerequisites

  • Ruby version 2.6.3 or higher
  • selenium-webdriver gem 4.0.1 or higher
  • ChromeDriver

Get Started

Installation

1gem install selenium-webdriver
2gem install evinced-rubysdk --source https://<USER>:<PASSWORD>@evinced.jfrog.io/artifactory/api/gems/private-ruby

To avoid using plain text password you can generate an encrypted password in your JFrog profile.

If there is an issue

ERROR: While executing gem ... (Gem::RemoteFetcher::FetchError)
bad response Not Found 404 please update the gem package manager
gem update --system
. We observed such an issue on gem version 3.2.3.

Your First Test

SDK Initialization

To work with WebDriver Ruby SDK you need to have authentication token. Please refer to licensing section for details.

The only command you need to add to add Evinced SDK to your code is

1require 'evinced/rubysdk'

Evinced uses the selenium-webdriver gem which will be added automatically.

All Evinced commands will be attached to the driver object upon initialization using the Selenium::WebDriver.for command.

Add Evinced accessibility checks (Single Run Mode)

This is a simple example of how to add an Evinced accessibility scan to a test. Please note the inline comments that give detail on each test step.

1#filename: evinced_rspec_demo_spec.rb
2
3require 'evinced/rubysdk'
4
5describe 'Evinced RSpec example' do
6 it 'gathers issues with ev_analyze' do
7 driver = Selenium::WebDriver.for :chrome
8 driver.get 'https://ruby-doc.org/'
9 issues = driver.ev_analyze
10 expect(issues.size).to be == 0
11 end
12end

Run this code with the command rspec evinced_rspec_demo_spec.rb

Add Evinced accessibility checks (Continuous Mode)

This is an example of how to add a continuous Evinced accessibility scan to a test. Using the ev_start and ev_stop methods, the Evinced engine will continually scan in the background capturing all DOM changes and page navigations as the test is executed. This will capture all accessibility issues as clicking on drop-downs or similar interactions reveals more of the page. The advantage of continuous mode is that no interaction with the actual test code is needed.

1#filename: evinced_rspec_demo_spec.rb
2
3require 'evinced/rubysdk'
4
5describe 'Evinced rspec example' do
6 it 'gathers issues with ev_start and ev_stop' do
7 driver = Selenium::WebDriver.for :chrome
8 driver.ev_start
9
10 url1, url2 = %w[https://ruby-doc.org/ https://ruby-doc.org/core-3.1.0/File.html]
11 driver.get url1
12 driver.get url2
13
14 driver.ev_stop
15
16 expect(Evinced::Rubysdk::OVERALL_RESULT[url1].size).to be > 0
17 expect(Evinced::Rubysdk::OVERALL_RESULT[url2].size).to be > 0
18 end
19end

Run this code with the command rspec evinced_rspec_demo_spec.rb

API

global initialization

Initializes the Evinced object within the project.

Example

1require 'evinced/rubysdk'

In this WebDriver Ruby SDK there is no specific command you need to call to initialize SDK. However, there is a mechanism to define global configuration options. For instance you can call it in BeforeAll hook. Please refer to examples that uses Evinced.global_config in configuration section

Please refer to configuration to see examples of using init with options.

ev_analyze(options={}) -> array

Scans the current page and returns a list of accessibility issues.

Note: This method is not supported if ev_start is already running.

Please refer to configuration to see examples of using init with options.

Example

1issues = @driver.ev_analyze(root_selector: '#some .selector')
2expect(issues.size).to eql 2

Return value

Array of Issue type

A Report object is returned containing accessibility issues. This is the recommended method for static page analysis. For more information regarding reports as well as the Report object itself, please refer to our detailed Web Reports page.

ev_start(options={}) -> nil

Continually watches for DOM mutations and page navigation and records all accessibility issues until the ev_stop method is called. This method is recommended for dynamic page flows.

Example

1 @driver.get 'http://example.com'
2 @driver.ev_start
3 expect(@driver.ev_stop.size).to eql 6

Return value

nil

ev_stop -> array

Stops the process of issue gathering started by ev_start command.

Example

1 @driver.get 'http://example.com'
2 @driver.ev_start
3 @driver.find_element(css: '#some.selector').click
4 @driver.find_element(xpath: '//div[@class="clickable"]').click
5 issues = @driver.ev_stop
6 expect(issues.size).to be > 0
7 expect(issues).not_to be_ev_valid

Return value

Array of Issue type

Returns an object containing recorded accessibility issues from the point in time at which the ev_start method was instantiated. For more information regarding reports as well as the returned object itself, please refer to detailed Web Reports page.

ev_save_file(destination: 'evinced-results.json', format: :json, report: nil) -> nil

Saves list of the issues in a file, with the specified format and location. For example, format could be ‘json’ or 'html. Please find detailed information on Web Reports page.

Example

1 issues = @driver.ev_analyze
2 @driver.ev_save_file destination: '/tmp/evinced.json'

Arguments

1 destination: 'evinced-results.json', # path to the file where to store results
2 format: :json, # output format, :json by default. :yaml and html are also supported
3 report: nil # the report to store. Optional. Will store the daefault report object.

Return value

nil

Aggregated Report

The aggregated report feature allows you to have a general aggregated report for the whole run (not only for one test or suite). This report will contain all the issues found by the tests where ev_start and ev_stop commands were called. It is still possible to use the ev_save_file command in any place of your code along with this Aggregated Report feature.

Example

1tmpfile = Tempfile.new('evinced-test-results').path
2Evinced.save_file destination: tmpfile, format: :html

Licensing

To work with WebDriver Ruby SDK you need to have Authentication token. There are to methods to provide the token: offline mode and online mode. The difference is if you use online mode the request to obtain the token will be sent to Evinced Licencing Server. You need to provide Service ID and Secret API key in such a case. They are available via the Evinced Product Hub.

Offline mode assumes that you already have token and can provide it directly. If an offline token is required please reach out to your account team or support@evinced.com.

We encourage to use environment variables to store credentials and read them in code

1# Offline mode
2export AUTH_SERVICE_ID=<serviceId>
3export AUTH_TOKEN=<token>
4
5# Online mode
6export AUTH_SERVICE_ID=<serviceId>
7export AUTH_SECRET=<secret>

Example

1 # Offline mode
2 config.before(:suite) do
3 token, service_id = [ ENV['AUTH_TOKEN'], ENV['AUTH_SERVICE_ID'] ]
4
5 Evinced.set_offline_credentials token: token, service_id: service_id
6 end
7
8 # OR
9 # Online mode
10 config.before(:suite) do
11 secret, service_id = [ ENV['AUTH_SECRET'], ENV['AUTH_SERVICE_ID'] ]
12
13 Evinced.set_credentials secret: secret, service_id: service_id
14 end

Configuration

The same configuration object can be used in global initialization, ev_start and ev_analyze methods but with a bit different consequences. By providing some options in global initialization method you define a global configuration for all calls of ev_start or ev_analyze methods. Please note that global configuration is not intended to be altered from tests due to it will affect all the rest tests as well as tests running in parallel threads. To alter configuration in specific tests you can provide config on the method's level, for instance with ev_start it defines local configuration for this particular session until ev_stop is called. If provided in both levels, the command's level config will override the global one.

Engines configuration

Evinced uses two separate engines when scanning for accessibility issues, one is the aXe engine and the other is the Evinced engine. By default, Evinced disables the aXe Needs Review and Best Practices issues based on customer request and due to the fact they are mostly false positives. Please note this setting when comparing issue counts directly. See an example of how to enable Needs Review and Best practices issues in our toggles section.

Configuration object type

EvincedConfig class is used as a configuration object.

Restoring default options

1Evinced::Rubysdk::EvincedConfig.restore_default

Root Selector

Instructs engine to return issues which belong to or which are nested to element with the specified selector. By default the issues for all elements from document root will be returned.

nil by default.

Example

Global

1Evinced.global_config.root_selector = "#my-top-level-selector"
2# OR
3Evinced.global_config[:rootSelector] = "#my-top-level-selector"

Local

1driver.ev_start(rootSelector: "#my-top-level-selector")

AXE Configuration

Evinced leverages Axe open-source accessibility toolkit as part of its own accessibility detection engine. The Evinced engine is able to pinpoint far more issues than Axe alone.

For the full Axe config options, see Axe Core API.

Example

1Evinced.global_config.plugins.axe[:resultTypes] = ['violations']
2
3# It'll replace default rules
4Evinced.global_config.plugins.axe[:rules] = {
5 'document-title': { enabled: false },
6 'html-has-lang': { enabled: true },
7}
8
9# It'll add/replace axe rules
10Evinced.global_config.plugins.axe.rules['document-title'] = { enabled: true }
11Evinced.global_config.plugins.axe.rules['accesskeys'] = { enabled: false }

Errors Strict Mode

When set to true it switches SDK in mode when Evinced SDK errors are thrown as runtime errors and stops current test execution. Otherwise errors are printed into console except critical ones.

false by default

Example


The feature is not yet implemented in this SDK. If you'd like to increase the priority of it, please contact our support support@evinced.com.


Engine Logging

Switches logging on for engine. Log messages from the engine will be printed to console.

Levels are: debug, info, warn, error

nil by default.

error is a default level on engine side.

1Evinced.global_config.set_engine_logging level: 'info', context: true

Reports Screenshots

The Screenshots feature allows to include screenshots to the Evinced reports. When the feature is enabled, Evinced will take screenshots of the page, highlight an element with a specific accessibility issue, and include them to the report.

screenshots_feature

false by default.

Note: The Screenshots feature may affect the test run performance.

Enabling the Screenshots feature

The method enable_screenshots should be used to switch the on/off:

1# To enable screenshots please use
2Evinced.global_config.enable_screenshots
3# OR
4Evinced.global_config.enable_screenshots true

Toggles

Toggles are feature flags which controls SDK and Engine behavior. With toggles we introduce experimental features or manage the way how accessibility issues are gathered. The list of toggles is not specified so we suggest not to rely on specific toggle name or value and use them in invetigation puprposes only.

Example enabling aXe Best Practices and Needs Review Issues

1Evinced.global_config.experimental_flags.toggles[:USE_AXE_BEST_PRACTICES] = true
2Evinced.global_config.experimental_flags.toggles[:USE_AXE_NEEDS_REVIEW] = true

Skip Validations

Skips specific validations for the given URL pattern and the selector.

SkipValidation[] by default.

Example


The feature is not yet implemented in this SDK. If you'd like to increase the priority of it, please contact our support support@evinced.com.


Knowledge Base Link overrides

This custom parameter helps to customize knowledge base links in the reports. Those links are displayed in the reports as follows:

kb_links

The knowledge base link can be overridden for every issue type ID.

Example


The feature is not yet implemented in this SDK. If you'd like to increase the priority of it, please contact our support support@evinced.com.


Shadow DOM support

Example

1# The support is off by default.
2# To switch on
3Evinced.global_config.enable_shadow_dom
4# OR
5Evinced.global_config.enable_shadow_dom true

When set to true, the accessibility tests run analysis on iframes that exist inside the page. Default is false.


The feature is not yet implemented in this SDK. If you'd like to increase the priority of it, please contact our support support@evinced.com.


Global Switch

Global Switch allows to disable or enable Evinced functionality. It could be needed, for example, while working on functional tests in your local environment or for running some CI jobs that are not intended to gather information regarding accessibility issues.

When switched off

  • ev_start and ev_save_file will be bypassed.
  • ev_stop and ev_analyze will return an empty report.
  • evValid() will never fail.

true by default.

Important! Global Switch environment variable overrides the global configuration option.

Switch on/off Evinced functionality in config

1Evinced.on false
2## OR
3Evinced.off

Switching off Evinced functionality with environment variable

1export EV_SWITCH_ON=false

Tutorials

You can find fully functional example projects on our GitHub

Step by step adding Evinced to your tests

1. Initial tests

1#filename: evinced_tutorial_spec.rb
2
3require 'selenium-webdriver'
4
5EVINCED_DEMO_SITE = 'https://demo.evinced.com/'
6ANOTHER_PAGE_ON_DEMO_SITE = 'https://demo.evinced.com/results/?what=Caravan&where=Canada&date=Mon%20Mar%2029%202021%2011:38:30%20GMT+0300%20(Israel%20Daylight%20Time)'
7
8BASE_SELECTOR = '#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container'
9SELECT_HOME_DROPDOWN = "#{BASE_SELECTOR} > div:nth-child(1) > div > div.dropdown.line"
10SELECT_WHERE_DROPDOWN = "#{BASE_SELECTOR} > div:nth-child(2) > div > div.dropdown.line"
11TINY_HOME_OPTION = "#{BASE_SELECTOR} > div:nth-child(1) > div > ul > li:nth-child(2)"
12EAST_COAST_OPTION = "#{BASE_SELECTOR} > div:nth-child(2) > div > ul > li:nth-child(3)"
13
14describe 'evinced demo site' do
15 before :all do
16 @driver = Selenium::WebDriver.for :chrome
17 Selenium::WebDriver::Wait.new(timeout: 10)
18 end
19
20 after :all do
21 @driver.quit
22 end
23
24 it 'has clickable home selector' do
25 @driver.get EVINCED_DEMO_SITE
26 @driver.find_element(css: SELECT_HOME_DROPDOWN).click
27 @driver.find_element(css: TINY_HOME_OPTION).click
28 end
29
30 it 'has clickable where selector' do
31 @driver.get EVINCED_DEMO_SITE
32 @driver.find_element(css: SELECT_WHERE_DROPDOWN).click
33 @driver.find_element(css: EAST_COAST_OPTION).click
34 end
35
36 it 'has another page' do
37 @driver.get ANOTHER_PAGE_ON_DEMO_SITE
38 expect(@driver.current_url).to eql(ANOTHER_PAGE_ON_DEMO_SITE)
39 end
40end

Run this code snipped with rspec evinced_tutorial_spec.rb

2. Add evinced SDK to your tests

1#filename: evinced_tutorial_spec.rb
2
3require 'selenium-webdriver'
4require 'evinced/rubysdk' # <---- Only one line of code is added
5
6EVINCED_DEMO_SITE = 'https://demo.evinced.com/'
7ANOTHER_PAGE_ON_DEMO_SITE = 'https://demo.evinced.com/results/?what=Caravan&where=Canada&date=Mon%20Mar%2029%202021%2011:38:30%20GMT+0300%20(Israel%20Daylight%20Time)'
8
9BASE_SELECTOR = '#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container'
10SELECT_HOME_DROPDOWN = "#{BASE_SELECTOR} > div:nth-child(1) > div > div.dropdown.line"
11SELECT_WHERE_DROPDOWN = "#{BASE_SELECTOR} > div:nth-child(2) > div > div.dropdown.line"
12TINY_HOME_OPTION = "#{BASE_SELECTOR} > div:nth-child(1) > div > ul > li:nth-child(2)"
13EAST_COAST_OPTION = "#{BASE_SELECTOR} > div:nth-child(2) > div > ul > li:nth-child(3)"
14
15describe 'evinced demo site' do
16 before :all do
17 @driver = Selenium::WebDriver.for :chrome
18 Selenium::WebDriver::Wait.new(timeout: 10)
19 end
20
21 after :all do
22 @driver.quit
23 end
24
25 it 'has clickable home selector' do
26 @driver.get EVINCED_DEMO_SITE
27 @driver.find_element(css: SELECT_HOME_DROPDOWN).click
28 @driver.find_element(css: TINY_HOME_OPTION).click
29 end
30
31 it 'has clickable where selector' do
32 @driver.get EVINCED_DEMO_SITE
33 @driver.find_element(css: SELECT_WHERE_DROPDOWN).click
34 @driver.find_element(css: EAST_COAST_OPTION).click
35 end
36
37 it 'has another page' do
38 @driver.get ANOTHER_PAGE_ON_DEMO_SITE
39 expect(@driver.current_url).to eql(ANOTHER_PAGE_ON_DEMO_SITE)
40 end
41end

Please see line 4 above. It’s the only line you need to add to your code to include the Evinced ruby SDK. Optionally, line 3 could be omitted because the Evinced SDK requires Selenium WebDriver implicitly.

3. Activate accessibility issue gathering and store results in file

1#filename: evinced_tutorial_spec.rb
2
3require 'selenium-webdriver'
4require 'evinced/rubysdk'
5
6EVINCED_DEMO_SITE = 'https://demo.evinced.com/'
7ANOTHER_PAGE_ON_DEMO_SITE = 'https://demo.evinced.com/results/?what=Caravan&where=Canada&date=Mon%20Mar%2029%202021%2011:38:30%20GMT+0300%20(Israel%20Daylight%20Time)'
8
9BASE_SELECTOR = '#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container'
10SELECT_HOME_DROPDOWN = "#{BASE_SELECTOR} > div:nth-child(1) > div > div.dropdown.line"
11SELECT_WHERE_DROPDOWN = "#{BASE_SELECTOR} > div:nth-child(2) > div > div.dropdown.line"
12TINY_HOME_OPTION = "#{BASE_SELECTOR} > div:nth-child(1) > div > ul > li:nth-child(2)"
13EAST_COAST_OPTION = "#{BASE_SELECTOR} > div:nth-child(2) > div > ul > li:nth-child(3)"
14
15describe 'evinced demo site' do
16 before :all do
17 @driver = Selenium::WebDriver.for :chrome
18 Selenium::WebDriver::Wait.new(timeout: 10)
19 end
20
21 after :all do
22 @driver.quit
23 end
24
25 before :each do
26 @driver.ev_start
27 end
28
29 after :each do
30 @driver.ev_stop
31 end
32
33 after :all do
34 @driver.ev_save_file destination: 'evinced-tutorial-results.json'
35 end
36
37 it 'has clickable home selector' do
38 @driver.get EVINCED_DEMO_SITE
39 @driver.find_element(css: SELECT_HOME_DROPDOWN).click
40 @driver.find_element(css: TINY_HOME_OPTION).click
41 end
42
43 it 'has clickable where selector' do
44 @driver.get EVINCED_DEMO_SITE
45 @driver.find_element(css: SELECT_WHERE_DROPDOWN).click
46 @driver.find_element(css: EAST_COAST_OPTION).click
47 end
48
49 it 'has another page' do
50 @driver.get ANOTHER_PAGE_ON_DEMO_SITE
51 expect(@driver.current_url).to eql(ANOTHER_PAGE_ON_DEMO_SITE)
52 end
53end

Please review the new lines from 25 till 35. This code will activate accessibility issues gathering before each test and stop it after test. You can add it into rspec_helper.rb to apply to all spec files in your test suite.

The new after :all block stores results in the file.

4. Store results in after :suite once

The previous code snippet shows how to store results after running each test in the spec file. It is possible to save a separate report for each test, but usually it is more convenient to create a single consolidated report once all the tests are complete. RSpec has an after :suite hook, however it is likely that the driver object is not initialized there. In this case the Evinced.save static method can be used.

1#filename: evinced_tutorial_spec.rb
2
3require 'selenium-webdriver'
4require 'evinced/rubysdk'
5
6EVINCED_DEMO_SITE = 'https://demo.evinced.com/'
7ANOTHER_PAGE_ON_DEMO_SITE = 'https://demo.evinced.com/results/?what=Caravan&where=Canada&date=Mon%20Mar%2029%202021%2011:38:30%20GMT+0300%20(Israel%20Daylight%20Time)'
8
9BASE_SELECTOR = '#gatsby-focus-wrapper > main > div.wrapper-banner > div.filter-container'
10SELECT_HOME_DROPDOWN = "#{BASE_SELECTOR} > div:nth-child(1) > div > div.dropdown.line"
11SELECT_WHERE_DROPDOWN = "#{BASE_SELECTOR} > div:nth-child(2) > div > div.dropdown.line"
12TINY_HOME_OPTION = "#{BASE_SELECTOR} > div:nth-child(1) > div > ul > li:nth-child(2)"
13EAST_COAST_OPTION = "#{BASE_SELECTOR} > div:nth-child(2) > div > ul > li:nth-child(3)"
14
15describe 'evinced demo site' do
16 before :all do
17 @driver = Selenium::WebDriver.for :chrome
18 Selenium::WebDriver::Wait.new(timeout: 10)
19 end
20
21 after :all do
22 @driver.quit
23 end
24
25 before :each do
26 @driver.ev_start
27 end
28
29 after :each do
30 @driver.ev_stop
31 end
32
33 it 'has clickable home selector' do
34 @driver.get EVINCED_DEMO_SITE
35 @driver.find_element(css: SELECT_HOME_DROPDOWN).click
36 @driver.find_element(css: TINY_HOME_OPTION).click
37 end
38
39 it 'has clickable where selector' do
40 @driver.get EVINCED_DEMO_SITE
41 @driver.find_element(css: SELECT_WHERE_DROPDOWN).click
42 @driver.find_element(css: EAST_COAST_OPTION).click
43 end
44
45 it 'has another page' do
46 @driver.get ANOTHER_PAGE_ON_DEMO_SITE
47 expect(@driver.current_url).to eql(ANOTHER_PAGE_ON_DEMO_SITE)
48 end
49end
50
51RSpec.configure do |config|
52 config.after :suite do
53 Evinced.save_file destination: 'evinced-tutorial-results.html', format: 'html'
54 end
55end

Support

Please feel free to reach out to support@evinced.com with any questions.

FAQ

  1. Can I configure which validations to run?

Yes, see the configuration section for details on how to configure Axe validations to your needs.

  1. Can I run tests with Evinced using cloud-based services like Sauce Labs, Perfecto, or BrowserStack?

Yes, we have tested the Evinced SDK on many of these types of cloud-based services and expect no issues.