Salesforce Accessibility Automation Libraries

Automated Accessibility Testing Libraries and Tools (@sa11y packages) based on axe-core providing support for Jest unit tests, WebdriverIO component/integration tests used by teams in Salesforce. However, they are not specific to Salesforce and can be used to test any UI supported by axe-core for accessibility. These libraries are designed to be flexible, customizable and reusable to support automated accessibility testing in different testing workflows from unit to integration tests.

• Docs
  • References
• Packages
  • Jest integration
  • WDIO integration
  • assertAccessible API
  • a11y results formatter
  • Preset accessibility rules
  • Minified bundle for testing in browser
  • Internal packages
    • Test utilities
    • Integration Tests
    • Common
  • Dependency graph
  • Epilogue

Docs

• Developer Guidelines
  • Refer to Quick start instructions to get started, if you are a developer looking to Contribute
• Code of Conduct
• Changelog
• LICENSE

References

• Automated Accessibility Tests with sa11y | Developer Quick Takes - YouTube
• Automated Accessibility Testing with sa11y | Salesforce Developers Blog
• Get Started with Web Accessibility | Salesforce Trailhead
• Accessibility Overview - Lightning Design System
• LWC Accessibility guide

Packages

This repo contains the following packages for automated accessibility testing:

Jest integration

• Provides a toBeAccessible() accessibility matcher for Jest
  • integrates the assertAccessible API with the Jest assertion API
• Provides an option to set up the sa11y API to be invoked automatically at the end of each test
• To add accessibility testing to your Jest tests use this package

WDIO integration

• Provides assertAccessible(), assertAccessibleSync() APIs that can be used with WebdriverIO to check accessibility of web pages rendered in browsers

assertAccessible API

• Checks DOM or HTML Element for accessibility issues and throws an error if a11y issues are found
• To add accessibility testing to your Javascript unit tests not using Jest, use this package

a11y results formatter

• Formats raw JSON output of a11y issues from axe into an easy to consume format by consolidating and cross-referencing
• Used by assert Accessible API and Jest a11y matcher
• To use axe directly and want to format the results from axe.run use this package
• A new formatter groupViolationResultsProcessor also made available to group the a11y violations per jest test case as the existing formatter would generate test failures for each violation

Preset accessibility rules

• Provides Base, Extended, Full accessibility preset rules as axe configuration
• The Base preset rule is used by default in the Jest a11y matcher and assert Accessible APIs
  • The APIs can be overridden to use the Extended or Full ruleset as necessary

Minified bundle for testing in browser

• Provides a minified version of selected @sa11y libraries to be injected into a browser (using webdriver) and executed from integration testing workflows.
• Gives WCAG SC for rulesets in addition with axe tags

Internal packages

Test utilities

• Private package providing test utilities for @sa11y packages

Integration Tests

• Private package providing integration tests for @sa11y packages

Common

• Common utilities, constants, error messages for @sa11y packages

Dependency graph

Epilogue

A wise person once said…

"Don't talk to me until I've had coffee and you've run axe"

(Image courtesy: @shleewhite, @jorycunningham)

Changelog

All notable changes to this project will be documented in this file.

4.0.0 (2022-10-20)

⚠ BREAKING CHANGES

• Explicitly supporting NodeJS 14.x and 16.x
• sa11y will no longer support Node.js 12, which is EOL since 30 April 2022

Features

• remove support for Node.js 12 (#116) (43ade48)
• semantic release (#227) (6b7a180)
• tsconfig package for tsconfig.json (#174) (5ff2fd2)
• webdriver7 (#150) (70dafc2)

Bug Fixes

• set package.json node engines (#228) (42de911)

Changelog

• 3.1.0 (2021-11-18)
  • Bug Fixes
• 3.0.2 (2021-11-02)
  • Reverts
• 3.0.1 (2021-10-21)
  • Refactor
• 3.0.0 (2021-09-16)
• 0.11.2-beta (2021-09-13)
  • WDIO
• 0.11.1-beta (2021-09-10)
  • Automatic checks
  • Results processor
  • WDIO
• 0.11.0-beta (2021-08-26)
  • BREAKING CHANGES ⚠️
  • Features
• 0.10.0-beta (2021-08-04)
  • Features
• 0.9.1-beta (2021-07-05)
  • Fix
• 0.9.0-beta (2021-06-15)
  • Features
• 0.8.0-beta (2021-03-19)
  • Features
  • BREAKING CHANGES
  • Refactor
• 0.7.0-beta (2021-02-24)
  • Features
• 0.6.0-beta (2021-01-22)
  • Features
• 0.5.0-beta (2020-12-01)
  • Features
• 0.4.1-beta (2020-11-24)
  • Bug Fixes
• 0.4.0-beta (2020-10-31)
  • Features
  • Refactor
    • 0.3.2-beta (2020-09-22)
  • Bug Fixes
  • Refactor
• 0.3.1-beta (2020-08-20)
  • Refactor
• 0.3.0-beta (2020-08-19)
  • Features
• 0.2.0-beta (2020-06-25)
  • Features
  • Bug Fixes
  • BREAKING CHANGES
• v0.1.0-alpha (2020-06-02)
  • Features
• (2020-05-29)
  • Features
• (2020-05-07)
  • Bug Fixes
  • Features
• (2020-04-07)
  • Bug Fixes
  • Features
• (2020-03-20)
  • Bug Fixes
  • Features

3.1.0 (2021-11-18)

Bug Fixes

• automatic checks file filter to be exclusion only to keep it simple
• add sa11y common pkg as dep instead of dev dep

3.0.2 (2021-11-02)

Reverts

• filtering of non-a11y results introduced in 3.0.1
  • as it is causing issues with certain CI workflows that produce no test results file

3.0.1 (2021-10-21)

Refactor

• exclude descendancy rules from Jest API
  • these are rules that might fail at unit/component level but might pass at page level
• change sa11y results processor to output only a11y failures

3.0.0 (2021-09-16)

• Moving to unified versioning of @sa11y packages instead of independent versioning for reducing complexity of managing independent versions
• No code changes, No breaking changes

0.11.2-beta (2021-09-13)

WDIO

• Fix: Replace custom polling logic to use WebdriverIO's wait util instead as it reliably fixes the race condition

0.11.1-beta (2021-09-10)

Automatic checks

• Add ability and convenience shortcut for excluding files from automatic checks (#74)

Results processor

• Move the WCAG metadata and Priority info to class name in Jest results so that downstream CI systems make use of the info accordingly
  • Remove WCAG version, Level for brevity
• Error msgs: Move help URL up in order to prevent it from getting truncated e.g. when CSS selectors are long

WDIO

• Fix race condition when loading axe using poll with timeout

0.11.0-beta (2021-08-26)

BREAKING CHANGES ⚠️

• Refactor rulesets into base, extended (#73)
  • with extended replacing recommended
  • base contains WCAG 2.1 AA rules available in axe-core
  • extended contains AAA rules, experimental rules and non-WCAG best practice rules
  • These ruleset changes could result in different test results from previous versions

Features

• Add priority and WCAG Success Criteria metadata to preset-rules
• Add config to run rules selectively by priority
• Add script to generate preset rules markdown table in Readme with provided rules metadata

0.10.0-beta (2021-08-04)

Features

• jest: add config to run automatic checks only on selected files
• preset-rules: add config to override default ruleset using environment variable

0.9.1-beta (2021-07-05)

Fix

• jest(result processor): move WCAG metadata from suite name to test name (#69)
• jest: add workaround for mock timer timeout issue for API and automatic checks (#69)
• build: move webdriverIO to dev dependency in sa11y common pkg (#68)

0.9.0-beta (2021-06-15)

Features

• jest: de-duplicate a11y failures for automatic checks (#61)
• jest: add a custom test results processor (#62)
  • create a new test suite to hold a11y errors
  • add wcag metadata to results output
  • transform a11y error details into suite, test names

0.8.0-beta (2021-03-19)

Features

• browser-lib: add support for selecting specific css selector to check for a11y
• wdio: add scope to target only given css selector for a11y checks
  • add custom browser commands for the sa11y APIs

BREAKING CHANGES

• @sa11y/wdio APIs take an WdioOptions object instead of individual parameters (Webdriver, ExceptionList)

Refactor

• refactor(test-integration): to use wdio runner rather than jest
• refactor(test-utils): remove obsolete wdio jest bridge utils
• test(wdio): refactor to common func to check a11y error
• refactor: extract common types, interfaces into common package
• build: update to axe v4.1.3
• chore: update run script for husky v6

0.7.0-beta (2021-02-24)

Features

• jest: add opts to invoke checks automatically after each test (#54)
• Update axe-core to 4.1.2 (#54)

0.6.0-beta (2021-01-22)

Features

• wdio: add exception list opt to wdio APIs (#51)
• typescript: include d.ts TypeScript definitions in packages (#52)

0.5.0-beta (2020-12-01)

Features

• preset-rules: add new rules from axe v4.1.1

0.4.1-beta (2020-11-24)

Bug Fixes

• Upgrade to axe v4.1.1 which include bug fixes from v4.1.0 and v4.1.1 (#46)

0.4.0-beta (2020-10-31)

Features

• browser-lib: add wrapper func to check a11y with opt to filter violations
• format: add an exception list filter to filter a11y violations

Refactor

• test(wdio): refactor to extract wdio standalone setup, teardown
• refactor(common): create type alias for axe results for reuse

0.3.2-beta (2020-09-22)

Bug Fixes

• preset-rules: disable preloading media in axe config to fix timeout (#41)

Refactor

• chore: update typescript and typescript/eslint plugin to v4 (#37)

0.3.1-beta (2020-08-20)

Refactor

• chore: update to the latest major release of axe

0.3.0-beta (2020-08-19)

Features

• browser-lib: add a new package for producing bundled minified version of sa11y (#32)

0.2.0-beta (2020-06-25)

Features

• wdio: add assert accessible API for WDIO (#21)

• preset-rules: add a full ruleset with all rules from axe (#21)

Bug Fixes

• format: fix a11y error message to use formatted output (#21)
• format: check explicitly for undefined to allow custom formatter (#21)
• jest: fix a11y error being undefined when used with not matcher (#21), closes #18

BREAKING CHANGES

• preset-rules: align name and behavior to follow common naming conventions in terms of preset configs (#21)
  • rename extended ruleset to recommended
    • it continues to be the default and recommended
  • rename recommended preset rule to base
  • If you are not using the preset-rules explicitly in the APIs, this change will not affect you
  • But if you are using the preset-rules explicitly in your APIs, you will have to change them
    • If you are using the recommended preset-rule rename it to base
    • If you are using the extended preset-rule rename it to recommended

v0.1.0-alpha (2020-06-02)

Features

• format: add num of issues, unicode chars, jest matcher helper (#13)
• format: sort a11y issues by impact (#13)

(2020-05-29)

Features

• jest: add jest accessibility matcher (#9)

(2020-05-07)

Bug Fixes

• preset-rules: revert object freeze on a11y config object

Features

• assert: add assert accessible api (#7)
• jest: add basic scaffolding for jest integration
• jest: add scaffolding for jest expect matcher
• jest: add toBeAccessibleWith jest a11y matcher for use with config
• test-utils: add a test utilities package

(2020-04-07)

Bug Fixes

• assert: fix stale import of a11y config
• rules: fix import paths of modules in rules package

Features

• assert: add scaffolding for assert package
• format: add basic scaffolding for format package

(2020-03-20)

Bug Fixes

• rules: add missed out rule label-content-name-mismatch

Features

• ruleset: add recommended, extended rulesets (#3)