capybara-accessible is an automated accessibility testing tool that has been recently released by Case Commons, with Pivotal Labs contributions. Here’s a description of capybara-accessible, how it works, and an explanation of some of the technical decisions we’ve made.
What does it do?
capybara-accessible automatically runs web accessibility audits within Rails application integration test suites. It will fail on accessibility errors when set as the Capybara webdriver. The failure messages have helpful details and links to documentation on how to fix errors.
How does it work?
capybara-accessible defines a Selenium-based webdriver that acts as a drop in replacement for Selenium or Webkit in your Capybara integration tests. Every time the driver visits a page, it automatically runs assertions from the Google Accessibility Developer Tools javascript library, and raises exceptions with detailed error messages.
Why automation?
capybara-accessible began as a component of a Case Commons project that has explicit accessibility requirements. We created capybara-accessible to automate what would otherwise be an impossibly onerous QA task. It doesn’t replace usability testing, but it has made it easy to quickly catch existing accessibility errors, while also guarding against future regressions.
Why Google Accessibility Developer Tools?
Google Accessibility Developer tools has a comprehensive, open source set of javascript assertions, along with a convenient API for running audits and generating reports. For example, the failure messages you’ll see in Rspec are composed by Google’s javascript library.
Why not add explicit accessibility assertions?
At first, we tried defining an Rspec matcher that ran the accessibility audits on a page when invoked. This was laborious and redundant. All pages should be accessible, so the audit should run on every page load by default. capybara-accessible supports configuring exclusion of individual rules from every audit, if necessary.
Why integration tests? Why not a site crawler?
For applications with complex workflows, resources, and associations, it’s impractical to use a crawler. Even running a crawler against your generated routes would leave major gaps in coverage. A comprehensive integration test suite should already cover at least the workflows that are most important to you, set up in a state that is plausible in the real world.
Why Capybara? Why Selenium?
The application already has an existing and extensive integration test suite that uses Capybara, so it was the most convenient option. Similarly, there are project specific requirements around using Firefox, so Selenium was the obvious choice.
Why open source?
There is a scarcity of existing open source tools for automated Rails accessibility testing. Case Commons has chosen to open source the project to help fill this gap.
Caveats
capybara-accessible’s audits are not exhaustive. For instance, they do not test an applications support for keyboard navigation, a feature that is critical for web accessibility. Also capybara-accessible’s audits are limited to your test coverage. Finally, the accessibility failures you see initially may be just the tip of the iceberg. Integration tests often visit multiple pages in a single example, but an audit failure will interrupt the driver before it can visit those pages. Expect to see new failures surface as you uncover new content to the driver.
Problem Solved?
Not entirely. capybara-accessible may help you fix low hanging accessibility failures. However, comprehensive web accessibility requires empathy and consideration that defy simple heuristics. Hopefully, seeing accessibility failures in CI will help foster a conversation about accessibility, build empathy, and guide you and your team to make your application more useful to more people.
