We'll respond shortly.
capybara-accessible is an automated accessibility testing tool that has been recently released by Case Commons, with Pivotal Labs contributions. Here’s a description of capybara-accessible, how it works, and an explanation of some of the technical decisions we’ve made.
capybara-accessible automatically runs web accessibility audits within Rails application integration test suites. It will fail on accessibility errors when set as the Capybara webdriver. The failure messages have helpful details and links to documentation on how to fix errors.
capybara-accessible began as a component of a Case Commons project that has explicit accessibility requirements. We created capybara-accessible to automate what would otherwise be an impossibly onerous QA task. It doesn’t replace usability testing, but it has made it easy to quickly catch existing accessibility errors, while also guarding against future regressions.
At first, we tried defining an Rspec matcher that ran the accessibility audits on a page when invoked. This was laborious and redundant. All pages should be accessible, so the audit should run on every page load by default. capybara-accessible supports configuring exclusion of individual rules from every audit, if necessary.
For applications with complex workflows, resources, and associations, it’s impractical to use a crawler. Even running a crawler against your generated routes would leave major gaps in coverage. A comprehensive integration test suite should already cover at least the workflows that are most important to you, set up in a state that is plausible in the real world.
The application already has an existing and extensive integration test suite that uses Capybara, so it was the most convenient option. Similarly, there are project specific requirements around using Firefox, so Selenium was the obvious choice.
There is a scarcity of existing open source tools for automated Rails accessibility testing. Case Commons has chosen to open source the project to help fill this gap.
capybara-accessible’s audits are not exhaustive. For instance, they do not test an applications support for keyboard navigation, a feature that is critical for web accessibility. Also capybara-accessible’s audits are limited to your test coverage. Finally, the accessibility failures you see initially may be just the tip of the iceberg. Integration tests often visit multiple pages in a single example, but an audit failure will interrupt the driver before it can visit those pages. Expect to see new failures surface as you uncover new content to the driver.
Not entirely. capybara-accessible may help you fix low hanging accessibility failures. However, comprehensive web accessibility requires empathy and consideration that defy simple heuristics. Hopefully, seeing accessibility failures in CI will help foster a conversation about accessibility, build empathy, and guide you and your team to make your application more useful to more people.