Something Old, Something New
A Google search for “do accessibility overlays work” will provide you with several websites that mostly put accessibility overlays/plug-ins in a negative space. This isn’t new information and probably doesn’t come as a surprise.
There are a lot of opinions about these types of solutions, and they do have their place. Unfortunately, the promises made around “making a website accessible” or “making web content compliant” are just marketing gimmicks that do not live up to the hype. When these solutions interfere with real-world assistive technology users, they become a hindrance rather than a useful tool. This is where these solution providers differ… some are blatant in their false claims and others take a more honest approach.
Over the last several years, we have been asked to look at various accessibility overlays by several clients, and their legal counsel, who were using these services . They asked us to do this because of their concern that these solutions were not making their site “compliant”. In every case, we have identified issues that show their site is not 100% compliant as promised by the solution providers. There are plenty of resources on the web that speak to this, and this is not the reason for this article.
While we have our personal opinions on accessibility overlays/plugins, when we performed these reviews, we did so hopefully with the purpose of identifying the “good” of these solutions and identifying where they can be beneficial to end users. In other words, we did our best to set aside our own prejudices as to if these solutions made things “compliant” and instead looked for how they make accessibility “better” for different user groups. As mentioned earlier, we do believe these solutions have their place… if they do not interfere with real-world assistive technology users and do not make inaccurate promises.
We want to touch on something “new” that we haven’t seen discussed previously. That is about the supporting test documentation provided by one of these providers.
Supporting Test Documentation?
What do we mean by “supporting test documentation”?
Using an accessibility overlay does not mean your site will be spared being targeted by a legal complaint. When this happens, the accessibility overlay company may provide additional supporting documentation to help prove your site is accessible or “compliant”. Because we also provide litigation services to assist the defendant’s counsel against these complaints, we have also been provided the additional support documentation provided by these accessibility overlay providers to their customers. We will focus on one specific provider's supporting documentation and will refer to them from this point forward as "The Provider".
Some of this supporting documentation provided by has been:
- Suggested Response – As defined by The Provider this is, “A suggested response phrased in standard language. This suggestion may be referred to your legal team, who will compose a professionally accurate legal response.”
- Compliance Overview – A document from The Provider that explains “down to the code level” how The Provider claims to achieve and maintain compliance. It is basically an overview of what the accessibility plug-in should do for the inaccessible content on a site targeted at the plaintiff who filed the complaint.
- Compliance Audit – This is an accessibility audit report provided by The Provider that will provide a “verdict” of the site’s compliance along with the testing results to 49 “requirements”. This appears to be the results of manual testing based on “requirements” noted such as, “Interactive elements that can be navigated using the keyboard should be surrounded by a visual outline whenever they are focused.”
Following are screen captures of the above supporting documentation to give you a visual example of what The Provider provides to its customers facing a legal complaint:
Suggested Response Image
Compliance Overview Image
Compliant Audit Cover Page Image
Compliance Audit Details Page Image
WCAG Compliant?
As you may have noticed in the The Provider's Compliance Audit samples provided, it appears that The Provider is performing some manual testing as many of their 49 “requirements” cannot be easily tested by a scanning tool, if at all.
As mentioned earlier, due to our working with clients and their legal counsel, we have had the opportunity to review a fair number of The Provider’s supporting documentation. In every case we have seen that the Compliance Audits provided by The Provider, based on their testing, always have a verdict of “Compliant!”. And because over the years we have been provided with a large sampling of these The Provider Compliance Audits we noticed that each included 49 requirements that were tested. We also noticed that those 49 requirements were never numbered in the same order. This got us thinking if each of these audit reports were testing different requirements.
To determine if the 49 requirements documented in each of these audit reports were the same or different, we took the time to enter all test results from these audit reports into a spreadsheet for further examination. What we found was that each audit report was indeed testing to the same 49 requirements but were never documented in the same order or using the same requirement number. While this makes comparing reports side-by-side difficult, our analysis did show these reports are all focused on the same tests.
Next, we wondered if those 49 requirements that were being tested fully cover WCAG 2.1 Level A/AA. So, we took the next logical step and analyzed each “requirement” to compare against the WCAG 2.1 Level A/AA Success Criteria. This was no small task as the The Provider's Compliance Audit report does not identify which success criterion their requirements are focused. What we found was that some of the requirements can be matched directly with some of the success criteria, some could be matched with multiple success criteria, and a few of those requirements had nothing to do with WCAG at all.
This still did not answer our question on if the Compliance Audits provided by The Provider are fully testing for WCAG 2.1 Level A/AA. We again turned to our spreadsheet and further analyzed the data. We were shocked at the results! We probably should not be shocked, but when you consider that the The Provider audit gives the impression it is testing against WCAG and prominently highlights in these reports, “Verdict: Compliant!”, we can say we were disappointed at the least. Why disappointing? Because as noted at the start of the article, we feel these types of solutions can benefit certain groups of users if they do not interfere with real-world assistive technology users and do not make inaccurate promises.
Verdict: NOT Compliant!
By now you may be wondering what our investigation discovered. Based on our analysis we found that the The Provider's Compliance Audit fails to test for 68% of WCAG 2.1 Level A/AA… even though their report clearly leads the reader to believe that the testing proves compliance at that level.
Here is the breakdown of what is fully covered, partially covered, and not covered as part of The Provider’s 49 requirements in their manual audit reports.
| WCAG 2.1 Level A/AA Success Criteria | Fully Covered (16%) | Partially Covered (16%) | Not Covered (68%) |
|---|---|---|---|
| 1.1.1 Non-text Content Level A | x | ||
| 1.2.1 Audio-only and Video-only (Prerecorded) - Level A | x | ||
| 1.2.2 Captions (Prerecorded) Level A | x | ||
| 1.2.3 Audio Description or Media Alternative (Prerecorded) Level A | x | ||
| 1.2.4 Captions (Live) Level AA | x | ||
| 1.2.5 Audio Description (Prerecorded) Level AA | x | ||
| 1.3.1 Info and Relationships Level A | x | ||
| 1.3.2 Meaningful Sequence Level A | x | ||
| 1.3.3 Sensory Characteristics Level A | x | ||
| 1.3.4 Orientation Level AA (Added in 2.1) | x | ||
| 1.3.5 Identify Input Purpose Level AA (Added in 2.1) | x | ||
| 1.4.1 Use of Color Level A | x | ||
| 1.4.2 Audio Control Level A | x | ||
| 1.4.3 Contrast (Minimum) Level AA | x | ||
| 1.4.4 Resize text Level AA | x | ||
| 1.4.5 Images of Text Level AA | x | ||
| 1.4.10 Reflow Level AA (Added in 2.1) | x | ||
| 1.4.11 Non-text Contrast Level AA (Added in 2.1) | x | ||
| 1.4.12 Text Spacing Level AA (Added in 2.1) | x | ||
| 1.4.13 Content on Hover or Focus Level AA (Added in 2.1) | x | ||
| 2.1.1 Keyboard Level A | x | ||
| 2.1.2 No Keyboard Trap Level A | x | ||
| 2.1.4 Character Key Shortcuts Level A (Added in 2.1) | x | ||
| 2.2.1 Timing Adjustable Level A | x | ||
| 2.2.2 Pause, Stop, Hide Level A | x | ||
| 2.3.1 Three Flashes or Below Threshold Level A | x | ||
| 2.4.1 Bypass Blocks Level A | x | ||
| 2.4.2 Page Titled Level A | x | ||
| 2.4.3 Focus Order Level A | x | ||
| 2.4.4 Link Purpose (In Context) Level A | x | ||
| 2.4.5 Multiple Ways Level AA | x | ||
| 2.4.6 Headings and Labels Level AA | x | ||
| 2.4.7 Focus Visible Level AA | x | ||
| 2.5.1 Pointer Gestures Level A (Added in 2.1) | x | ||
| 2.5.2 Pointer Cancellation Level A (Added in 2.1) | x | ||
| 2.5.3 Label in Name Level A (Added in 2.1) | x | ||
| 2.5.4 Motion Actuation Level A (Added in 2.1) | x | ||
| 3.1.1 Language of Page Level A | x | ||
| 3.1.2 Language of Parts Level AA | x | ||
| 3.2.1 On Focus Level A | x | ||
| 3.2.2 On Input Level A | x | ||
| 3.2.3 Consistent Navigation Level AA | x | ||
| 3.2.4 Consistent Identification Level AA | x | ||
| 3.3.1 Error Identification Level A | x | ||
| 3.3.2 Labels or Instructions Level A | x | ||
| 3.3.3 Error Suggestion Level AA | x | ||
| 3.3.4 Error Prevention (Legal, Financial, Data) Level AA | x | ||
| 4.1.1 Parsing Level A | x | ||
| 4.1.2 Name, Role, Value Level A | x | ||
| 4.1.3 Status Messages Level AA (Added in 2.1) | x |
Stop the Insanity!
Sadly, the marketing gimmicks and promises made around “making a website accessible” or “making web content compliant” continues at a deeper level than just making a sale for many of these accessibility overlay/plug-in providers. And this will likely continue until some type of legislation is put in place to hold companies, like The Provider, accountable for the claims they make around accessibility and compliance.
That is not to say there are similar solution providers that are honest in their marketing efforts and claims. Yet it is those that make these false claims that seem to be reaping the reward because so many are looking for a ‘quick fix’ instead of doing the right thing and creating accessible content from the beginning.
What do you think? We would love to hear your opinion. If you have one, please leave a comment below!
The Sane Way to Include Accessibility into Your Web Content Lifecycle
If you are ready to stop the insanity and take the sane approach to incorporating accessibility into your web content development lifecycle, please give us a chance to introduce you to WebAlign. The one-of-a-kind accessibility compliance resource on the market today that is flexible enough to fit into your process with minimal effort and cost.
Want to See More Content Like This?
Want the latest blog posts, videos, white papers, and announcements? Sign up for our mailing list and stay in the loop!
We're Here to Help When You're Ready
Take a deep breath. Then feel free to reach out to our team when you're ready to discuss your accessibility needs.
I’m not sure I would summarize this situation as a “sane” vs “insanity” dynamic, the issue here is people having a simplistic, reductive attitude towards accessibility. They view accessibility issues not as their basic failure to communicate but as an unfair demand being made on them, and they feel there should be a “press a button” solution that requires no effort on their part. I would describe this as unrealistic or misguided.
On the topic of overlays and legal replies, I think it’s so bizarre for these software companies to provide these canned responses which are basically meant to say “You’re wrong! Our site is accessible! This fancy testing results say we are!” because it totally confuses the priority of importance regarding WCAG and other testing software.
The point of making a website WCAG-compliant is that hopefully that will make the site compatible with assistive tech, but there is a reason why WCAG doesn’t provide direct technical guidance and why manual testing is so crucial, because automation does not catch everything.
If a user comes to you and says “Hey this part of the site isn’t accessible to me” the response should not be to rebuff but to welcome that real-time user testing and use their feedback to further test and devise a solution. I think part of the problem is trying to treat accessibility as something that can be “fixed” once and never thought about ever again.
But the reality is that accessibility is an ongoing process largely because there are many different types of assistive tech out there and many ways that users can use that assistive tech to go on websites or interact with the rest of the world.
The best thing people in any organization can do is cultivate an attitude of curiousity and desire to learn about these different technologies and discover the best way to integrate that understanding in your organization to improve your workflow/services/products to do what is reasonable within the limitations of your organization (ie budget, staff skills, equipment available, etc) and not try to make your website be some sort of super-WCAG-passing perfect site in all areas. Sometimes it’s a simple matter of just providing content in alternative formats, such as both a PDF but also live text, or both a captioned video but also a transcript of said video. That way, if a user is not able to access any one element of your site, you can offer them alternatives that are hopefully more compatible.
I completely agree with you Christine. Some do have a simplistic and reductive attitude toward accessibility. We are hoping to change that attitude and help others realize that accessibility should be part of the process in the Web Development Lifecycle and that everyone has a part to play. It is not just a developer issue. We do realize that for many, the Web Content Accessibility Guidelines (WCAG) can be difficult to grasp and apply, and the reason we created our WebAlign resource.
As for using the word “insanity”, that is more of attempt at humor than an absolute statement. In the 1990’s the phrase “Stop the Insanity” was made popular by Susan Powter. This phrase was everywhere. You may have heard some define insanity as, “Insanity Is Doing the Same Thing Over and Over Again and Expecting Different Results.” The way many organizations approach accessibility to their websites is just like that. They keep doing the same things over and over but keep expecting their site to be accessible as a result. This tells me that what they are doing isn’t working (i.e., insanity). Which is why I choose to use the term, “Stop the Insanity” and offer “The Sane Way to Include Accessibility into Your Web Content Lifecycle”.
Thank you for your comment and, as you, I am hoping that we see more people cultivate an attitude of curiosity and desire to learn about accessibility, focusing on ensuring their site is usable rather than a super-WCAG-passing perfect site. This again, is something our WebAlign resource helps achieve.