Accessibility Testing – Pay Attention to the Details!

Patience, Aptitude, & Experience

Testing website accessibility is a special skill. While almost anyone can learn about the Web Content Accessibility Guidelines (WCAG) and review a website for conformance with the Success Criteria, not everyone has the patience or aptitude to test for accessibility properly.

A skilled accessibility tester can focus all their senses on testing. They are also able to filter out details that can impact their review depending on the type of testing being performed. For example, testing with a screen reader requires listening for the right details and considering if what is being announced by the screen reader will allow a non-visual user to understand and interact with the page content.

Too often testers who are new to screen reader testing can fail to really listen to the output of the assistive technology. As a result, critical issues can be missed because their brain is adding the details of what they are not hearing announced. This could be because they have either already visually seen the page, or they are visually looking at the page while testing.

For visual testers who are new to using screen readers as a testing tool, it is important to first learn to use a screen reader without a mouse and without being able to visually see the screen. Only then can an individual begin to truly appreciate what is involved with accessibility website testing using a screen reader.

Many bad habits that accessibility testers develop can be avoided by taking steps to ensure testing results are not unknowingly skewed. It can take some time for an accessibility tester to gain the experience required to avoid this pitfall.

Diligence is Needed

Even experienced accessibility testers can fall into bad habits which will impact their testing results. Therefore, we must be diligent in making sure we stop and reassess our methods from time to time.

Consider Testing Tool Limits

One example of how we can reassess our methods is to consider our use of testing tools. We are not talking about automated accessibility scanning tools. While automated scanners are an important part of our testing toolkit, they should not be our primary means of evaluating a website for accessibility. We are talking about tools that help us manually review a website…and there are a lot of choices out there!

When using testing tools, we must keep in mind that a specific tool may not provide everything we need to properly evaluate the accessibility of a webpage. In situations like this, we may opt to use several tools to evaluate a specific topic. This means we should think about the tools we use and exactly what they are doing for us.

Overconfidence & Haste

Another bad habit a skilled tester can develop is becoming overconfident. This typically happens when looking at certain accessibility principles that can become second nature. Testing for alternative text for images is a common area where this can happen.

Testing for alternative text on images quickly becomes routine. Because of this, a tester may start to rely on automated scanning tools to identify images that have no alternative text and then never examine the images that do have alternative text. This can be true even when using a manual tool to test for alternative text on images. If an image has alternative text, we still must evaluate if the text equivalent provided is appropriate and accurate for the purpose the image serves on that page.

Even for accessibility testers who look for the accuracy and applicability of text equivalents, this effort can suffer when we get in a hurry and start to feel confident because most images on the site we have already reviewed are implemented correctly. It is then that overconfidence and haste can cause us to miss critical issues and dilute our testing results.

Example: Tool Limits and Overconfidence

Here is a real-world example of how not considering the limits of our testing tools and being in a rush can negatively impact our accessibility testing.

The following image shows a webpage where a user can redeem points earned for products. The page is being tested for WCAG Success Criterion 1.3.1 Info and Relationships. While several aspects must be reviewed to fully test for this criterion, in this example we are focused on reviewing the page for the proper use of headings.

Website product detail page with text that reads, "Let's test this page for proper use of headings against 1.3.1 Info and Relationships."

To evaluate if headings on the page are being used properly to outline the structure of the content, the HeadingsMap browser extension is being used as shown in the next image. This tool quickly gives us a view of the heading structure and will immediately alert us if any heading levels have been skipped. After reviewing a handful of pages that all apply headings properly, overconfidence can creep in, which can tempt a tester to focus only on the HeadingsMap tool output for the rest of the evaluation.

Website product detail page showing the Headings Map browser extension in use with text that reads, "Everything looks good here!".

Deciding to rely only on the HeadingsMap tool in this situation is a mistake. As demonstrated in the following image, the tester decided to use a second tool to evaluate another aspect related to the use of headings. By using the Headings Accessibility Bookmarklet, the tester can quickly identify content on the page where visual headings exist but are not coded as headings. This is a failure of Technique H42: Using h1-h6 to Identify Headings, which would be a failure of Success Criterion 1.3.1 Info and Relationships.

Website product details page with visual heading that are not coded as headings. There is text that reads, "Wait a second! What about these?!" with arrows pointing to the visual headings.

Using both the HeadingsMap extension and the Headings Accessibility Bookmarklet provides two different views for the use of headings on the page.

  • The HeadingsMap allows the tester to quickly see the hierarchy of those headings in a side panel and if any headings levels have been skipped. The HeadingsMap tool is only going to show us headings that are coded as such.
  • The Headings Accessibility Bookmarklet highlights the headings on the page itself, which allows a tester to quickly evaluate if any visual headings exist that are not coded as headings. Determining if the headings follow a proper hierarchy using this tool can be difficult on some pages, which is why it is a good idea to use both tools as part of the review.

Using both tools allows the tester to quickly assess the use of headings and covers several aspects that should be considered to fully meet WCAG when using headings to convey the information and relationship of page content. Paying attention to the details helps the tester stay focused and not take unnecessary shortcuts.

In Conclusion

It can be easy to become overconfident and get in a hurry when evaluating a website for conformance with WCAG. This is true even for seasoned accessibility testers. This is why testers need to diligently think about the limits of their testing tools, avoid being overconfident, pay attention to the details, and maintain a balance of testing quality and speed.

A great way to ensure all aspects of the WCAG Success Criteria are being accounted for and to help maintain the focus of a skilled accessibility tester is to utilize the WebAlign® resource. Find out more about WebAlign® today.

Want to See More Content Like This?

Want the latest blog posts, videos, white papers, and announcements? Sign up for our mailing list and stay in the loop!

We're Here to Help When You're Ready

Take a deep breath. Then feel free to reach out to our team when you're ready to discuss your accessibility needs.

1 comment on “Accessibility Testing – Pay Attention to the Details!

Leave a Reply

Your email address will not be published. Required fields are marked *

Click to access the login or register cheese