Ask AI – Accessibility Scanners vs Manual Testing

Just like we did with our last interview with ChatGPT, this section presents each question we asked ChatGPT followed by the AI’s response.

If you prefer to jump to our summary of what ChatGPT has told us about accessibility scanners verse manual accessibility testing, feel free to jump to the “In Conclusion” section of this post. Again, just keep in mind… AI is watching.

What is a website accessibility scanner?

A website accessibility scanner is a tool that is designed to scan websites for accessibility issues. It uses automated testing to identify common accessibility issues and provide feedback on areas where the website may not be accessible to users with disabilities.

Website accessibility scanners can be useful for identifying common accessibility issues quickly and efficiently. They can help to save time and resources compared to manual testing, which can be time-consuming and costly.

However, it's important to note that website accessibility scanners are not a replacement for manual testing. While they can identify many common accessibility issues, they cannot replace the human judgment and expertise required to ensure full accessibility compliance.

It's important to note that website accessibility scanners are not a replacement for manual testing.

Some examples of website accessibility scanners include Axe, Wave, and AChecker. These tools use automated testing to scan websites for accessibility issues, providing feedback on issues such as missing alt text, inadequate color contrast, and missing form labels. They can be used to identify common accessibility issues quickly and efficiently, and provide a starting point for manual testing and remediation.

What kind of common accessibility issues can website accessibility scanners identify?

Website accessibility scanners can identify many common accessibility issues. Here are some examples:

1. Missing alternative text: Scanners can identify images that do not have alternative text, which is important for users who are blind or have low vision and rely on screen readers to navigate websites.
2. Inadequate color contrast: Scanners can identify text that does not have sufficient contrast with its background, which can make it difficult for users with low vision to read.
3. Missing form labels: Scanners can identify forms that do not have properly associated labels, which can make it difficult for users with disabilities to understand the purpose of form fields.
4. Missing language tags: Scanners can identify pages that do not have properly defined language tags, which can make it difficult for users with screen readers to understand the language of the content.
5. Inaccessible multimedia: Scanners can identify multimedia content, such as videos and audio, that does not have proper captions or transcripts, which can make it difficult for users who are deaf or hard of hearing to understand the content.
6. Inaccessible navigation: Scanners can identify navigation menus and links that are not accessible to keyboard users, which can make it difficult for users with disabilities to navigate the website.

Overall, website accessibility scanners can identify many common accessibility issues and provide valuable feedback on areas where a website may not be accessible to users with disabilities. However, it's important to remember that these scanners cannot identify all accessibility issues, and manual testing and remediation is still required to ensure full accessibility compliance.

It's important to remember that... manual testing and remediation is still required to ensure full accessibility compliance.

Is it true that an accessibility scanner can identify missing captions or transcripts for multimedia or is this something that needs manual review?

Website accessibility scanners can identify some common multimedia accessibility issues, such as the absence of alternative text or proper labeling of multimedia controls. However, automated scanners are not always reliable in identifying whether captions or transcripts exist for multimedia content.

While some scanners may be able to identify certain multimedia accessibility issues, it's always a good idea to perform manual review to ensure that captions or transcripts are available for all multimedia content. This is because automated scanners can sometimes produce false positives or false negatives, and may not be able to identify issues that require human judgment and interpretation.

Automated scanners are not always reliable in identifying whether captions or transcripts exist for multimedia content.

Manual review can ensure that captions or transcripts are accurately and completely provided, and that they meet the requirements for accessibility guidelines such as the Web Content Accessibility Guidelines (WCAG). It's important to note that accessibility guidelines can change over time, so ongoing manual review is also important to ensure that multimedia content remains accessible as guidelines evolve.

In summary, while website accessibility scanners can be useful tools for identifying some accessibility issues, manual review is still necessary to ensure that multimedia content is fully accessible to users with disabilities.

Automated scanners can sometimes produce false positives or false negatives, and may not be able to identify issues that require human judgment and interpretation.

Is it true that an accessibility scanner can identify navigation menus and links that are not accessible to keyboard only users or is this something that also requires manual testing?

Website accessibility scanners can identify some common accessibility issues related to keyboard navigation, such as missing keyboard focus indicators or improperly coded keyboard navigation. However, automated scanners are not always reliable in identifying all accessibility issues related to keyboard navigation.

While some scanners may be able to identify certain keyboard navigation accessibility issues, it's always a good idea to perform manual testing to ensure that keyboard-only users can navigate the website effectively. Manual testing can also help to identify issues that require human judgment and interpretation, such as the logical order of page content and the placement of keyboard focus indicators.

Manual testing can ensure that keyboard-only users are able to navigate the website effectively, without encountering any barriers or difficulties. It's important to note that accessibility guidelines can change over time, so ongoing manual testing is also important to ensure that the website remains accessible as guidelines evolve.

Automated scanners are not always reliable in identifying all accessibility issues related to keyboard navigation.

In summary, while website accessibility scanners can be useful tools for identifying some accessibility issues related to keyboard navigation, manual testing is still necessary to ensure that keyboard-only users can navigate the website effectively.

What kind of accessibility issues would require human judgment and expertise required to ensure accessibility compliance for a website?

While automated accessibility scanning tools can detect many common accessibility issues, there are some issues that require human judgment and expertise to ensure accessibility compliance for a website. These issues include:

1. Color contrast: Automated scanners can detect whether there is sufficient contrast between foreground and background colors, but human judgment is required to determine whether the contrast is adequate for people with low vision or color blindness.
2. Alternative text: Automated scanners can detect whether images have alternative text, but human judgment is required to ensure that the alternative text accurately describes the content of the image.
3. Logical order of content: Automated scanners can detect whether the HTML code follows a logical order, but human judgment is required to ensure that the order of content makes sense and is intuitive for users with disabilities.
4. Keyboard accessibility: While automated scanners can detect some keyboard accessibility issues, human judgment is required to ensure that keyboard-only users can navigate the website effectively and without encountering any barriers or difficulties.
5. Multimedia accessibility: Automated scanners can detect whether multimedia elements have captions or transcripts, but human judgment is required to ensure that the captions or transcripts accurately convey the content of the multimedia element.

In general, any accessibility issue that requires human interpretation or judgment may require manual testing or evaluation to ensure accessibility compliance for a website. It's important to understand that accessibility guidelines and best practices can change over time, so ongoing evaluation and testing are necessary to ensure continued compliance.

There are some issues that require human judgment and expertise to ensure accessibility compliance for a website.

Can website accessibility scanners report issues that do not impact users once the page is rendered?

Yes, website accessibility scanners can report issues that do not necessarily impact users once the page is rendered. For example, an accessibility scanner may flag an image with missing alternative text, which would not affect a user's ability to access the content of the page once it has been fully rendered. However, this issue could still impact the accessibility of the page for users who rely on screen readers or other assistive technology to access the content of the page.

In general, accessibility scanners are designed to identify any accessibility issue that could potentially impact users with disabilities, regardless of whether the issue would affect the user experience once the page is fully rendered. This is because users with disabilities may interact with web content in different ways than users without disabilities, and any accessibility issue could potentially impact their ability to access and use the content.

Therefore, it's important to address all accessibility issues identified by an accessibility scanner, even if they do not appear to impact users once the page is fully rendered. By doing so, website owners can ensure that their website is accessible to the widest possible range of users, including those with disabilities.

Website accessibility scanners can report issues that do not necessarily impact users once the page is rendered.

Which is most useful and effective when first testing a website for accessibility, a scanning tool or manual testing?

Both automated accessibility scanning tools and manual testing are important for testing website accessibility. However, each approach has its own strengths and weaknesses, and the most effective approach will depend on the specific needs and goals of the testing process.

Automated accessibility scanning tools are useful for quickly identifying common accessibility issues and providing a starting point for further testing and evaluation. They can scan a large number of pages and identify issues that would be time-consuming to find through manual testing alone. Automated scanning tools can also help ensure consistent testing across a large number of pages or a complex website.

Manual testing is important for identifying issues that may not be detected by automated scanning tools, such as issues related to color contrast, keyboard accessibility, and multimedia accessibility. Manual testing can also provide a more comprehensive understanding of the user experience and identify issues that may not be immediately apparent through automated scanning.

Therefore, the most effective approach for testing website accessibility is to use a combination of automated scanning tools and manual testing. Automated scanning tools can be used to quickly identify common accessibility issues, while manual testing can be used to identify more complex issues and ensure a comprehensive understanding of the user experience. By combining these approaches, website owners can ensure that their website is fully accessible to users with disabilities.

Manual testing can provide a more comprehensive understanding of the user experience and identify issues that may not be immediately apparent through automated scanning.

Should a scanning tool be set first or should testing a website for accessibility start with manual testing?

When testing a website for accessibility, it is generally best to start with manual testing before using scanning tools. This is because manual testing can provide a more comprehensive understanding of the user experience and identify issues that may not be immediately apparent through automated scanning.

Manual testing can also help identify issues that are not covered by automated scanning tools, such as issues related to color contrast, keyboard accessibility, and multimedia accessibility. Manual testing can also provide insights into the overall usability of the website, which can be important for ensuring that users with disabilities have a positive user experience.

After manual testing has been completed, scanning tools can be used to identify common accessibility issues and provide a starting point for further testing and evaluation. Scanning tools can scan a large number of pages and identify issues that would be time-consuming to find through manual testing alone. Automated scanning tools can also help ensure consistent testing across a large number of pages or a complex website.

Therefore, while both manual testing and scanning tools are important for testing website accessibility, it is generally best to start with manual testing before using scanning tools. This approach can provide a more comprehensive understanding of the user experience and help ensure that all accessibility issues are identified and addressed.

When testing a website for accessibility, it is generally best to start with manual testing before using scanning tools.