Best Practice Guide – Accessibility Testing
About these guides
We understand that while there can be common aspects, organisations work in different ways and what works for one might not fit so well with another. These guides are written as an example of what best practice might look like in your organisation, but it may be that you have to adjust what is recommended to accommodate your particular circumstances.
Similarly the guides do not include detailed technical information as this would tie them to a specific technology or set of circumstances.Instead the guides convey important principles and approaches that can be applied in any industry and using any technology.Where appropriate the guides reference other sites and resources which contain more technical detail at the time of publication/last review.
Introduction
Accessibility testing is a critical part of your organisation’s roll-out of new software, along with upgrades and updates to existing software. Every IT department is aware of the benefits of testing software before implementation and this guide will outline some of the best practice, the different options available to you, and whether automated tools are as accurate as manual testing.
Authors: Lucy Ruck, BDF
Contributors: Paul Smyth, Barclays, Neil Milliken, Atos, Chris Felton, DWP, Will Houston, Enterprise-Rent-A-Car, Pete Hall, Cisco, Neil Eustice, KPMG
Editors: Lucy Ruck and Bela Gor
What is accessibility testing?
According to W3C (World Wide Web Consortium):
“Web accessibility testing is a subset of usability testing where the users under consideration have disabilities that affect how they use the web. The end goal, in both usability and accessibility, is to discover how easily people can use a web site and feed that information back into improving future designs and implementations.”
In essence, it’s testing to ensure that all your users/customers have access to your systems, regardless of a disability or impairment. Any technology department will be aware of the benefits of testing new applications before rolling out onto a network/system, and accessibility testing is no different. If testing is not carried out, a large number of your users and customers may find your website/systems inaccessible to them.
Are there automated tools that I can use?
There are many automated tools available to you, many of them are free to use. These have a limited application and should be used with caution (see manual testing).
Automated tools that have been used include this comprehensive list of automated accessibility testing tools for web accessibility from the W3C. https://www.w3.org/WAI/ER/tools/ as well from The Paciello Group https://www.paciellogroup.com/resources/
The advantage of using automated tools means that it can be easier to identify any issues quite early in the testing process, and pick up any of the major bugs and issues. They are also quick and easy to use, so as soon as a screen is drafted or changed it can be tested ‘little and often.’
Manual testing
Manual testing is always the preferred option of accessibility experts for a number of reasons. Many of the automated tools follow WCAG guidance, but there is no substitute for human intelligence. For example, the automated tool may test to see if a picture/photo/graphic has a label on it, to ensure that it can be used by screenreader software, but it would be unable to identify if the label was accurate or correct. The two options do complement each other, and automated tools have their uses, but they are only recommended as a starting point, rather than a full assessment.
Testing also needs to include a range of Assistive Technology (AT) to ensure it is compatible with a wide range of software, and therefore, supports the largest number of individuals. To find out more about the AT you may want to test against, please see our AT catalogue guide.
In the UK, the Government Digital Services (GDS) carried out an excellent piece of research outlining the benefits of user testing vs. automated testing. During their research, they created the ‘world’s most inaccessible website’ and then tested it. You can see more about their findings on the GDS Accessibility blog.
What options are there for manual testing?
Organisations have different approaches for manual testing depending on a number of factors. There are businesses that specialise in accessibility testing who will come into your organisation and test your applications, websites or software. They are generally very effective, but there is a cost associated.
Members and Partners of Business Disability Forum are able to utilise the Access Pathways service, which is a three-step process designed to ensure that websites and mobile applications meet best practice and international standards for accessibility. Further details can be found on the BDF’s Access Pathways web page. The Access Pathway can also be used for non-members, but there will be an upfront charge. Further details can be found on the website.
Some organisations have internal accessibility teams who can carry out the testing. The advantage of this is that they are potentially already familiar with the software they are testing, along with the AT used within the organisation. Some of these people may have disabilities themselves, although this is generally the minority.
Other organisations utilise their staff disability networks and AT user groups, although this is purely optional as it is often in addition to their day job. These individuals have the option to opt in or opt out, depending on their availability. The benefits for individuals can vary, from being rewarded for additional work, through its being recognised at their appraisals, or it may be that they will be one of the end users of the product and it is in their best interest to be involved or just to raise the profile of accessibility.
Often, the most productive and cost effective approach is a flexible one using a combination of automated and manual testing, using both external organisations and internal testers.
3
Best practice guide: Accessibility Testing | Version 1.0 | November 2017
www.businessdisabilityforum.org.uk
Business Disability Forum is committed to ensuring that all its products and services are as accessible as possible to everyone, including disabled people. If you wish to discuss anything with regard to accessibility of this document please contact us.
Registered charity no: 1018463.
Registered Office: Nutmeg House, 60 Gainsford Street, London SE1 2NY.
Registered in England under Company No. 2603700
3
Best practice guide: Accessibility Testing | Version 1.0 | November 2017