Are Separate Interfaces Inherently Unequal? An Evaluation with Blind Users of the Usability of Two Interfaces for a Social Networking Platform
Pre-print version, should be cited as: Wentz, B. and Lazar, J. (2011). Are Separate Interfaces Inherently Unequal? An Evaluation with Blind Users of the Usability of Two Interfaces for a Social Networking Platform. Proceedings of the iConference 2011, 91-97.
Brian Wentz
Keystone College
Division of Business, Management, and Technology
La Plume, PA 18440
+1-717-514-9891
Jonathan Lazar
Towson University
Department of Computer and Information Sciences
Towson, MD 21252
+1-410-704-2255
ABSTRACT
With the increasing use of web-based applications in the workplace, it is imperative that all users can equally access those applications. It has been previously reported that blind users have problems accessing Facebook, but little empirical data on the topic exists. It has also been suggested by Facebook and anecdotal user comments that the mobile interface (hereafter referred to as “Facebook Mobile”) for the application is more usable than the standard, desktop interface (hereafter referred to as “Facebook Desktop”) for individuals who use screen readers to access the Facebook interface from their computers. This paper presents empirical data from 15 blind users, who took part in the usability evaluation of Facebook Desktop as well as a second phase of usability testing with 15 blind users to evaluate Facebook Mobile (when accessed from a computer and web browser). This research concludes that Facebook Mobile is more usable than the Facebook Desktop interface; however, the mobile interface is missing some features and is not consistently aligned with the Facebook Desktop interface. The implications of this study raise the question of whether there is often a usability and functionality difference between different interfaces for an application when one interface is suggested to be the “accessible” version.
Categories and Subject Descriptors
K.4.2 [Computers and Society]: Social Issues – Assistive technologies for persons with disabilities.
General Terms
Human Factors
Keywords
Blind, visual impairment, Facebook, Facebook mobile, social networking, screen reader, usability, universal usability, human-computer interaction
1. INTRODUCTION
Many applications today provide various versions and interfaces for accessing the same data and functionality. This is not just a temporary matter of versioning (e.g., where one interface is the version 6.0 and the other is the version 7.0); there are permanently multiple versions. With the increasing use of mobile devices, many web-based services also provide a mobile-friendly interface for their application. The increasing awareness and concern for accessibility has caused some developers to suggest that a particular interface (usually the mobile version) is simpler and therefore more accessible to users with disabilities. Sometimes web sites (such as Amazon) publicly recommend their mobile interface as the “accessible” interface due to its simplistic design for mobile devices [2]. Yahoo Mail is another example of a web-based application for which a different interface version is recommended to address accessibility challenges (i.e., Yahoo Mail Classic rather than the current Yahoo Mail interface is recommended as being accessible). Our previous usability evaluation of seven common email applications as used by blind screen reader users showed that web-based versions of email applications are often less usable than desktop email applications for blind users (54% task success rate for web-based email applications versus 78% task success rate for desktop-based email applications) [22]. Facebook suggests the mobile interface for its application as the HTML-only alternative to the standard interface [5]. As such, is it unfair or discriminatory for a particular group of users to only be able to fully use a particular version of a web-based interface that is unequal to the standard or newest version?
Social networking applications can be used for communicating, collaborating, and strengthening professional relationships [16]. Facebook reports an active user community of over 500 million users [24]. Even government entities (e.g., the Central Intelligence Agency and the Environmental Protection Agency) are beginning to use social networking as a workplace tool and have been developing guidelines for the use of this and other web-based tools [19]. It has also been noted by researchers that the current shift to cloud computing presents a number of new policy implications relating to the use of such applications [6]. As social networking becomes more acceptable as a legitimate collaborative workplace tool, the level of accessibility and usability for all users should be continually evaluated. The high unemployment rates of individuals who are blind (70-75% in the U.S. [10]) places an onus on researchers to identify possible usability problems with social networking applications such as Facebook, since it may impact employment.
The World Health Organization (WHO) estimates that there are nearly 314 million individuals worldwide who are visually impaired (low vision), and this figure includes 45 million individuals who are blind with no residual vision [23]. A screen reader (such as JAWS, System Access, or Window-Eyes) is software that audibly reads the visual content on a computer screen to a blind user, and this is the dominant method that blind users use to access computers and web sites. Braille devices are often cost-prohibitive, and the rate of Braille literacy among blind users is extremely low (an estimated 10 to 20% in the United States) [11].
Blind individuals face unique challenges when using computer software and web sites. A recent survey on screen reader usage by WebAIM indicated that Facebook is a web site that is avoided by a majority of screen reader users [21]. In 2008, AbilityNet (in the UK) conducted an analysis of popular social networking web sites (including Facebook) [1]. Many accessibility flaws were highlighted, including the use of visual CAPTCHAs, a lack of skip links and alternate text, and no alternatives for features using JavaScript. It has been previously documented that blind users are more likely to avoid content when they are aware that it will cause them accessibility problems (such as the problems often presented by dynamic web content) [3].
In 2009, the World Wide Web Consortium (W3C) noted that many social networking web sites had not yet addressed accessibility requirements [20]. Facebook does include a page in its Help Center on Accessibility and Assistive Technology which discusses the audio CAPTCHA, the HTML-based mobile site, browser keyboard shortcuts, and other topics in an effort to address accessibility [5]. However, a statement or statements regarding accessibility may not equate to accessibility, usability or full functionality. This and anecdotal data of comments from blind users suggested that we needed to evaluate whether the mobile interface for Facebook was a more usable and equivalent alternative to the standard Facebook interface.
2. RESEARCH METHODOLOGY
2.1 Data Collection
Usability testing participants were self-labeled as blind with no residual vision, screen reader users not able to use screen magnification, and at least 18 years of age. Recruitment emails were sent out to members of both the Maryland and Pennsylvania chapters of the National Federation of the Blind and the Pennsylvania Council of the Blind in order to obtain participants. Phase one, the usability testing of Facebook Desktop, was conducted from January through March 2010 with 15 participants, using a netbook with an external keyboard, external speakers, JAWS 10 (screen reader software), and Internet Explorer 8. Data logging software was used to record the keystrokes typed, and a stopwatch was used to record the time spent on each task.
Phase two, the usability testing of the Facebook Mobile, was conducted from July through August 2010 with 15 blind participants. The mobile interface was tested with a netbook, using Internet Explorer 8 and the same hardware and software configuration as in phase one. Once again, data logging software was used to record the keystrokes typed, and a stopwatch was used to record the time spent on each task. Prior to the usability testing, time was spent creating test Facebook and email accounts that would be functional for testing Facebook but would also ensure that no personal data of the participants was used. Each Facebook account was populated with simulated posts from friends. Pilot testing of the methods took place with two blind users before both phase one and phase two of the data collection. The tasks and scenarios associated with those tasks were identical for both usability testing phases.
3. RESULTS
3.1 Demographics
Participants in phase one ranged in age from 22 to 60, with the mean being 37.6. Most of the participants were female (87%), and 93% of the participants were college graduates or had completed some college. The average number of years that participants reported using social networking web sites was one year, and social networking applications that participants had previously used included Facebook, LinkedIn, MySpace, and Twitter. A total of 67% of the participants reported previous experience with Facebook, and 60% were current Facebook users.
Participants in phase two were of a very similar demographic makeup (80% of the participants were also involved in phase one), except for a slightly higher mean age of 38.7 and a slight decrease in the number of female participants (80%). Once again, 67% of the participants had previously used Facebook, and 53% of the participants had previously used the Facebook mobile version from their computer. Because a majority of the participants had previous experience using Facebook, we do not believe that there were learning effects for those who participated in both phase one and phase two.
3.2 Phase One—“Facebook Desktop”
Table 1 represents the usability tasks that were used when testing Facebook Desktop with the study participants. The mean completion rate of all tasks by users with previous experience with Facebook was 54%. Users without previous experience with Facebook exhibited a lower overall completion rate of 32%.
Task one (logging in to Facebook) had a high completion rate, with all but one user being able to successfully complete the task. Two users did, however, note that the labeling for the password field was confusing. The password edit field appeared to be labeled correctly, but the username edit field seemed to be occasionally not reading the descriptive label.
Task two (reading a Facebook post from a “friend”) was also completed by all but one user, but two users did have difficulty when trying to find the location of the current posts by the simulated Facebook “friends.”
Task three asked users to add a status posting to the Facebook account. Five users mentioned difficulty trying to find a way to update the status, and two users lost focus to the web browser menus when tabbing with the keyboard through the interface.
Table 1. Rate of Completion and Mean/Standard Deviation of Time (in seconds) for Successfully Completed Tasks on the Facebook Desktop InterfaceTask / Task Description / Task Comp.Pct. / Mean Time / Std
Dev
1 / Open and log in to a Facebook account / 93% / 63.2 / 52.7
2 / Read one of the new posts on a Facebook account / 93% / 158 / 240
3 / Add status posting to a Facebook account / 27% / 55.3 / 342
4 / Send an email to a specific external email address / 0% / - / -
5 / Upload a picture to a Facebook account / 20% / 324 / 26
Mean / 47% / 120.1
No users were able to successfully complete task four, which involved sending an email through Facebook to an email address of a non-Facebook user. The pop-up box to compose a new message was not accessible to 10 users who did manage to find the link to compose a message. Four users who were able to eventually reach the pop-up box by tabbing randomly through the interface were not able to ultimately send the email message due to another pop-up box containing a visual CAPTCHA (with an audio option) that was not evident to any of the users (see Figure 1). They believed that the message was successfully sent.
Figure 1. CAPTCHA Pop-up Not Evident to Users on the Facebook Desktop Interface
Only three out of the 15 participants were able to successfully complete task five, which involved uploading a picture to the Facebook account, due to difficulty when navigating the upload photo interface, and a Facebook “update pop-up” to update the photo upload capabilities of Facebook was not accessible to the five users who received that update notice during their usability testing sessions (see Figure 2).
Figure 2. Inaccessible Facebook Plug-In Update on the Facebook Desktop Interface
Some users noted that they also had experience with using Facebook Mobile from their computers (not mobile devices), and that while it is more accessible, it is missing some features, such as the ability to send emails to friends that are not already on Facebook. Out of the 15 users who participated in this study, 10 had previous experience with Facebook Desktop. The correlation to task completion and time performance is presented in Table 2. Task four (sending an email to an external email address) is not included since no participants were able to successfully complete that task. The major comment about Facebook Desktop was that navigation was sometimes difficult with a lot of non-relevant content and links.