Comparing Managers' and Usability Practitioners' Views of Usability
Minttu Linja-aho
Helsinki University of Technology
Abstract.In this paper, the conceptions that managers and usability practitioners have about usability are discussed. A literature review and a description of a case study in a Finnish company is included. In the case study, managers’ and usability practitioners’ views of usability definitions, methods, end user roles, and responsibilities are reviewed. Finally, the implications of this information to usability work in organizations is discussed.
Keywords: Usability, Management
1Introduction
In business organizations, usability practitioners work in an environment where the business goals of the company steer the product development efforts. Usability professionals’ goals are to find ways to develop more usable products. In an organization, decisions about usability activities require communication between different parties, for example managers and usability practitioners. Even if all parties talk about the same thing with partly the same terminology, there can still be significant differences in how those parties understand those terms and underlying issues. In this study, the goal is to get some understanding of the differences in conceptions that usability practitioners and managers have about usability.
In our research, I take a constructivist perspective, where I assume that different people give different meanings to usability, and the meaning also depends on the context. In this discursive approach, I will analyze how people in organizations use the word 'usability' and what the word means for them. I also review the expectations that different people have for usability work in organizations.
I first do a literature review concerning both management literature that deals with usability and literature written by usability professionals. After that, I present a case study that was done in a Finnish software company. In the case study, I interviewed usability practitioners and product managers.Product managers’ responsibilities include identifying and prioritizing product requirements and deciding how product development resources are used in the organization. The company reviewed in the case study develops software products. In development of services or tangible products, some issues may be the same but some may be different.
2 Managers' View of Usability
In many publications that report the views of managers, good usability is mentioned as a goal in product development. However, it is often not explained any further what managers mean by good usability. (See for example Bedford, Daniels, Desbarats, and Hertenstein, 2006). Berkun (2002) describes that many project leaders and business managers mention usability or customer satisfaction in their vision documents and specifications, and sometimes even use these concepts to justify business plans or budgets. Sales and marketing managers describe the value of a product in terms of customer convenience or satisfaction. In a case study by Chen, Haney, Pandzik, Spigarelli, and Jesseman (2003), for example, Getzs' management aimed to "create a better customer experience through improved look, functionality, and usability, while having a scalable design".
Cajander, Gulliksen, and Boivie (2006) in turn have noticed in their study that different managers define usability from the perspective of their own work: for the head of operation managers, usability is a question of performance and response times; for the head of project office, usability is related to the business process.
CHI 99 panel participants named lack of understanding of usability as one obstacle for usability work in organizations (Rosenbaum, Rohn, and Humburg, 2000). The education of business managers has not included much information about usability, which according to Zhang (2006) is one reason for the lack of knowledge. However, Berkun (2002) has noted that executives and managers are trained on the effects of a good design; however, they may have little information on the methods, skills and challenges to create it.
The level of knowledge on usability varies depending on the type of business that companies are involved in. According to Mao, Vredenburg, Smith, and Carey (2005), the growing popularity of e-commerce has made usability and user-centered design more appealing. The number of publications that mention the role of usability in the development of web services is quite high. As a sample, in Chen et al.’s(2003) case study about a small business internet commerce, and usability improvements are mentioned several times in the report. Also Eid and Trueman (2004) discuss the role of usability in business-to-business Internet marketing. In web services, the feedback about usability comes quite fast in the form of users either using or leaving the site, and therefore, it is easier for managers to consider usability important. In web services, tracking features can often be used that indicate how many users have left the site at some point.
According to Janice Redish (in Rohn, Spool, and Ektare, 2002), it is a rather common misconception to equate usability practice with usability testing. The same issue was mentioned by Rosenbaum et al. (2000) when they described the results of the questionnaire about strategic usability. There are also examples in the management literature where managers confine usability to usability testing. A sample comes from a roundtable discussion concerning the benefits of good design. George Daniels, group manager of HP Enterprise Design Center, mentioned usability tests in HP's internal usability labs but made no other references to usability (Bedford et al., 2006). The reports do not include usability practitioners' own descriptions of their duties, so we do not know if they actually do mainly usability testing, or is usability testing just the activity that is most visible to managers. However, these companies would not be the only ones concentrating mainly on usability testing: according to a case study reported by Caims & Beech (1999), the essential part of what for example Reuters' usability group does is usability testing. In addition, in the questionnaire reported by Rosenbaum et al. (2000), usability practitioners rated usability tests as the most effective method at contributing to strategic usability. Rohn et al. (2002) argue that lab-based formative usability tests are usually the first step to incorporating usability to organizations. A different question then is, if usability work in these organizations should be mainly usability testing or something else. Rohn et al. (2002) say that typically organizations that first started with usability testing, later use a wider variety of methods to more proactively drive decision making in the earlier stages of product development. Another question is that even if usability practitioners' duties include other activities than usability testing, are the results of usability tests the most visible outcome so that managers expect that usability means mostly usability testing?
We must also note that for managers, usability is just one component in a successful product development process. For example, Tarasewich (1996) reviews the factors that affect succesful product development. Usability is one of the factors, but there are many remarkable others: strategy, supplier involvement, customer involvement, cost, time, manufacturability, management, marketability and serviceability. Top and middle management need to be aware of the interplay of these factors and keep the balance between them, and that may be one reason why they cannot always put as much emphasis on usability as usability practitioners would like to.
3 Usability Practitioners' View of Usability
Usability practitioners commonly define usability as the ISO 9241:11 – Guidance on usability standard suggests: “Usability is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use”. (See for example Cajander et al., 2006)
Usability practitioners may also use other ceiling terms for their usability work User-centered design is one of those terms. An example of the definition given for this term by usability practitioners is "the active involvement of users for a clear understanding of user and task requirements, iterative design and evaluation, and a multi-disciplinary approach". This definition was used by Mao et al. (2005) in their questionnaire delivered to usability practitioners.
Another ceiling term used by usability practitioners is user experience. Rosenberg (2004) writes: "Like most practitioners in the HCI community, I define my professional goal as adding value to products by improving the user experience." However, user experience can be thought of as a broader issue than usability: according to Sherman and Quesenbery (2005), it comprises brand design, document design, user interface design, information architecture, ergonomics, technical writing, computer science, testing, and training. It means people for several disciplines need to collaborate to improve usability, usefulness, and pleasure of products.
4 Sharing the Responsibility for Usability
There are some research reports about dividing responsibilities related to usability in organizations. Cajander et al. (2006) have done a case study where they also asked how the responsibility for usability is divided in the organization. In the reviewed organization, there was one person that had been recruited to work specifically with usability. In addition, there were other people with usability-related responsibilities in other parts of the organization as well. For example, in the communications unit, usability-oriented tools and methods were used when developing company's website and intranet site.
In the interviews that Cajander et al. (2006) ran, it turned out that employees had different views of who is responsible for the usability in the organization. Some of them said that the usability expert is responsible for usability, some claimed that the end users are responsible, or the general director. However, a majority of the interviewed claimed personal responsibility for a subset of issues related to usability. Many of them mentioned that the responsibility for usability is shared in the organization. However, many of the interviewed employees also complained that shared responsibility often means no responsibility, and if no one is specifically responsible, it is easy to skip usability issues in the development process.
Iivari (2006) has identified four roles for usability practitioners in organizations. The roles are informative, consultative, participative, and configurer, and they indicate how usability practitioners are involved in "configuring the user", or designing the product for the specified users. Practitioners in the informative role provide data to the development process, in the form of usability tests or reports for example. Practitioners in the consultative role may for example review the design proposals and provide feedback and suggestions for improvement. Practitioners in the participative role spend a lot of time in personally participating in the design process. Practitioners in the configurer role contribute directly by being the ones that design or implement the product. In the last scenario with the configurer role, usability practitioners are not actually in a separate specialist role, but everyone in the development process is assumed to possess capabilities to do user-centered design.
According to Iivari (2006), all four roles were found in the three organizations that were reviewed in the case study. All roles have both positive and negative sides. For the informative and consultative role, a problem is that contribution to the design process often comes too late. The participative role, for its turn, requires a lot of usability professionals to be employed in the organization, if they want to be involved in all development. In the configurer role, the responsibility for usability is shared, which can mean that nobody takes the real responsibility, which was noticed also by Cajander et al. (2006). In addition, a lot of knowledge about human-computer interaction and usability need to be acquired by all members of the development project.
3 Case Study: Views of Usability in a Finnish Company
In this case study, I interviewed three managers and three usability practitioners in a Finnish software company[1]. I selected the managers that are responsible for selecting the product requirements and deciding about the product development resources for these interviews, as these managers are in a key role in deciding about the usability development that is done for the product. The usability practitioners that were interviewed work in the company’s usability group.
The interviews were semi-structured and lasted for about one hour each. The interviews were organized at the company’s premises.
3.1 Managers’ Definitions for Usability
In this company, managers seemed to know rather much about usability. They frequently use the word usability in product roadmaps, for example. One of the managers mentioned that the familiarity might be due to the long history of usability work in the organization. There has been a person working on usability issues since the turn of the century.
Below are definitions given by three managers.
“Usability refers to how efficiently the customer can perform his/her tasks.”
“Usability means that the user can quickly, efficiently, and easily enough perform the tasks he/she needs to perform.”
“The most important issue in usability is that the user can do what he/she needs to do. The second most important issue is that the user can do that easily.”
One of the managers also explained that lack of understanding of the user interface, lack of understanding of the functionality, errors, and functionality that does not correspond to user’s needs decrease usability. He also mentioned that motivated and satisfied customers are connected to usability.
Two of the managers mentioned the concept “easy to learn” when asked to define usability. They considered learnability to be connected to usability and refer to the first phases of using the product.
One of the managers mentioned that usability is sometimes confused with the word availability. Hesaid that he had just been in a meeting where the customer had used these terms interchangeably. The customer named availability problems caused by excessive memory usage as a usability problem.
Another manager mentioned that a common misconception is that usability is limited to how controls are laid out in the user interface. He pointed out that it is even more important to ensure that the product contains the right functionality, and that should be included in usability work. In this view, usability comes closer to the management of requirements.
3.2 Managers’ Suggestions for Reaching Good Usability
The interviewed managers gave very different answers when asked how good usability could be reached in the products of the case company. Here I review the suggestions given by them.
One of the managers stressed that users spend 95% of their time with the core functionality of the product. The emphasis should be put into making this functionality as easy to use as possible. The functionality should be intuitive, unambiguous, and errors should be prevented.
Another manager said that the first step in improving usability is to find out what the users really do when they use the product, and which things they consider difficult, especially when starting to use the product. After that, improvements should be made to overcome the problems that have been observed. A profound user interface change, or moving to a next-phase user interface, can be a solution to many of the observed problems. Another way to proceed is to fix the individual problems that have been observed. Many of the individual problems are already known by the members of the development organization, but the information should be collected and taken forward to the production phase.
Third manager considered that as it is not possible to develop a user interface that corresponds exactly to every user’s needs, the goal should be to develop a generic user interface that is customizable to the needs of individual users. With the generic user interface, it should be possible to perform all the tasks that users need to perform, and with the customized layer of the user interface, the tasks that individual users often perform should be made easily accessible. This manager stressed the role of writing use cases on two levels to make the needs for the product explicit. The first level of use cases should describe the tasks that users currently perform with their own systems. These use cases could be used as a basis for understanding the needed functionality. The second level of use cases should describe the user interface controls and their usage.
3.3 Managers’ View of the Role of End Users
All managers considered it important that end users are somehow involved in the development process. Their views of how this should actually be done varied a little.
Two of the managers mentioned that there is not enough information of the real work processes of users. They said it would be beneficial to get this kind of information by for example observing users doing their real tasks with the product.
One manager mentioned that user profiles would be a useful way to describe what is known about the users of the product. User profiles should contain information on what functionality of the product each user profile is using mainly and what are the secondary usages.
One of the managers considered piloting and prototypes as a good way to get information about the needs of users and feedback about the current version of the product. He had some good experiences about developing a product iteratively with a customer. He also mentioned that it is often difficult to get information just by interviewing end users. It would be better to actually go and see what users do, but due to practical limitations, this is sometimes difficult.