Prototyping

Puneet Ralhan

CIS 732

Fall 2000

12/18/2000

Table of Contents

Introduction

Prototype Development

Methodology

Types

Paperboard Prototypes

Contextual Prototyping

Prototype Evaluation

Protocol Analysis

Cognitive Walkthrough (Norman’s Model)

Communicability Evaluation

Expert Reviews

User Participation

Rapid Ethnography

Experience Prototyping

Tools

Hypermedia Management Tools

Interface builders Tools

4th Generation Systems

Object-Oriented Application Frameworks

Shortcomings of current tools

Web Applications Prototyping

Benefits

Final Thoughts

References:

Articles

Books

Prototyping

Introduction

Software customers / users usually find it very hard to express their real requirements and it is almost impossible to predict how a system will actually be used and how it will interact with other systems in the user’s workspace. Careful requirements analysis along with systematic review of the requirements helps reduce the uncertainty of what a system should do but there is no real substitute for trying out a requirement before committing to it. This is possible if a prototype of the system to be developed is available.

Prototyping is a development approach used to improve planning and execution of software projects by developing executable software systems (prototypes) for experimental purposes. It is very suitable for gaining experience in new application areas and for supporting incremental or evolutionary software development.

Prototyping has many objectives including evaluation of the design, functionality and user interface. This report will be focusing on the user interface aspect with some linkage to the functionality. A function defined in the specification may seem useful and well defined but when the users finally try to use the application they find that their view was incorrect. Prototypes thus let user validate and evaluate their requirements and thus users can discover requirements omissions early in the process. Rapid Application Development Methodology uses development of the prototypes by the software team, working closely with the users in the requirements identification phase. Then these prototypes are either discarded (throw away prototypes) or enhanced (evolutionary prototypes) into production systems.

In the last decade the biggest development has been the advent of the Web based applications where development times are short, large amount of functionality a must and the user interface critical. Prototyping techniques have been adapted to serve this purpose and there is a portion of this report that will be focusing on the Web Applications.

Prototype Development

Thompson, Wishbow [1992] state that rapid prototyping allows members of the development team to confirm whether the product interface (this can include its online documentation) can be easily learned and used. Goals for rapid prototyping include:

  • Conducting iterative testing and revision early in the product development cycle. This approach allows user testing to be an integral part of product design.
  • Using measurable objectives (benchmarks) for user performance and attitude. Empirical measures prevent the endless debate that can sometimes follow user testing. If subject behavior falls below the benchmark there is little doubt among team members that something must be done.
  • Using simple, proven testing techniques. The think-aloud protocol method advocated by Newell and Simon, and discussed frequently in the technical communication literature allows for efficient data collection. If the coding scheme is kept simple, reviewing data and reporting results can be accomplished in a matter of a few days. Protocol analysis also has the virtue of requiring only a small subject sample.

Methodology

A methodology [Purtilo, Larson, Clark. 1991] that can be used for development of prototypes is:

  • Identify Objectives: First, a definition of the problem to be solved must be expressed, along with the criteria by which its success is to be judged.
  • Identify Risk: No realistic software effort can expect a clear and complete statement as the result of a step labeled ‘define the problem’, as the above item might suggest. In the gray and uncertain areas of the problem definition lurk great risks. These gray areas of risk must be identified and made explicit.
  • Formulate Prototyping Hypotheses: Once the risk areas have been expressed, the developer is in a position to design experiments to specifically address those risks. Each experiment addresses a hypothesis concerning the nature and potential remedy of the risk to a product’s success. The hypothesis should include explicit linkage to the parts of the project that may be affected.
  • Construct Prototype System: A system that implements the hypothesized solution must be developed. Traditional programming practices are too expensive to make this practical in many situations. Thus there is a need to use specialized tools that facilitate inexpensive prototyping.
  • Instrument Prototype: Since the primary purpose of constructing a prototype is to answer one or more questions concerning its functional or other characteristics, it will usually be necessary to gather dynamic information regarding the operation of the software. In a large prototype, the complexity and volume of information is likely to exceed the limit of a single person’s ability to comprehend it. It would be desirable to use automated tools for this purpose.
  • Experiment: The prototype system must be exercised to determine its behavior and gather the output from the system instrumentation.
  • Evaluate Results: The results of the experiments must be evaluated to assess the efficacy of the hypothesized solution.
  • Repeat. This entire process is repeated until one of three outcomes is reached:
  • sufficient information has been derived from the prototyping activities to begin the development of a product
  • no solution to the problem can be found or, equivalently, that the solution would be too complex or costly to be of benefit
  • the prototype system itself is of sufficient quality to serve as the production-quality system or, that it can be modified to serve as the production-quality system at less cost than a full-scale production-quality development effort.

Often the prototyping phase is considered as a part of the requirements gathering phase and are outsourced to a vendor (external / internal) and then bids are gathered for the development of the production system.

Types

There are three types of prototypes [Sommerville, 1995] –

  • Throwaway Prototypes
  • Evolutionary Prototypes
  • Incremental Development

Throwaway prototypes essentially use prototypes to clarify functional / user interface requirements and also help the managers identify any potential risks. Once the prototype is evaluated and the requirements updated the prototype is discarded and the development starts afresh. The main problem faced in this approach is when the management is tempted to take the prototype and enhance it into the real application. I actually have seen this in my professional life and this can lead to problems like – uncontrolled changes during the prototyping stage, design compromises while adding new features and missing features like security, performance & reliability at base levels. Since the prototype is not the final product the cost of iterations should be kept well in control because of the tendency of the development team to go into too many details of the application and then tempting to reuse some of that application in the final product.

Evolutionary prototyping is based on developing an initial application and then iterating through multiple user evaluations and improvements. The important aspect of this is that the application has to be written in an environment that allows such development. A good language for such application development would be Visual Basic or Powerbuilder. The important aspects are that the design should be updated thoroughly at every iteration to avoid any possibility of compromises being introduced in the application.

Incremental development is the methodology where each delivered production software is taken as a prototype and after the evaluation by the users the changes and suggested and incorporated in the next release. This is a useful in the environment where the development cycles are small and the user group is anxious to get some functionality for being able to work. The main drawback in this methodology is that the system architecture needs to be determined initially and any changes to the architecture could lead to compromises in the application robustness/reliability.

Baumer, Bischofberger, Lichter, Ziillighoven [1996] concluded that:

  • The reuse of prototypes for the development of a target system can only be recommended if the development tools produce prototypes with clean system architecture. Many presentation prototypes are planned as throw always for this reason.
  • There is a strong trend for one team to carry out the entire development cycle. Due to lack of know-how many organizations are still dealing with different teams for prototyping and the development of target systems.
  • A newly emerging trend is to use prototypes as a vehicle for developing and demonstrating visions of innovative systems. This source for innovation can be tapped not only for individual software projects but also for various kinds of marketing research and field studies.

Another way of looking at prototypes is [Wiegers, 1996] –

  • Horizontal
  • Vertical

A horizontal prototype consists of a superficial layer of a user interface, revealing the parts of the system that the user will see with little underlying functionality. Such a prototype could be equated with a movie set with realistic street scenes with two by fours holding up the street fronts. Reason of creating a horizontal prototype include exploring interface styles, determining if adequate functionality has been planned for, assessing usability and identifying required navigation paths between different parts of the application. This application actually goes down to level of detail necessary for being able to give the users accurate information for being able to evaluate the user interface and navigational parts.

A vertical prototype consists of a complete slice of functionality for a limited portion of the whole system. Such prototypes are useful for validating design approaches and exploring alternative architectures for high-risk projects. For example a vertical prototype of an application might actually take a functionality and implement the user interface, middle layer (network communication) and some of the functionality of the backend databases. Once the soundness of technical approach is established the rest of the application can be developed.

Paperboard Prototypes

Paperboard prototypes [McConnell, 1998] are an approach that is very useful for developing early understanding of the user interface requirements. In this approach the developers or the end users can start by developing pictures of the screens, dialogs, toolbars and other elements they would like the user interface to have. Developers and users meet in groups and draw simple screens on flip charts. The work involves redrawing the prototypes on the flip charts until the users and developers agree on the systems appearance and functionality.

This approach offers several advantages –

  • Users can develop prototypes on their own
  • Users don’t have to know prototyping software
  • Prototypes can be generated and modified rapidly
  • Prototypes are very inexpensive

Paperboard prototypes also eliminate some of the most common risks associated with prototyping. On the developer side, they eliminate the need of unnecessarily overextending the prototype and of spending too much time fiddling with the prototype tool. On the user side the paperboard prototypes eliminate the risk of the users thinking that the prototype is the finished product.

One disadvantage of the paperboard prototype is that some developers and users simply can’t visualize the software on the paper mockups. This is a strong disadvantage since the essence of the prototypes is to help users visualize the finished product and if in a development effort it becomes apparent that there are cognitive gaps between the understanding of the paper prototypes between the developers and users then this approach should be abandoned and the effort should begin on software prototypes.

Contextual Prototyping

Stary [2000] talks about developing prototypes using the context of use of the application. The contextual development differs from traditional user interface development:

  • It focuses on the context of usage the user population rather than on the technical features required for interaction. However, the latter come into play when transforming context specifications into user-interface code.
  • It considers design to be a non-linear process based on activities (re)engineering work processes rather than performing traditional software-engineering tasks.

The understanding of end users and their organization of work require a conceptual framework of context-sensitive components of interactive systems. The TADEUS framework puts the following components into mutual context:

  • Task model: The task model comprises the decomposition of user tasks according to the economic and the social organization of work.
  • User model: The user model details the individual perception of tasks, data structures and interaction devices, as well as task/role-specific access modalities and permits to data.
  • Problem domain data model: The (problem domain) data model provides the static and dynamic specification of the data-related functionality.
  • Interaction domain model: The interaction model captures all devices and styles that might be used by the users in the course of interaction.
  • Application model: The final specification is an application model. It integrates a synchronies structure and behavior specifications.

A novel representation an interpretation scheme allows in TADEUS to integrate different perspectives (including the application context) through semantically linked models. In addition, the specification of the entire application can be executed for prototyping purposes. Another novelty concerns the relations between elements of the models and between the models. They are automatically checked at a high level of operational semantics with the help of a series of algorithms. In addition, the software architecture of the environment is open to embed existing specification techniques and interaction platforms and also generating frameworks and code.

Prototype Evaluation

Prototypes evaluation is a critical part of the prototyping process and there are two parts of prototypes evaluation - User evaluation and Expert Evaluation. There are many User evaluation techniques including surveys, focus groups, application usage feedback, ethnography observations etc. One of the most effective methods of user evaluation of prototypes is – Protocol Analysis.

Protocol Analysis

Protocol Analysis [Turoff, Michie] is one of the most effective methods for assessing the usability of an information system and for targeting aspects of the system, which should be changed to improve usability. The central part of protocol analysis is the complete recording (in written, audio, and/or video form) of the interaction of a user with a system, while that user "thinks out loud" in order to allow the recording of his or her perceptions, reasoning, and reactions to the system. The analysis is then provided by the researcher, who examines a number of these cases and reaches conclusions about aspects of the system that cause problems for users.

Protocol Analysis requires only six to eight such recorded observations in order to reach conclusions, and the procedure may be completed in a week or less, start to finish. This gives it great advantages over methods such as surveys and experiments, which generally require a hundred or more subjects, and can take weeks, months, and even years to complete. For a relatively small amount of effort, a designer can quickly discover any basic flaws in the design and has considerable chance of exposing more subtle problems.

The technique of Protocol Analysis includes the following steps –

  • Identification of the Subjects: The appropriate subjects have to be selected from the user group. The focus is on getting a fair representation of all the user groups who will be using the application and there should also be an attempt to get at least three subjects of each group to eliminate any sampling errors. Since protocol analysis is conducted with each iteration of the prototype it would be desirable to get different people involved in every phase.
  • Identification of Task: Central to protocol analysis is the task to be performed. The task could range from gathering user thoughts on the screen shots to actually having them use a system that has implemented the entire interface without much of underlying functionality. The objective once again is to get each group to interact with the aspects of the system that most impact them. Some problem areas could be included for multiple groups.
  • Instructions to Users: Users should be given good instructions including the assurance that the exercise is on the evaluation of the system and them. Often the users are afraid to speak their mind and their fears should be allayed.
  • Execution of Protocol Analysis: In this step the users actually go through the steps identified in the task sheet and they are asked to “think aloud” as they perform their task and are recorded. Instead of Audio/Video recording sometimes the analyst just takes notes since the recording could be influencing some subjects
  • Post Task Interview: Subjects are interviewed at the end of the task and are asked specific questions about different choices that they made. They are also asked to fill a survey that gathers details on different aspects of the design.
  • Analysis: Finally the designers take all the data and try to identify patterns in the problems faced by the users. Also each transcript is reviewed to see if some specific misunderstandings exist.

Protocol Analysis is a technique that can be adapted to suit the individual organizations and the cost savings in terms of future redesigns and usage of new applications usually outweigh the expenses of conducting the experiments.