To:Kevin De Pew, Instructor
From: Tonia Carroll, Jennifer Campbell, Amy D., and Douglas Judkins
Date:April 24, 2003
Re:Justification of Usability Test Design
As per your request, we are submitting this memo to justify the decisions we have made thus far with respect to the first draft of the usability test. In this memo, we will be describing the test, discussing the sources we drew upon, and justifying our rhetorical choices. We will also be discussing the choices we have made about who to recruit for the usability test. We hope that this memo provides you with enough information to determine if we are headed in the right direction and to suggest improvements if need be. If you have any questions regarding the matters discussed here, please contact us via e-mail.
Description of the usability test
Before beginning to draft our own usability test, we reviewed the examples presented in class through in-class demonstrations, outside readings, and The Guide. Having reviewed these tests, we were able to begin to construct a template for our test. We decided to follow the standard of usability tests as we saw fit; therefore, our test consists of an introduction that informs the participants about the purpose of the research and assures them that they are not being tested, rather, the Quick Reference Guide is. The introduction is followed by a series of demographic questions and the actual task list. At the end of the test are questions intended to spark responses during the debriefing.
Sources drawn upon
As mentioned previously, we drew upon examples of usability tests presented in class in order to begin drafting our own test. We also used information that the Audience Group summarized in its report in order to understand the audience to whom we are writing. Additionally, we relied heavily on the draft of the Quick Reference Guide given to us by the Documentation Group to understand both the program and the tasks presented in the guide Information gathered from the Reporting Group was necessary for us to understand what exactly Mr. Razzouk was expecting of us.
Justification of rhetorical choices made
The first choice that we made involved constructing a template for our usability test. We decided that, for the most part, the examples of usability tests that we consulted followed a format that would suit our needs. We wanted to make the format of the usability test easy enough to understand so that if we test a beginning user, he or she will still be able to comprehend the task at hand. Additionally, we wanted to assure the participants in the introduction that the guide is being tested, not them. Also in the introduction, we wanted to make it clear that the guide was to be their main source of help so they would not ask us for help, thus swaying the results of the test. We wanted them to know that the research they are participating will help users of the guide in the future so that they would understand their important role in the revision of the guide. We also felt that it was important for them to feel a connection to the DLC so that they can take advantage of its resources in the future.
The Audience Group’s report detailed our audience for this test and suggested that potential users of the DLC had the desire to create multimedia projects but had not yet done so. From that information and Mr. Razzouk’s suggestion that we make the instructions for the tasks of the test fairly open-ended, we decided that our instructions would allow the users to make a document of their choice. However, in order to ensure that the users consult the guide during their test, we added a few loose guidelines—that they incorporate two video files, one audio file, and one transition. We would also like for them to be able to play their project on a media player so that we can ensure that they also consult that portion of the guide. Because we did not have to make a very detailed task list, we ended up relying less heavily on the guide than we anticipated. We used it mainly to gather a general idea of what kinds of tasks are possible in Adobe Premiere.
Because the Audience Group had already developed a survey based upon the class’ ideas about the target audience, we decided that it would be appropriate to use many of these questions in the pre-test questions. We changed some of them and omitted others when we did not feel like the information was necessary for the task at hand. We felt that it was very important to ask the user what he or she wanted to accomplish during the course of the task so that after the test, we could judge how easily the user met that goal with the aid of the guide; additionally, we wanted to see if parts of the guide will hinder the completion of the desired task. We also designed questions about the users’ knowledge of and experience with computer technology because we felt that knowing that information about the user might have a significant impact on our understanding of the outcome of the test.
We felt that it was important to ask in the post-test questions whether the user was able to complete his or her desired task and to gather the participants’ opinions about what hindered and helped them complete the task. We also asked them for their overall impression of the guide for similar reasons—to find out what they liked and what they disliked. In order to assess whether the guide was effective in its presentation, we asked them about their preferred format and media. Additionally, we felt that it was important to ask them to let us know specific points in the test that were unclear to them so that the Documentation Group can work specifically on revising those sections. Finally, we decided to ask them about their experience with the program as a whole and what they anticipate doing with Premiere in the future; we wanted to gather this information for the Audience Group to report to Mr. Razzouk.
Potential participants for the usability test
We have decided that we would like to recruit at least two beginning users and two advanced users so that each end of the spectrum of users is covered. We would, of course, like to use as many participants as possible (up to 10), but we realize that there may be time constrictions. If we can only find four participants, we will at least know how two very different kinds of users feel about the guide.
1