ICS 616 Information Architecture
Instructor: Luz M. Quiroga
Bao Huy Ung
5/5/2011
Usability Plan
Project: APAN Web URL:
Goals/Objectives:
An implicit recommendation system for the APAN website is being designed as a term project by an ICS616 Information Architecture group. To make it accurate, users are required to enter in accurate profile information. As it stands profiles are all text boxes with no limitation for input. The proposed test will examine users as they go through the process of filling out a user profiles. For profiles, we want to test if given the current labels and the free form text input, are users able to enter valid data that can be used for recommendations or will there be too much variety.
User Group:
Giving the time span for looking for test subjects, all subjects will be university students for a variety of backgrounds, which include major, volunteer activity, gender, age, etc. Possible 2-4 test subjects.
Location:
The test will be conducted in small office cubical.
Equipment:
A laptop will be used to access the webpage, a camera to record the screen and audio comments, a script with tasks.
Script
(Before task)
- Hello, today we are going to do a user walk through of a website. You will be asked some general questions and then I will ask you to do a few tasks on the website.
- Please state you name, age, and occupation, if you are a student please state your major.
- Have you ever been in a relief organization? Examples include aid groups, such as the Red Cross, Salvation Army, UNICEF, etc.
- Do you know or have any experience with APAN?
- APAN is a social networking site for disaster relief. Think of it as Facebook but with the goal of relief efforts. You can network with people and find the latest information on a current disaster or plan relief events, etc.
- For this test, we are going to fill out a user profile. We have a test account setup but please fill in your own personal information as you would, but don't worry as this information will not be kept.
- At any point during the test, please feel free to make comments on anything you are doing and ask questions although I will not be able to answer them immediately.
(After Task)
- So why did you leave (blank profile items)?
- What things about this profile could be improved?
- Thank you for your time.
Task Goal
The implicit recommendations project for APAN relies on the information provided by uses in their profile to determine matches. The less information that is filled out in the profile the less likely a recommendation will be given. The accuracy of the information provided and the formatting of how it is written are also important in matching users.
Form fields provide a guidance effect that helps users understand the information that is needed. The proposed usability test will examine the effectiveness of the current layout of the APAN site’s user profile based on the ease of filling out, and the completeness of the profiles.
In the test performed, one male and one female subject were introduced to the website for their first time. Both subjects are university students from two different majors, secondary education and travel industry management. Neither subjects have experience with APAN or have been or are currently in an aid or relief organization.
Each subject will be asked to fill out a user profile of the APAN website. During the task, the subjects will be asked to feely express any opinions and comments on the items. Questions may be asked but only questions not relating to the task may be answered. All relevant questions will be answered and discussed after the subject has finished the task, and further questions will be asked of them, including a discussion of the form fields not filled in.
Analysis
For the purpose of the analysis we will refer to the subjects as the male subjects and the female subject.
Male subject is 23 years old, majoring in secondary education. The subject was able to quickly fill out the user profile. The subject did not stop to think and completed the entire profile within the first minute but omitted a couple fields, which included: Topics of Interest, Organization, Skills, Regional Expertise. There is an empty area in each of the two categories of the user profile, About and Professional Information. When asked to comment on why the subject left the items unfilled he responded that the labels were vague because the site is very oriented toward relief efforts he felt items included should be geared toward that area. As for items under the Professional Information section, he claimed to not have enough experiences, but did provide an interesting comment. He felt that Regional Expertise was vague but too limiting to a drop down box that allowed only one choice of expertise. When asked if there was anything that could be improved upon he said that he would have completed more if he had examples of what he should fill out, referring the system that Facebook employs that begins to auto fill in items that other users have included.
Female subject is 24 years old, majoring in travel industry management. The subject took extra time filling out the profile, approximately fifteen to thirty seconds between each form, making numerous comments on their vagueness. The subject only completed the About section of the profile. During the test she made comments that the categories were vague and could be improved if they gave examples or locked the options by using drop down boxes. Under the Professional Information section she clearly indented to fill them in but was confused as to what information would be relevant. Her main concern was whether the information asked for was pertaining to her current occupation or the relief organizations that she has experience with. She also made the comment on regional expertise claiming that it is odd you are locked into only one region but may have experience in different areas. Her other comment was about citizenship, this is the only other field that was locked with a dropdown box other than Regional Expertise, she claimed that she had a duo citizenship but had to choose only one.
Recommendation
In analyzing the two subjects’ performance and comments, the vagueness of labels and the type of form used on profile items are two issues that can be examined. A reference to the Facebook system in which examples from other users are used to guide users in filling out information is a possible idea that can be looked at. The other item requires an examination of the text only form system that is currently being employed. Determining which areas can benefit from highly structured data or multiple inputs can aid users in filling out the forms. To structure the data, alternative types of form fields can be employed. Certain categories can be locked with only a preset amount of choices using radio buttons or drop down menus. This will limit the variability of responses and help guide users in the task. The completeness of the user profiles is vital to the accuracy of the implicit recommendation system.
1