1.2. General Instructions
The following short description and a set of general instructions were provided to the user before he was assigned any tasks to perform. The reason behind providing a terse description and general instructions was to prime the user with the motivation of the application and make the evaluation process sound simple and within his ability.
Time Contextual Organizer-in time, with time
Overall intent of the product:
We are developing a plugin application Time Contextual Organizer for your smartphone that can help you better organize your digital life around work and home.
Key Tasks supported:
The TCO is supposed to do certain tasks like providing a work view and personal view of your significant digital services like emails, calendar, message inbox and home screen. It can also be your virtual personal assistant and hold notifications for certain third party apps that it will provide you in either work mode or personal mode. It can also let you set an auto responder for your callers and specify if you want the phone settings to change during the event while you add an event in your calendar.
General Instructions:
· Below is a list of 6 independent tasks that we would like you to carry out.
· Please think aloud and verbalize your thoughts while you are performing these tasks.
· Do not switch from the work mode of the phone to the personal mode if not explicitly asked to. You can always use the iPod Home button at the bottom end of the screen to go to your home screen while in work mode.
· If you would like to leave the evaluation at any point of time, please let us know.
· Also, in case of any confusion please do not hesitate to consult us.
1.3. Task Cards
We had asked the user to perform 6 independent tasks in the same order as a way to explore the design prototype. We had provided exact input information and any limitations due to the product being a prototype to the user. Each of the tasks was provided to the user in a separate task card and he was allowed to read it before and within the task duration. We did not specify any explicit task completion point as one of our evaluation criteria was also to figure out if the user was able to comprehend it. Once the user informed the observer that s/he thinks that s/he was able to complete the task, the observer handed him/her the next task card. In case of any doubt from the user, the observer had to intervene and inform him/her that the task was not complete or complete.
Task 0: Switch ModesYou are in the personal mode of your phone.
Please go to the work mode of your phone.
Task 1: Adding email sender to priority list
You have your phone in the work mode.
Please check your email inbox. You will find a message titled “Need your ideas today” from John Doe. John is an important sender for you. Please add him to the all-time priority list so that you can access his emails in both personal and work mode.
Task 2: Creating a calendar event and set up phone options
Imagine today is 30th August, 2009. You are in your office. You remember you have a meeting today evening.
Please add an event for today and set up an auto responder. Also please set the phone to go silent during that time.
[Please ignore putting the title and time of the event. These functionalities are not active in our prototype version of the design.]
Task 3: Modifying Working Hours
Imagine you have been recently notified that instead of your regular working hours of 8:00 AM to 5:30 PM on Monday you have now a change in shift to 10 AM to 7PM. This change is only for Monday. So you would like to change the settings of the Time Contextual Organizer to reflect your change in working hours.
Please change the Time Contextual Organizer settings to reflect your new working hours for Monday.
Task 4: Change 3rd Party Notification
You are leaving your workplace. You have your phone in work mode. You had set up the Time Contextual Organizer to block some third party applications during work.
Please go to the personal mode. Please read the new notification and choose the option reflecting that you do not want to be notified about this new event again. Please change the Time Contextual Organizer settings to show Buzz (A 3rd party app) notifications at work. Please go back to the home screen and stay in the personal mode.
Task 5: Reflection on different Home screen in different modes
Please take 5 seconds to take a look at the icons on the home screen of your phone in the personal mode. Now please switch to the work mode. Take 5 seconds to look at the icons on the home screen.
2. Internal Walkthrough
One of the team members took the role of an expert user and performed the same tasks as assigned to the users. In the following tables, we provide the results of the internal walkthrough with respect to each of the tasks.
Task 0: Switch ModesInputs/Assumptions / The user is at the Personal mode of the screen
Steps / Switch to “Work” mode from “Personal” Mode
Time for expert / 4 seconds
Instructions for user / Task Card 0
Walkthrough Comments / 1. The font for labels “Work”, “Personal” and “All” are not in sentence case.
Task 1: Adding email sender to priority list
Inputs/Assumptions / The user is at the “Work” mode of the screen
Steps / · Click the TCO inbox icon
· Look for the email “Need your ideas today” by “John Doe”.
· Click the add icon
· Click on the “Add to All list”
· Click OK on the confirmation button.
Time for expert / 20 seconds
Instructions for user / Task Card 1
Walkthrough Comments / 1. The TCO mailbox icon is at the bottom row of the icons in the home screen. The users start looking at the screen from top row of the home screen icons, always from the leftmost.
2. The name of the sender is not shown below the Message title and above the first line of the message in the inbox view.
Task 2: Creating a calendar event and set up phone options
Inputs/Assumptions / · Imagine today is 30th August, 2009.
· The user is at the “Work” mode of the screen
Steps / · Click the TCO calendar icon
· Look at the calendar to see your current date is 30th August 2009.
· Click the add icon on top right of the screen
· Click on the “Add Auto Responder” button
· Click on the “Compose New” button
· Click done
· Click OK on the confirmation button.
Time for expert / 40 seconds
Instructions for user / Task Card 2
Walkthrough Comments / 1. The name of the month “august” is not in sentence case.
2. The subtitle of the “Add Auto Responder” is not descriptive enough for the user to sense the meaning. Something like “Record a playback message for calls missed during this event” can be more informative.
3. The subtitle of the “Modify Phone Options” is not descriptive enough for the user to sense the meaning. Something like “Specify Phone ringer settings during this event” can be more informative.
4. After doing one of the tasks, at the new event screen there is no feedback to the user which task the user has completed. The blank square at the left can be used to give simple feedback like “Not Set” and “Set”, when the user activates an option.
5. The labels for the options are lengthy and ambiguous to glance and get the meaning. Using “Auto Responder” and “Phone Ringer Settings” as labels can display more information scent.
6. The Auto Responder page has incorrect label: “Auto Reponder”.
Task 3: Modifying Working Hours
Inputs/Assumptions / · The work hours for Monday have changes to 10 AM to 7 PM for the user.
Steps / · Click the TCO Settings icon
· Click on Modify Working Hours
· Click on Monday
· Enter 10 in the Start box
· Enter 7 in the End box.
· Click OK
Time for expert / 18 seconds
Instructions for user / Task Card 3
Walkthrough Comments / 1. In what format to input the working hour is not intuitive. Giving the user an option to input like below can be better:
Hours Min AM PM
Task 4: Change 3rd Party Notification
Inputs/Assumptions / · The user has blocked the notifications from the 3rd party applications Google+, Buzz and Facebook to not to be shown in the Work mode.
· The user is at the Work Mode
Steps / · Click on Personal mode
· Click Ignore on the notification dialog.
· Click on TCO settings.
· Click on Change Buzz settings.
· Click on Add to Work.
Time for expert / 27 seconds
Instructions for user / Task Card 4
Walkthrough Comments / 1. The Snooze button is not intuitive. Using just an “OK” button should be sufficient.
2. The dialog says “Add to Work List” and asks for “Notify Buzz at”. The question should be specific like “Notify Buzz at what time?” “The dialog options should say “Work”, “Personal” and “All”.
3. There is no feedback, that the user is able to change the settings. A confirmatory screen saying “Buzz will notify you at Work” can help as a feedback.
Task 5: Reflection on different Home screen in different modes
Inputs/Assumptions / The user is at the personal mode of the phone.
Steps / · Look at the home screen in the personal mode
· Click on Work tab
· Look at the work screen in the personal mode
Time for expert / 12 seconds
Instructions for user / Task Card 5
Walkthrough Comments / N.A.