“Seeds of Peace” User Test Notes
August 6, 1999
Subject 1: Age: 16
Gender: Female
Nationality: Israeli
Background: Lives near both the Jordanian and Egyptian borders in a small town in a relatively rural part of Israel. There are no Arabs in her town or school. Has a very strong command of reading and speaking English.
Subject 2: Age: 17
Gender: Female
Nationality: Israeli
Background: Lives near Tel Aviv in a mixed neighborhood. Has a sister dating a Christian Arab. She and her parents are all right with this because “they are not religious”, but her grandmother is very religious and, thus, very upset. Has an excellent command of spoken English and an average English reading ability. She will be joining the Israeli Army next June.
Testing Environment: Both subjects used the software at the same time in a focus group setting. The subjects were given a background on the product and had no questions. Then the subjects were directed on the order of the sections to follow. Subject 1 operated the computer exclusively. On a few occasions, Subject 1 had to translate or clarify the English instructions to Subject 2.
Coexistence Section:
· The subjects had a discussion in Hebrew about the first screen. They told us Subject 1 was translating the screen text to Subject 2.
· Subject 2 wanted to know the nationality the character “Moran”. She was unable to discern the nationality from the onscreen text.
· Subject 1 entered gibberish into all the text fields. No “real” answers were given.
· For the first story (for the character “Moran”) Subject 2 wanted more explanation about the story…the background and surrounding events.
· For the activities, both wanted a more direct question rather than the open-ended question that was presented.
· Subject 2 felt the Moran story about the army was too old for younger children. She felt that students in Israel do not concern themselves with these matters until they are closer to 16-17.
Respect Section:
· Subject 1 felt the definition of respect on top of the screen was unclear: “The definition doesn’t end. How does it fit in?”
· Both subjects did not know what to do on this screen. Kathy needed to explain the purpose of the section and what to click on. This continued for every screen in this section.
· Both subjects read the first passage in this section quietly and did not discuss.
· Both felt the first question in this section was too open. They asked Kathy for clarification Subject 1 suggested to “Make it more specific.”
· The first major passage of this section was “much better than in the last section.”
· For the long passages, Subject 1 read much faster than Subject 2.
· No discussion of the second reading took place.
· Both subjects did not understand what to do at the second question of this section. Subject 2 commented “The story should have been shorter.”
· Subject 2 commented that she didn’t understand all the words in the second passage. “Lots of people will not understand all these words.”
· Subject 1 felt that other users would be offended by the “dreaded Israeli solder” phrase in this passage (this passage was told from a Palestinian point of view.) Subject 1 commented that some people would say “Why would I want to show her respect? She is saying bad things about my country.”
· Subject 2 did not understand the meaning of the word “experience” in the activity instructions. Subject 1 explained it to her in Hebrew.
· Upon completion, both commented at length that the written language was too difficult for non-English speakers.
Methods Section:
· Both understood immediately what to do in the video section. They actually completed the video activity and hit the buttons at the appropriate times. There were two occasions in this section where they conferred in Hebrew about when and what button to press. Both were notably more engaged then elsewhere in the prototype.
· Subject 1 made the puzzling comment “Jewish people would rather do something like this than writing.”
· Both commented that there was too much writing in the earlier portions of the software. They felt the writing needed to be “broken up more” with other activities.
· Neither subject understood the spoken instructions to the “Ground Rules” activity. Kathy had to explain. (Note: when we had them go back to this section during the post-interview, neither of them still understood what the audio instruction wanted them to do.)
· Both laughed at the audio “This ad copy is terrific!”
· Once at the publishing page for this section, Subject 1 asked, “Would a student have to post this?” I was unable to determine what she meant by this.
· Both commented that they liked the publishing metaphor (they elaborated on this more during the post-interview.)
Post Interview:
· Subject 2 emphasized the need for the use of current events to “make sure nothing goes out of date.”
· Subject 1 commented that this would not be used in a home, but would be used at school.
· Both felt that whatever is published from this software should be kept anonymous. No student names attached to the written work.
· Subject 1 thought the “Methods” section was her favorite because it required the least amount of writing.
· Subject 2 thought the “Respect” section was her favorite because she saw something from another point of view (unfortunately, my notes are unclear on the exact quote).
· Subject 1 felt that 13-year-olds were too busy in Israel to use this product. Subject 2 added that it should be kept for 8th grade and up.
· Subject 2 asked whether it would be possible to see what people from other countries had published. If this was “just used in the classroom it would be not as good. I want to see what people from other countries.” (Verbatim quote).
· Both felt that audio instructions were better because it was easier to follow (however, I observed that Kathy had to explain just as much on the “audio” enhanced screens than those without.)
· The subjects were asked to revisit the “Ground Rules” screen of the “Methods” section. Upon hearing the audio instructions a second time, they still asked Kathy to explain the instructions. Both commented that the audio was too confusing. Subject 2 suggested that a written example accompany the audio.
· Since they chose not to write about any of the topics presented in the software, I asked whether they would in a classroom environment. Not surprisingly, both answered yes. I decided not to press the issue.
Initial Observations:
· The subjects were just as confused by the spoken audio as the written instructions.
· Both really liked the publishing metaphor.
· Unsure on whether the perceived offensive terminology “dreaded Israeli solder” should be removed. This promoted quite a spark in both participants. Was this offensive, or were they reacting to how the other side thought of them?
· Neither enjoyed the idea of an opportunity to write about topics like respect and coexistence. They preferred non-writing activities like the video clicking segment.
· The open-ended questions were not well received. More specific, targeted questions were suggested.
· Subject 1 seemed unaffected by her experience with the product.
· By contrast, Subject 2 seemed to have a firm grasp of the intent of the software and recognized the importance of seeing issues from opposing points of view. She also seemed genuinely interested in what Arabs would write about these issues.
· Neither subject showed interest in the cookies, milk, or soda. Perhaps we should have offered these to them.
· Both subjects were happy about receiving their Stanford t-shirts. One should never forget the power of the “free t-shirt.”
· Dave stupidly drove by the “cute” restaurant on the way back to the highway. 10 hours later, he is still filled with regret.
· Kathy and Janette are very silly for thinking that crappy white van in front of us actually paid our toll for the Golden Gate Bridge. Kathy claimed this “happened all the time”. Poor Kathy is clearly over-stressed.
· Janette should have known better, since she is “the smart one.”
3