Deliverables Report
IST-2001-33310 VICTEC
January 2004
Learner Scenarios: Revised Scenarios Requirements Definition
AUTHORS: Sarah Woods, Lynne Hall, Sandy Louchart
STATUS:Final version
CHECKERS:Malcolm Padmore, Ruth Aylett
PROJECT MANAGER
Name: Ruth Aylett
Address: CVE, Business House, University of Salford, University Road,, Salford, M5 4WT
Phone Number: +44 161 295 2912
Fax Number:+44 161 295 2925
E-mail:
TABLE OF CONTENTS
1. PURPOSE OF DOCUMENT
2. EXECUTIVE OVERVIEW
3. BULLYING
3.1. Bullying Roles
3.2 Bullying Types
3.3 Personality constructs for bullying roles
3.4 Emotional constructs for bullying roles
4. ROLE OF EMPATHY & BELIEVABILITY FOR SCENARIO DEVELOPMENT
4.1 empathy and believability in scenarios
4.2 Children’s empathy and believability towards bullying scenarios
4.3 The use of Emergent Narrative to create empathy
4.3.1 Research background
4.3.2 The Emergent Narrative Learner Scenario
4.3.3 Overall structure of an episode
5. Final Scenario Development
5.1 The importance of gender in scenario design
5.2 The importance of cultural differences for scenario design
5.3 Final speech acts for scenarios
5.3.1 Agent-to-agent speech acts
5.4 Final physical actions/behaviours for scenarios
5.5 Final props devised for scenarios
5.6 Coping strategies defined for scenarios
6. School involvement and scenarios
6.1 School involvement
6.2 Teacher Workshop, Kassel, Germany
6.3 Participatory design with children
6.3.1 Design
6.3.2 Results
6.3.3 FearNot Interaction
6.3.4 Interface design
6.3.5 Characters in FearNot
6.3.6 Educational Implications
6.3.7 Conclusions
6.3.8 References
7. Conclusions and future timetable of work
8. References
ANNex 1
Annex 2
1. PURPOSE OF DOCUMENT
The purpose of this document is to report on the progress made since deliverable D2.1.1 concerning the refinement of the requirements for bullying scenarios to be implemented in a Virtual Learning Environment (VLE). Concrete decisions have been reached concerning the design parameters of the bullying scenarios resulting in the central aims of deliverable D2.3.1 being met in terms of scenarios which have believable storylines, and are created in order to allow the children interacting with them to establish empathic relationships. The scenarios also demonstrate high educational impetus in terms of personal and social education.
This document aims to illustrate the progress and final decisions reached by the VICTEC team to ensure that the bullying scenarios are interesting and engaging, involve the appropriate environmental contexts for the children interacting with them, clearly capture the profiles and roles of the characters to be depicted in the scenarios, and report on development for an appropriate evaluation methodology for the scenarios with the assistance of the schools.
Since deliverable D2.1.1, numerous small-scale pilot studies have been conducted in the U.K., Germany and Portugal to ensure that the bullying roles, contexts and content of the scenarios are appropriate and attractive for 8-12 year olds.
2. EXECUTIVE OVERVIEW
The current document provides details of the final research decisions made for the bullying profiles which need to be included in the scenarios for the VLE, and the personality and emotional constructs which are important for each of the bullying roles to be interesting and believable. The document then provides an overview of the research developments towards ensuring that the scenarios are believable for the children and produce empathic reactions. This is followed with the progress made on the use of Emergent Narrative (EN) to create empathic relations for the scenarios.
A great deal of evaluative work has been carried out since deliverable D2.1.1 to ensure that the importance of cultural and gender differences are considered for the final scenario requirements. An overview of the implications from these evaluation studies is provided.
The document then details the speech acts, physical actions and props necessary for generating engaging and interesting scenarios that have been thoroughly researched to ensure that the finer details of the scenarios are accurate. Further research has also been carried out into the types of coping strategies used by children to deal with bullying problems and the indices that alter the likelihood of certain strategies being chosen, alongside the success rates for each strategy.
School involvement with the VICTEC project in the U.K. Germany and Portugal has greatly increased since deliverable D2.1.1. Schools have assisted with the design of bullying scenarios and have taken part in the numerous pilot studies carried out. Some of the VICTEC project members as well as teachers associated with the project attended a Teacher Workshop was held in Germany in October 2003 by the Minerva project VES, with which VICTEC have been working. A report of the outcomes of this workshop of relevance to VICTEC is provided.
The document concludes with an overview of the final psychological and pedagogical evaluation protocols. These evaluations have been carefully devised to ensure that both psychological and technical aspects of the project can be evaluated simultaneously. A timetable of future activities follows.
-1-
Deliverable D2.1.3/Learner Scenarios
Deliverables Report
IST-2001-33310 VICTEC
January 2004
3. BULLYING
3.1. Bullying Roles
The findings from numerous research studies have been integrated to reach the final decisions for the bullying roles and profiles necessary for each character depicted in the scenarios. The roles of the ‘bully’, ‘bully/victim’, ‘victim’ and ‘neutral’ child were identified in Deliverable D2.1.1 in terms of the individual differences between the roles, social cognitive styles of each role, sociometric status and family functioning. Table A1.1 in Annex 1 gives the different profiles for each of the bullying roles identified. Further research highlighted that the roles ‘bystander’ ‘defender’ and ‘bully assistant’ identified by Salmivalli (1996) would be important for the scenarios to be believable. A ‘bystander’ is defined by Salmivalli as being a child who is present at bullying situations as an onlooker only. They do not intervene in the bullying situation in any way and remain neutral throughout. A ‘defender’ is defined as a child who assists a child who is being victimised. This may take on different forms of behaviour ranging from the defender talking to child who is being victimised, standing up to the bully and telling them to stop bullying the victim, telling an adult about the bullying on the behalf of the victim, beating up the bully on behalf of the victim and so forth. The ‘bully/assistant’ is quite frequently also a bully/victim and takes the role of assisting the ‘pure’ bully in bullying a victim. They may egg the bully on or may assist in physically attacking or verbally abusing the victim. The bully/assistant generally does not initiate bullying incidents but is usually present after the bully starts an act.
3.2 Bullying Types
The types of bullying to be modelled in the scenarios have also been defined for physical/direct bullying, verbal bullying and relational bullying. The parameters that influence the likelihood of each of these types of bullying occurring have been identified and are:
a)the types of actions displayed for each type of bullying
b)the environmental context,
c)gender of child
d)the character mix
e)age of child
Table A1.2 in Annex 1 illustrates the bullying types and defining parameters.
-1-
Deliverable D2.1.3/Learner Scenarios
Deliverables Report
IST-2001-33310 VICTEC
January 2004
3.3 Personality constructs for bullying roles
An essential consideration for the construction of believable and engaging scenarios concerns the personality constructs that will characterise each character. It is important for the technical designers to have different threshold levels for a number of personality traits for the different bullying roles according to different situations. Personality traits for each of the bullying roles will also be influenced by the start goals of a character and the goals that proceed for a character as the scenario progresses onto to different episodes. Work is still ongoing towards developing the final personality model for implementation into the VICTEC architecture, but for the scenarios to run accurately the following personality dimensions and traits have been identified as necessary components of bullying characters. The theoretical model chosen as a foundation for modelling the bullying personality traits for VICTEC was developed by Costa & McCrae and is known as the 5 Factor Model or the ‘Big Five’. Table 3.1 below illustrates this model. It is apparent that not all of the traits associated with the dimensions are relevant for bullying scenarios and therefore some traits should be omitted. Table A1.3 in Annex 1 gives a modified version of the bullying personality traits that we think are essential components for the bullying profiles in VICTEC scenarios.
Table 3.1: Costa & McCrae 5 Factor Model
5 dimensions / Traits associated with dimensionsNeuroticism / Anxiety, angry, hostility, depression, self consciousness, impulsiveness, vulnerability
Extraversion / Warmth, gregariousness, assertiveness, activity, excitement-seeking, positive emotions
Openness / Fantasy, aesthetics, feelings, actions, ideas, values
Agreeableness / Trust, straightforwardness, altruism, compliance, modesty, tender-mindedness
Conscientiousness / Competence, order, dutifulness, achievement striving, self-discipline, deliberation
-1-
Deliverable D2.1.3/Learner Scenarios
Deliverables Report
IST-2001-33310 VICTEC
January 2004
3.4 Emotional constructs for bullying roles
Linked to the personality traits identified for the bullying characters are the emotions which the characters should feel and express in the bullying scenarios. The OCC model (Ortony, Clore & Collins, 1988) has been selected for emotion synthesis of the characters in the scenarios. The OCC model specifies 22 emotion categories which are based on valenced reactions to situations constructed either as being goal relevant events, as acts of an accountable agent, or as attractive or unattractive objects. The model also offers a structure for the likelihood of an event or the familiarity of an object which determines the intensity of emotion types. Ortony (2003) identified the importance of consistency in relation to the believability of a character. This consistency can be described as the personality of the character and Ortony believes that it is a good idea to actively design the perceived personality of the character. The Five Factor model of personality by McCrae (1987) provides a good test-bed for measuring the perceived personality of the characters.
4. ROLE OF EMPATHY & BELIEVABILITY FOR SCENARIO DEVELOPMENT
4.1 empathy and believability in scenarios
As was previously detailed in Deliverable 2.1.1 empathy is a somewhat multi-faceted term and can be viewed from many different perspectives. For the project work it has been defined as the affective or cognitive change within an observer due to the perceived inner state of a target. This affective or cognitive change of the observer’s inner state can be interpreted as the two possible outcomes of empathic processes which can be mediated either through situational cues or through cues derived from the emotional expression of the target.
A major goal within the project work is to instigate and facilitate an empathic relation between the child user and the virtual characters. According to the basic theoretical assumptions there are two ways the child can empathise with the character: either they develop cognitive empathy, meaning they try to understand the character’s feelings and thoughts by monitoring the character’s situation and/or expression, or they empathise affectively, meaning they feel something due to the perception of the character’s situation and/or expression.
Situational mediated empathy can be triggered by any cue derived from the situation the character acts in, whereas expressional cues cover facial and bodily expression as well as paraverbal parameters (voice-pitch, speech-rate), and physiological parameters (e.g. flush).
The conclusions that can be drawn for scenario development are that it needs to be ensured that the character’s expressions as well as the situations the character acts in provide cues that the child user can connect to, which they know from their own experience and which they find to be believable. Believability of the agents and the environment they inhabit therefore constitute a precondition for empathic relations between child users and virtual agents.
Research has shown that to develop believable agents there is no need to go for maximum realism as far as their physical appearance is concerned: the more agents look like real human beings, the more the expectation of human-like behaviour is likely to be frustrated. Minor differences in behaviour then lead to uncanny feelings within the user that something is wrong with the agent (“uncanny valley“, Mori, 1982; Dautenhahn, 2002) and therefore constrict the agent’s believability. In contrast, believable agents are equipped with a personality, with goals and motivations, and the ability to express emotions and empathy; and they allow for interaction.
It takes this kind of believable, empathic agents that engage in social relationships to reach the child user and mediate the educational message the project is aiming at.
4.2 Children’s empathy and believability towards bullying scenarios
Results from three different pilot studies carried out to evaluate the empathy expressed by children towards characters in the bullying scenarios and the believability of the storyline from a physical and relational bullying scenario have revealed encouraging findings.
Study 1: Evaluation of VICTEC prototype demonstrator
The first study was carried out using a prototype of the VICTEC demonstrator that provided a test-bed for evaluating the design of bullying scenarios. A single episode from a physical bullying scenario was illustrated where a physical bullying incident occurs between Luke (the bully) and John (the victim), followed by Luke verbally abusing John. John initiates a discussion with the user about possible coping strategies for the incident (fight back, ignore him, tell the teacher or parents) and the user selects the strategy that they believe will be the best for John to deal with the situation (Woods, Hall, Sobral, Dautenhahn & Woods, 2003).
An 8-section questionnaire was developed to evaluate aspects of the VICTEC bullying demonstrator (character preference, realism, voice, conversation content, movement, school environment and match with characters, story plot and length, user feelings and satisfaction) mainly measured using 5-point Likert scales.
The questionnaire was administered at a ChildLine conference after the demonstrator had been presented to delegates as part of a seminar regarding the nature of bullying behaviour in schools. Participants watched the presenter interacting with the demonstrator using the trailer episode. 76 questionnaires were completed.
Results
Believability expressed towards the physical bullying storyline
68% of respondents found the storyline believable. There was a significant association between story believability and match between character appearance and the school environments modelled (chi-square = 10.62, df = 2, p = 0.01). 65% of children reported high story believability if the characters and school environment were highly matched compared to 35% who found a low match between school environment and characters and high story believability.
Conversation content of scenarios
Characters’ conversational content was rated as highly believable by 75% of respondents (mean: 2.2., SD: 0.79). 52% found the conversation highly interesting (mean 2.5, SD 0.89); and 61% found the conversation true to life (mean 2.3, SD 0.94). Aggregated scores (believable, interesting, true to life) for what the agents talked about received high scores for 73% of respondents, 16% found it average and 11% found it unbelievable, boring and false. No significant differences for gender and age.
Physical aspects of the characters
71% of respondents rated the likeability of the voices as neutral or dislikeable, with a trend towards the voices being unbelievable (mean rating 3.3, SD: 0.87) and dislikeable (mean 3.0, SD: 0.91) with 30% of users reported disliking the voices. 84% of respondents found the agents’ movements unbelievable, (mean 3.7, SD: 0.99), 88% found the agents’ movements unrealistic (mean 3.8, SD: 0.91), 89% found the agents’ movements jerky (mean 3.7, SD: 0.99). There was a significant finding between voice acceptability and character movement (chi-square = 13.34, df = 4, p = 0.01) indicating an association between the acceptability of character movements and voices. Users who found character movement unacceptable also had low acceptance of character voices.
Conclusions
Whilst some of the results appeared unfavourable in relation to the characters’ physical characteristics, this does not seem to have any detrimental effects on the comprehensibility and believability of the story. It seems possible to suggest that the users transpose their own feelings onto the characters: ‘getting under the agents’ skins’ and ‘filling in the gaps’ left by technology. This is an extremely
positive result in light of the developments that still need to be carried out on the VICTEC architecture before the final demonstrator is available.
Study 2: Empathy expressed towards characters depicted in bullying scenarios
A further evaluation study was carried out using a high fidelity prototype that was provided using Kar2ouche ( This high fidelity storyboarding tool allowed the creation of scenarios, populated with prototypical partly animated agents, to capture both direct and relational bullying behaviour taking into account the different group roles. 80 children from schools in the U.K. watched both the direct and relational bullying scenarios. For further details of the bullying scenarios, please refer to (Woods, Hall, Dautenhahn, Wolke, in submission). After watching the two scenarios, children completed a questionnaire. Questions enquired about the believability of the scenarios according to children’s past experiences of bullying, and children were asked whether they felt sorry for any of the characters and if so which characters and why; whether any of the characters made them feel angry and why; and, finally, how they felt overall after watching both the direct and relational bullying scenarios (very happy, quite happy, neither happy nor sad, quite sad and very sad).