A Worked Example of Virtual World Research

Chapter 23

A worked example of virtual world research

© Stewart Martin

This section presents a worked example of using a virtual world as a safe environment in which researchers and participants explore sensitive issues in educational research and education itself.[1] It reports how researchers can study the way young people see themselves in relation to the society in which they live. It is based on exploring virtual worlds as safe environments for the study of sensitive issues; in this case, of citizenship-related topics. Such a study might explore tensions between the internal image that individual young people have of themselves (their constructed identity) and the civic responsibilities and duties they are expected to discharge publicly, the loyalties which they are expected to defend and the behaviours which they are encouraged to display. A study of these elements of their ‘citizenship’ identity can explore: (a) individuals’ perceptions of, and attitudes towards, others; (b) their relationship to the dominant values, laws and legal systems in their country; (c) the social expectations of other individuals and groups; and (d) their experience of discrimination or disempowerment. Many countries have significant groups in their population with different cultural, religious and social values and associated backgrounds. These pluralistic societies are often concerned about social fragmentation and political (dis)engagement and, in some cases, the marginalization and radicalization of groups within them. Such a study would therefore be very likely to touch upon sensitive issues, explore areas of ‘dangerous knowledge’ (that knowledge which is unsettling and which could provoke social unrest) and highlight social and individual tensions.

Using a virtual world in this context, as with any other approach, would involve developing appropriate research questions, deciding on the data that would be needed to answer them, determining how the required data could best be collected and analysed and the most suitable methodology for the study.[2]

Suppose our research interest is in exploring what kind of society young people in secondary education wish to become part of, where the context is an economically developed, pluralistic democracy. Our research question might be ‘What do young people think it should mean to be a citizen in our country?’, assuming that its basis should be tolerance and the empowerment of individuals. We might also be interested in exploring how citizenship education might be developed. To address our research question, we would need to know what the sampled young people associated with citizenship and how they defined ‘tolerance’ and ‘empowerment’. What values might they invoke and how could we discover how their definitions and understandings of these sometimes sensitive and contested issues emerged and operated?

In this study, we might be interested in exploring the internal world of individuals and the way in which their values and feelings are externalized, projected, expressed and used, and how these drive behaviour. We might also wish to study how these can be shared and discussed in a safe environment. We therefore may decide that we need not only to measure and ask our sample about the values and feelings they associate with a tolerant, empowered citizenship, but also that we wish to observe how these are deployed in practice, during discussion and negotiation with others, especially in situations where the values and behaviours that are acceptable might be contested. We might be interested in tracking emergent values and beliefs within discussions about citizenship.

Hence, we decide to provide a number of scenarios featuring ‘moral dilemmas’ of citizenship for our participants to discuss, so that we can observe how groups explore and discuss these, how different values and assumptions are brought into play and the relative weights that are accorded to each. Observing real-world group interaction in such circumstances would be unlikely, we conclude, to be ethically acceptable or practically useful because of the significant risk during discussion of self (or group) censorship, the influence of individual sensitivity to group expectations, individuals’ anxieties about exposure when offering contentious views or arguments and the associated risk of conflict or the fear of it. We might reasonably feel that even if ethical concerns about placing subjects in environments in which they may fear personal threat or harm could be overcome, our data would be subject to too many uncontrolled variables for them to be useful. Given these difficulties, we can better conduct our experiment in a virtual environment and address and minimize many of the problems that could be foreseen with real-world groups. In a virtual environment, the participants would be able to project their feelings, values, arguments and evidence anonymously via their avatar and interact with others in live (synchronous) debate to explore issues that arise, but with fewer anxieties about potential conflict and greater confidence in sharing thoughts openly.

To operationalize our study and collect our data we therefore select an appropriate population for our study – in the example here we focus on young people in secondary education.[3] For practical reasons, we cannot involve every such young person in our study, so we wish to select a sample. If the sample is ‘fair’ (i.e. representative of the population) we will be able to generalize (apply) our findings reliably back to the wider population from which our sample was drawn. This is not straightforward because, in order to be representative of the population, the sample should include as many features as possible from the wider population, in the same proportions as they are found in that population. In our study we would need to include males and females in the right proportion, and, within these groups, also the range of ages, cultural backgrounds, religious beliefs and everything else we might consider to be important – each in the correct proportions. To prevent this from becoming unmanageable, we recognize that, unless we are going to have a very large sample, we will have to be measured in the claims made from the study. We will not be able to generalize fully (unless everyone in our defined population is included) and so we should narrow our study – say to secondary school pupils in a particular age group or year of study – and be explicit about the population to which we are trying to generalize. For us this means restricting the range of features covered in the sample and being clear that our conclusions will apply less and less widely as the contexts to which our findings are being applied become increasingly dissimilar to those sampled.

Our participants will complete a questionnaire which collects basic demographic information for use as the independent variables in our analysis and also measures appropriate citizenship-related attitudes. A questionnaire is developed from focus group discussions with individuals who are representative of our population. Such discussions identify relevant key themes and items associated with demographic factors, identity, citizenship, tolerance and empowerment. For each theme (i.e. ‘tolerance’ or ‘the rule of law’) we design a number of Likert-style questions to sample individual views and feelings, and then ask the focus group(s) to assess each of these items for relevance, clarity and ‘fit’ to each theme. After any necessary modifications, our questionnaire is completed by participants pre- and post-experiment.

Then we construct a virtual environment similar to Second Life but host this securely on our own computers and allow entry only via a secure password system so as to address concerns about access.[4] In our environment, we have an ‘inventory’ of relevant items developed from focus group discussions. Avatars access the inventory to select and use items, but every time they must attach a value and associated definition. Values and definitions are stored in a ‘Values Dictionary’, which is augmented over time as more avatars are created and discussion of scenarios proceeds.

Within our experimental environment, we ask participants to create an avatar and attach to it an associated biography/back story. Participants are asked to make their avatar depict how they see themselves as a citizen, using clothing and artefacts drawn from the inventory. Avatars are provided with a dwelling, which participants are asked to furnish and decorate with inventory items in order to create an ‘installation’ depicting who they are and what is important to them as a citizen. These data will be used to understand the items and attached values drawn upon and how they are defined in relationship to citizenship identity.

Groups of avatars are subsequently offered a series of contentious scenarios, each designed to highlight known areas of difference, tension or controversy within debates about citizenship. Participants’ avatars are invited to discuss these collectively and attempt to reach a consensus. Discussions are monitored discreetly to ensure civilized and courteous debates, but otherwise they are uncensored. Each avatar response must be accompanied by one or more attached values and the selection of an ‘emoticon’ that visually conveys to other avatars and their drivers the feelings attached to the response. Discussion continues until a majority view or impasse is reached.

Following each session, participants complete an online diary entry, unseen by other participants, where they talk about their feelings and views on the session. They are asked to reflect upon anything of significance they wish to identify, on suggestions for other scenarios for inclusion, on whether their experience would be likely to change their real-world behaviour, on how their experience within the experiment was different from the way in which they were normally taught about such topics in school, together with the advantages and disadvantages of each approach. These data are used to explore developing views and feelings as different scenarios unfold (e.g. views on how citizenship education might develop), and to discover topics for exploration during the experiment and exit interviews.

At the close of all the scenario discussions, a still image of each avatar is placed alongside its installation/dwelling together with its biography and attached values; each participant is then asked to review these and write a confidential note in their diary for each, in which they respond to these examples of citizenship identity. These data are used to gather reflections of others’ constructed citizenship identity for thematic and correlational analysis.

Participants also create a ‘Legacy Document’ in which they leave a statement for future users, which records what they have learned from taking part in the experiment and offers advice that they wish to give to new participants in the experiment. These data are used to improve subsequent experiences of participants and as an indicator of personal change.

Participants then complete again the initial questionnaire and undertake a real-world face-to-face exit interview in which they are asked to reflect on their experiences within the virtual world and to respond to the anonymous notes left by other avatars about their installation/dwelling and their created avatar. These data are used as participant confirmation for the extracted inferences and conclusions and to enrich the data set.

We may analyse our data via correlation, factor analysis or by applying Item Response Theory or thematic analysis, to discover relationships between the independent variables (demographics and initial views and values) and any changes in values, definitions and perceptions of what participants think it should mean to be a tolerant, empowered citizen in our country (dependent variables). Statistical and thematic analysis will provide us with measures of internal validity for our findings.[5] We will also use one or more control groups to which we will compare our findings from the experimental group, to help ensure that outputs from our analysis are not unduly skewed by the experience of the experiment and its features which are unrelated to the study focus or exposure to the environment we have created, and are not just outcomes that arise simply because individuals know they are being studied.[6]

We may wish to compare findings from several of our sources so that, by the triangulation of our data, we may strengthen internal validity and reliability in our conclusions. Demonstrating external validity is more difficult but we could provide increasing support for this by repeated use of our experiment with more samples from our population.

For further material on this project and its developments, please see the project website http://tinyurl.com/ch9xk5s (see also Martin, 2010).[7]

Further guidance on topics referred to in the book chapter

A. Data storage and presentation

Google docs: www.google.co.uk/docs/about

Dropbox: www.dropbox.com

Gapminder: www.gapminder.org.

B. Recording sound and video in virtual environments

These features are often provided within the software itself, but see examples at:

http://wiki.secondlife.com/wiki/Video_Tutorial/Record_voice_chat_and_sounds for guidance on how to record sound or live talk in Second Life, or

http://wiki.secondlife.com/wiki/Recording_Video for information on how to video activity.

These links also provide advice on ethical and legal implications.

C. Virtual and augmented reality equipment

Oculus Rift: www.oculus.com

PlayStation VR: www.playstation.com/en-gb/explore/playstation-vr

HTC Vive: www.wareable.com/vr/htc-vive-vr-headset-release-date-price-specs-7929

Gear VR: www.samsung.com/us/explore/gear-vr/?cid=AFL-hq-mul-0813-11000758

Microsoft Hololens: www.microsoft.com/microsoft-hololens/en-us?tduid=(303568b44ff558a31a855b964bd68800)(266696)(2142931)(05004p1jnrmt)()

Google Daydream: https://store.google.com/category/virtual_reality

Google Cardboard: https://store.google.com/product/google_cardboard.

D. Data capture from social networks

Many commercial companies invest a great deal of time and money acquiring and analysing data from social media. These companies often use expensive specialist companies to do this work but are sometimes prepared to offer substantially lower prices to students or academic institutions, so check to see if your institution has or can secure this. There are also some relatively straightforward methods of collecting social media data that can be used by researchers for free or at minimal cost.

Twitter:

Entrepreneur: www.entrepreneur.com/article/242830

The Chorus Project: http://chorusanalytics.co.uk (see also Tweetcatcher).

Facebook:

Graph: www.facebook.com/graphsearcher

Statista: www.statista.com/statistics/264810/number-of-monthly-active-facebook-users-worldwide (check to see if your institution provides access).

VennMaker (a mapping tool for collecting and analysing data in social network analysis): www.vennmaker.com/?lang=en.

Some straightforward guidance is also available on obtaining data from most social media softwares at WikiHow: www.wikihow.com/Main-Page.