Computer-based “decision trap” games and Transfer

Final Assignment for:

EDUC 218

Prof. Dan Schwarz

Fall Quarter 2006

Greg Warman

STANFORDUNIVERSITY

T: (415) 816-3749

ABSTRACT

Successful application of knowledge regarding ‘decision traps’ logically requires both a belief in one’s vulnerability (to motivate effective learning and subsequent vigilance) and a keen awareness of the conditions under which such traps can occur. This logic was tested via a four condition experiment primarily focused on the learning and transfer of a well documented decision trap, Anchoring. Half the subjects (conditions A and B) played a simulation game that revealed their susceptibility to the Anchoring trap. The other half (conditions C and D) read a story about a third party who was ensnared by Anchoring. Conditions A and C were also given a second example of Anchoring in the form of a Word Problem which doubled as a transfer task. In this first measure, no support was found for the value of the game. However, a second transfer task was issued covertly the day following the in-session experiment. Although no support was found for the value of either the game or the introduction of the second example in the ‘spontaneous noticing’ of an Anchoring problem, those that played the game ultimately transferred three times as often as those who had read the story. A second unexpected finding supported by both the first and second transfer tasks was that matching the medium of the learning tool to the medium of the transfer problem significantly increased the likelihood of transfer.

INTRODUCTION

Anchoring is an heuristic that influences decisions under uncertainty. An act of the subconscious, Anchoringrefers to the human tendency to ‘anchor’ to an implicitly suggested reference point when making decisions with imperfect information. First, described by Amos Tversky (March 16th, 1937 – June 2nd, 1996) and Daniel Kahneman (born March 5th, 1934), Anchoring has become a favorite tool of business negotiation experts. By anchoring a counterpart to an extreme position early in a negotiation, a negotiator increases the likelihood of achieving a favorable deal.

Indeed, Anchoring has become a staple of business school negotiation courses and is featuredin numerous academic articles including the Harvard Business Review classic “The Hidden Traps in Decision Making” (Hammond et al., 2000). As the name of the article suggests, the Anchoring trap and its brethren are often taught from a defensive perspective. Students are made aware of these cognitive phenomenain the hopes that they will avoid the subconscious triggers that result in suboptimal decisions.

Unfortunately, simple awareness may be necessary but insufficient for students to learn and avoid these “traps”. Anchoring, for example, is an automatic act of the subconscious, and therefore is difficult to subjugate. Building effective conscious level defenses logically requires a belief in one’s personal vulnerability to motivate both effective learning and subsequent vigilance.Additionally, a strict Thorndikian approach to transfer suggests that learning about Anchoring in the context of haggling over the cost of an item will not lead to spotting disparate examples of Anchoring – for example, determining how much volunteer time one will donate to a charity – and acting accordingly. Anchoring can be found in multiple domains and forms, and therefore successful transfer requires one to be especially attuned tothe conditions under which it can occur.

One educational tool which holds promise for successfulAnchoringeducation is computer-based simulation games. Beyond its use of multimedia which has the potential to heighten the engagement of a learner, a simulation game allows first-hand experience with Anchoring and can demonstrate that one is not immune to this phenomenon. By providing such experience, the game taps into one of the strongest driving forces in human psychology – the desire to detect and avoid cheats. Sagarin et al. (2002) demonstrated that when people are shown they’re susceptible to the influence of advertising, it causes a significant increase in resistance.

Another potential means of mitigating the transfer challenge associated with teaching Anchoring is the use of multiple examples. In several experiments,Gick and Holyoak (1983) found that having more than one example of isomorphic problems increased subsequent transfer to analogous problems in the future. Additionally, Fried and Holyoak (1984) demonstrated that when given multiple instances of a class (i.e. in the case of the experiment, people were shown multiple examples of an artist’s work) a learner develop an estimate of the variability of application. Through the use of multiple examples, it is therefore possible that learners may more readily spot Anchoring in its various forms.

My study focuses on the effectiveness of simulation game induced first hand experience and multiple examples – both separately and in tandem – in teaching Anchoring for successful real-world transfer. For the reasons stated above, it seems reasonable that, separately, the approaches will increase the likelihood of transfer. Because the two techniques tackle different dimensions of the transfer challenge (i.e. belief of susceptibility AND understanding the breadth of application), I believe it is also reasonable that when used in tandem, the techniques will greatly increase the likelihood of transfer.

Finally, because business education ostensibly prepares students for real-world success, I’m particularly interested in exploring the degree to which these techniques allow for transfer beyond the safe confines of a laboratory. Therefore, this study was designed specifically to accommodate a real-world transfer measure of Anchoring.

This research has implications not only for developing optimal instructional techniques for teaching Anchoring to business students, but also for helping people from a variety of fields appreciate and avoid suboptimal decision-making.

GENERAL DESCRIPTION OF EXPERIMENT

The experiment tested three distinct but related hypotheses:

  1. If someone is shown they are susceptible to Anchoring rather than merely made aware that such a trap exists, they are more likely to spot and avoid this trap in the future.
  2. If someone is given two disparate cases of Anchoring rather than a single case, they are more likely to spot and avoid this trap in the future.
  1. If someone is BOTH shown they are susceptible to Anchoring AND given two disparate cases, they are MUCH more likely to spot and avoid this trap in the future.

Given these hypotheses, it was necessary to manipulate two variables in the experiment (see Table 1 for an overview of the independent variables and resulting experimental conditions). The first was ‘Susceptibility Demonstration’ which refers to how the concept of Anchoringwas introduced.Ithas two states:First-hand experience was where participants played a computer-based game that was likely to result in making an Anchoring error; and Second-hand account was where participants read a story (which was the same scenario as the computer-based game) about a fictional character who fell into an Anchoring trap. The second independent variable was “Number of Examples” which refers to the total number of Anchoring examples a participant experienced during the in-session component of the experiment (minimum of one – i.e. the Susceptibility Demonstration only – and a maximum of two).

To achieve several goals with the experimental design, it was necessary to introduce a second decision trap lesson – this time regarding “Sunk Cost”. Sunk Cost is often described colloquially as “throwing good money after bad” and refers to one’s tendency to continue to invest in underperforming alternatives when one is responsible for the initial investment. Although Sunk Cost’s inclusion reflected the same computer-based game versus paper story approach used for Anchoring, I decided that it was not possible (without greatly complicating the experimental design) to convincingly demonstrate via First Hand Experience a subject’s susceptibility to both traps. After discovering the initial scenario was a ruse, subjects were apt to focus on ‘beating the game’ in any subsequent scenarios. (Indeed, this concern was confirmed in the actual experiment when several participants confessed their intent to unearth ‘the trick’ going into the second scenario). Therefore, to focus my research I ensured Anchoring was always the first scenario in the game. Nonetheless, by including Sunk Cost as a second learning in the experiment, I introduced more opportunities for negative transfer in the subsequent measures.

Table 1 – Overview of Independent Variables

NUMBER OF EXAMPLES
Two
(Demonstration plus problem) / One
(Demonstration only)
SUSCEPTIBILITY DEMONSTRATION / First hand experience / Condition A / Condition B
Second hand account / Condition C / Condition D

To determine differences among the four conditions, I used measures on both in-session and post-sessiontransfer tasks. The second examples (problem statements) of both Anchoring and Sunk Cost doubled as my in-session transfer task. Using a procedure adapted from Gick and Holyoak (1983), I separated the number of ‘spontaneous noticing’ of these decision traps from ‘learning’ through the use of a hint. For my post-session measure, again I used the Gick and Holyoak procedure to determine how many subjects were ‘Spontaneously Noticing’ another example of anchoring.

Table 2 – Overview of Measures

Measure / Description
Transfer task 1 = in-session Word Problems
Spontaneous noticing / When solving the Word Problems did the subject articulate that they felt the problem was an example of Anchoring or Sunk Cost when the problem was indeed an example of said decision trap.
For subjects in conditions A and C there were two possible ‘Spontaneous noticing’, one Anchoring and one Sunk Cost. For subjects in conditions B and D, there were no possible ‘Spontaneous noticing’.
Learning (post hint) / After having attempted all of the Word Problems, a hint is given indicating it is possible some of the problems are examples of either Anchoring or Sunk Cost. A ‘Learning’ was recorded when the subject correctly identified and named the appropriate examples.
Assuming subjects in conditions A and C did not spontaneously notice the decision trap examples, there were two possible ‘Learnings’. Again for subjects in conditions B and D, there were no possible ‘Learnings’
Total / The sum of ‘Spontaneous noticing’ and ‘Learnings’
Spontaneous Negative Transfer / The total number of articulated misapplied decision traps while subjects attempted to solve the Word Problems
Each subject had three Word Problems and therefore three opportunities for ‘Spontaneous negative transfer’
Negative Transfer (post hint) / The total number of articulated misapplied decision traps while subjects attempted to solve the Word Problems
Assuming that subjects did not spontaneously negative transfer, each subject had three opportunities for ‘Negative transfer (post hint)’
Transfer task 2 = post-session confederate’s email
Spontaneous noticing / Does the subject report that they realized the confederate’s email was an example of Anchoring when they initially replied back to the confederate?
Learning (post hint) / After having received my email regarding the ruse, did the subject correctly identify the confederate’s email as an example of Anchoring?
Total / The sum of ‘Spontaneous noticing’ and ‘Learnings’
Negative Transfer (post hint) / After having received my email regarding the ruse, did the subject incorrectly identify the confederate’s email as an example of Sunk Cost.

To support my first hypothesis that First hand experience via a game provides better transfer than a passive, paper-based story, the in-session instances of ‘Spontaneous noticing’ of Anchoring for conditions A and B should minimally be better than the instances for D

(i.e. (A > D) AND (B > D))

To support my second hypothesis that two disparate examples provide better transfer than a single example, the in-session instances of ‘Spontaneous noticing’ of Anchoring for conditions A and C should minimally be better than the instances for D.
(i.e. (A>D) AND (C>D)).

To support my third hypotheses that a First hand experience via a game plus two examples is the superior instruction method for transfer, the ‘Spontaneous noticing’ performance of condition A on the post-session measure should be better than the performance of both B and C, which in turn should be better than D.
(i.e. A> (B AND C)>D)

METHOD

Design

The experimental design (see Figure 1) was structured with several goals in mind. First, to ensure manipulation of the ‘Susceptibility Demonstration’ variable in the Learning portion of the experiment, the design had to maximize the likelihood that subjects who played the simulation game would indeed be anchored by the scenario and thus discover their vulnerability to this decision-making trap. Therefore, to limit the possibility that participants would learn to ‘look for the trick’ prior to tackling the Anchoring scenario, the Sunk Cost scenario was consistently given second. The paper-based story version reflected this order.

Second to minimize the recognition of the Word Problems as a transfer task, participants were told that they would be playing through three scenarios and the Word Problems were framed as a tool for clearing short-term memory between the second and third scenarios (fortunately, the interface for the simulation game supported this ruse by clearly showing three distinct scenarios – see Figure 2). Similarly, participants in the Second hand account conditions were told they would receive three scenarios with Word Problems intervening between the second and third.

Figure 1 – Experimental Design. In the ‘Learning’ component, the simulation provided First hand experience with the decision traps whereas the paper-based story provided a Second hand account. The first transfer measure (Transfer 1) doubled as the 2nd example of Anchoring for conditions A & C.

Third, to ensure ample opportunity for negative transfer, all conditions received some Word Problems which were neither examples of Anchoring nor Sunk Cost. Additionally, it was possible for subjects in conditions A and C to misapply Sunk Cost to an Anchoring Word Problem and vice versa.

Fourth, following the Word Problems all participants were told the experiment was finished. Furthermore, participants were told that the design simply measured the difference in transfer performance on the Word Problems between those that played through the simulation game and those that had merely read about the scenarios. Participants were therefore not expecting subsequent contact nor a final transfer task the following day.

Finally, of critical importance to the experiment’s ability to explore real-world transfer was the second Transfer measure(‘Transfer 2’ in Figure 1) – an email from a confederate soliciting advice on a problem. The email concerned the number of hours the confederate should give to a volunteer group over the holiday period and included all of the elements of an anchoring trap – insufficient data and an implicit numerical reference point (20 hours).

Participants

Although the ‘Transfer 2’ task had the desired benefit of providing an indication of real-world transfer, it also significantly limited the scale of the experiment. All subjects had to see the confederate’s request as legitimate and therefore had to have a vested interest in her well being. Fortunately, the Stanford LDT Masters cohort provided such a subject pool. Twelve subjects were randomly assigned to the four conditions resulting in three subjects per. Because five of the subjects were also students in EDUC 218: Cognition and Transfer and could have an associated predisposition to transfer, I decided to adjust the assignment ensuring a relative balance of the 218 students across all four conditions (this counter-balance required switching only two subjects: one from condition A to condition D and vice versa).

Materials

In the learning component of the experiment, subjects in the First hand experience conditions played through a web-based game that challenged them to make decisions as the new General Manager (GM) of a professional hockey team, the Sockeyes. Subjects learn via an animated prologue that the team has just completed another disappointing season and that the GM has been fired. The subject has just been promoted from Player Scout to the GM position and is tasked with returning the team to respectability.

The game had three separate scenarios, each focused on a different decision-making trap (see Figure 2). The Sockeyes game was developed by my company for use as a public relations tactic with the Globe & Mail newspaper online edition ( Because over 4,000 readers played through the game and an overwhelming majority was ensnared by the decision-making traps, I was confident that my subjects too would discover their vulnerability. In ‘The Russian’ module, the participant tries to win the services of a young Russian superstar by making a salary offer. In the version used in the experiment, subjects were subtly anchored to a figure of $4M. In ‘The Sophomore’, the participants must determine how much latitude (in terms of number of games) to give an underperformer prior to attempting a trade. In the version used in the experiment, subjects are told the previous GM was responsible for bringing

the underperformer to the organization thus eliminating a sense of personal responsibility that could trigger the Sunk Cost phenomenon. It is important to note that the debrief of each scenario does not provide any special clues regarding the pervasiveness of Anchoring or Sunk Cost. Therefore, subjects are not primed to spontaneously notice the Word Problems as a transfer task. Additionally, to
Figure 2 – The Sockeyes Interface. The three scenarios focus on various decision traps. Specifically, ‘The Russian’ focuses on Anchoring; ‘The Negotiation’ focuses on Decision Framing; and ‘The Sophomore’ focuses on Sunk Cost. The game is available for free online at

ensure the Sockeyes context did not confer some special learning advantages, subjects in the Second hand account conditions were given the exact same scenarios, but on paper and written in the third person. For example, with The Russian module, the participants read a story about the new Sockeyes GM who, in his quest to secure the services of a Russian superstar, was subtly anchored to a figure of $4M.