A cognitive 1
Running head: Relative Personality Judgments
A Cognitive Model of Reference Group Effects on Personality Judgment
This is the pre-publication version of the following paper, the full version of which can be downloaded at:
Reference: Wood, A. M., Brown, G. D. A., Maltby, J., & Watkinson, P. (2012). How are personality judgments made? A cognitive model of reference group effects, personality scale responses, and behavioral reactions. Journal of Personality, 80, 1275-1311.
A cognitive 1
Abstract
This paper provides a cognitive model of reference group effects on personality judgment. Five experimental studies show that the same person is evaluated differently depending on how their behavior (a) ranks within a reference group, and (b) falls within the overall range of behavior shown by other reference group members. Rank and range position strongly predict how the individual will be judged. The results were invariant across stimulus type and response options (seven point Likert scale, 990 point allocation task, or dichotomous choice), and persisted even when participants were specifically told not to compare the target person to the reference group. Simulated occupational scenarios suggested that these effects would lead participants to give different sized bonuses and employ different people as a function of context. The results suggest that personality judgments are made using the same cognitive mechanisms as are used to judge psychophysical stimuli.
A cognitive 1
A Cognitive Model of Reference Group Effects on Personality Judgment
Understanding how people make personality judgments is essential to personality psychology, both to understand the cognitive processes underlying important social phenomena and to enable accurate use and interpretation of the self-report Likert scales that underpin many of the findings of the field. Since Hyman (1942), it has been understood that personality judgments are affected by the social context within which a person is being judged. Reference group (or “social comparison”) perspectives see personality judgments as arising from a comparison of an individual with a reference group of other people (e.g., Festinger, 1954; Heine, Buchtel, & Norenzayan, 2008; Heine, Lehman, Peng, & Greenholtz, 2002). Thus a person who starts conversation and feels comfortable around people 50% of the time may be viewed as quite extroverted by their colleagues if they are a librarian, but quite introverted if they are an entertainer (as they will respectively be relatively high and low on extroversion within the relevant reference group). Thus two people with identical personalities may be viewed very differently on the same personality dimension if they are being compared to different reference groups.
Although the existence of the reference group effect is well known (e.g., Festinger, 1954; Heine et al., 2008; Heine et al., 2002; Peng, Nisbett, & Wong, 1997), there is little understanding of how these relative judgments are made. Here we present a quantitative model of how people compare a target’s personality to a reference group, where the model makes use of the same principles as are used to judge such psychophysical stimuli as tone, weight, or size. Linking these processes to psychophysical judgments is in line with aims for a more “unified psychology” (Sternberg & Grigorenko, 2001), where universal psychological processes are identified and integrated between the basic level of cognitive science and more social and applied areas of psychology.
The expectation above that reference group theory should be able to predict an individual’s rating of a target personality is subject to the qualification “all other factors held constant”, as personality judgments would normally be influenced by many other processes including halo effects (Thorndike, 1920), similarity of the target to self (Markus, Smith, & Moreland, 1985), and aspects of survey design (Tourangeau & Rasinski, 1988), to mention just a few important influences on personality judgment. However, in principle, if these factors were held constant (for example through experimental design), a clean test of the reference group predictions should be possible.
Reference Group Theory in Personality Research
The study of reference group effects on personality judgments has a long history of research and a sizable empirical base (e.g., Festinger, 1954; Heine et al., 2008; Heine et al., 2002; Peng et al., 1997). However it would be to misread reference group theory to suggest that personality judgments are normally inaccurate, when in reality people are actually surprisingly good at judging each other’s personalities (Funder, 1995). The validity of personality judgments are supported, for example, by the high stability of personality ratings over time and agreement between self- and peer ratings (McCrae & Costa, 1987; McCrae & Costa, 1990). Such findings would, however, be expected from reference group theory, which would predict that personality judgments are valid when two people share the same reference group. For example, correlations between self-ratings and the peer-ratings from a friend would be expected to converge, as both people would likely share the same reference group. Similarly, personality ratings should remain the same over time, as the person’s reference group would not generally change much after a certain age (indeed, personality is much less variable after age 30, Srivastava, John, Gosling, & Potter, 2003). Research designs associating multiple self-ratings (e.g., associating a given personality trait with happiness) should be equally unaffected, as the person would likely use the same comparison group when making both ratings, providing an accurate indication of the association between the two constructs. In trying to understand a person’s personality, knowing the percentage of time a target behaves extrovertedly is not very useful information when presented in isolation. Rather one needs to know how this person compares to the social context of other people. If a person moves, for example, from an individualist to a collectivistic culture, simply classifying almost everyone as highly collectivistic would not help decision making, as it would allow little discrimination between individuals. It would be far more adaptive to form a new context of personality, and judge the least collectivistic person as individualistic, even if in different cultural contexts they would not be seen as such. Using reference groups to make personality judgments is normally an adaptive and informative process, which can provide accurate information for decisions based within the same social context.
Problems, however, emerge when comparing the personality ratings of people who use different reference groups. Heine et al. (2002) asked participants to complete measures of individualism vs. collectivism who were either (a) Canadian students with experience living in Japan, or (b) Japanese students with experience of living in Canada. Three versions of the questionnaire were produced, with items either asking participants specifically to (a) compare themselves to North Americans, (d) compare themselves to Japanese people, or (c) use no cultural referent and simply complete the questionnaire as per normal instructions. Contrary to the opinion of a panel of cultural experts, there were weak and largely non-significant differences between Canadian and Japanese people on collectivism when participants used the normal questionnaire instructions or rated themselves relative to people from their own culture. However, there were large differences when participants rated themselves relative to people in the other culture. Thus people appear able to switch reference group easily and provide different personality self-ratings depending on which reference group they are using. This research is consistent with findings that increased contact with other cultures leads to greater agreement about cultural differences (Triandis & Vassilio, 1967). Such effects may also explain why cultural differences in personality normally appear when using behavioral indices but not always when comparing mean levels of self-reported personality across different cultures (Heine et al., 2008; Peng et al., 1997).
Cognitive Basis of Relative Judgments
Central, although often implicit, in accounts of reference group effects is a cognitive process whereby a rater compares a target personality to a reference group. How this process occurs is not known. Previous accounts of reference group theory have largely not addressed this point, and where it has been addressed the explanation has been some form of Helson’s (1947) adaptation level theory (e.g., Wilson & Gilbert, 2008). This approach to relative judgment suggests the target is compared to the mean of the reference group, so that the rating of an individual’s extroversion would depend, for example, on how frequently the individual behaved extrovertedly compared to the average behavior of the reference group. Such an approach is intuitive but contrasts with independently motivated expectations derived from well-established models of judgment as described below. Specifically it would not make use of the three additional and readily available pieces of information at the rater’s disposal; (a) how the individual’s behavior compares to the least extroverted person in the group, (b) how the behavior compares to the most extroverted person, (c) how the individual ranks within the reference group.
The first two pieces of information can be combined into the range principle; where the target’s behavior falls on the overall range of the reference group’s behavior. Formally,
Range position = (Si – Smin) / (Smax – Smin) (1)
where Smin and Smax are respectively the lowest and highest values in the reference group, and Si isthetarget’s behavior. The range principle is supported by social psychology research into anchoring, where people are pay attention to the highest or lowest value in a context (Stewart, 2009; Tversky & Kahneman, 1974).
The third piece of information, the individual’s rank standing in the reference group, is likely to be particularly essential to judgment. Both humans and other animals are very sensitive to rank position (e.g., Allen & Badcock, 2003; Gilbert, 2006; Grant et al., 1998; Yeh, Fricke, & Edwards, 1996). Whilst people are rather bad at estimating the actual size of stimuli they are good at judging the size of one stimuli relative to another (e.g., Stewart, Brown, & Chater, 2005), and encoding and manipulating frequencies (e.g., Gigerenzer & Hoffrage, 1995). The recent decision by sampling (Stewart, Chater, & Brown, 2006) account of relative judgment is based solely on rank and specifies the lower-level cognitive processes whereby such judgments are made. Formally, judgment depends on the relative ranked position of the stimulus in the set:
Rank position = (ri – 1)/(N-1) (2)
where ri is the ranked position of the stimulus in the context, and N is the number of stimuli in the set.
Range Frequency Theory (Parducci, 1965, 1995; Wood, Brown, & Maltby, in press) suggests that relative judgments are based on a weighted average of the range and rank principle:
Judgment of stimuli = wRange position + (1-w) Range position (3)
where w is an empirically derived weighting constant. RFT originally developed in psychophysics, where it was shown to provide highly accurate prediction of how people would judge such stimuli as weights (Parducci, 1963), length (Parducci & Marshall, 1961), size (Parducci, Calfee, Marshall, & Davidson, 1960; Parducci & Perrett, 1971; Parducci & Wedell, 1986), and sweetness (Riskey, Parducci, & Beauchamp, 1979). Somewhat later, RFT has also been shown to explain how a wide variety of more social judgments are made, including (a) fairness of wages and taxation (Mellers, 1982, 1986), (b) satisfaction with wages (Brown, Gardner, Oswald, & Qian, 2008), (c) other people’s psychopathology (Wedell, Parducci, & Lane, 1990), likability (Wedell, 1994), and attractiveness (Wedell, Parducci, & Geiselman, 1987), (d) personal happiness (Smith, Diener, & Wedell, 1989; Wedell & Parducci, 1988), (e) body image satisfaction (Wedell, Santoyo, & Pettibone, 2005), (f) satisfaction with performance (Mellers & Birnbaum, 1983), (g) price perception (Niedrich, Sharma, & Wedell, 2001; Niedrich, Weathers, Hill, & Bell, 2009) and (g) gratitude following aid (Wood et al., in press).
These processes have not, however, previously been used to account for reference group effects on personality judgment. We suggest that people will compare a target personality to the reference group using the rank and range principles, combined into a single judgment as predicted by RFT. Further, we suggest that when a rater’s reference group and the target’s position in the reference group are known, RFT can be used to predict what personality judgments the rater will make.
Overview of Studies
Five studies were conducted, each of which involved three procedures: (a) providing participants with a reference group of people varying in behaviors associated with a personality trait (e.g., with varying likelihood of starting conversations), (b) experimentally manipulating the placement of a target person’s behavior within the group, and (c) testing whether the target’s rank and range position within the group affected how they were rated on a personality trait (e.g., extroversion). Studies 1 and 2 performed these three procedures specifically to test the rank principle. Study 1 focused on extroversion as an initial test of the hypothesis and Study 2 expanded the results to the remaining Big Five traits of agreeableness, conscientiousness, openness, and neuroticism (Costa & McCrae, 1995; Goldberg, 1992). Study 3 used a more precise methodology, allowing mathematical modeling of the results to test how well personality judgments can be predicted from the target’s rank and range position within the reference group. This modeling was also used to test whether both the rank and the range principle are necessary to predict personality judgments. Study 4 experimentally tests the range principle, examining whether the results persist when studying behavioral intentions, and when telling people specifically not to make relative judgments (through asking participants to rate each person singly and objectively without regard to the reference group). Study 4 also explored how the results may influence behavior in a workplace. Study 5 showed that the results generalize to a dichotomous decision – deciding on whom to employ from a group of potential candidates. Taken together the studies were designed to show that rank and range principles explain reference group effects on person perception and that the results are invariant across (a) stimulus type, (b) response option (7 point anchored scale, 990 point scale, or dichotomous choice), and (c) task instruction. The studies provide 50 replications of the basic finding, demonstrating the reliability of the model across various different personality traits and situations.
Study 1
Method
Participants
Participants were a convenience sample of 53 adults (30 female) aged between 18 and 78 years (M = 33.75, SD = 15.59) who volunteered to take part in the study, without payment. Most participants described themselves as White (90.6%). Participants were randomly allocated to one of two groups (25 to 28 per group).
Design and Procedure
All participants read descriptions of nine different people. Each description focused on how likely the person was to exhibit characteristics associated with extroversion. Four characteristics were chosen: (1) start conversations around people, (2) feel comfortable around people, (3) have little to say (indicative of introversion), and (4) be quite around strangers (indicative of introversion). The characteristics were taken from the extroversion sub-scale of the International Personality Item Pool - Five Factor Model (IPIP-FFM scale) (Goldberg, 1999; Goldberg et al., 2006) which was developed based on the Goldberg’s (1992) Big Five Markers. These characteristics appear to reliably and validity assess extroversion, with the IPIP-FFM extroversion scale showing high internal consistency (= .87), 3-week test-retest reliability of r = .89 (Donnellan, Oswald, Baird, & Lucas, 2006), and convergence with several behavioral indices of extroversion, including interpersonal behavior (r = .45) and leadership (r = .48) (Oswald, Schmit, Kim, Ramsay, & Gillespie, 2004). Selecting the characteristics from an established and psychometrically valid inventory increases confidence that the characteristics genuinely represent extroversion.
The four characteristics were put together into a coherent sentence: “In general, [name] will [probability word] start conversations and feel comfortable around people. There is [probability word] that she will have little to say or be quiet around strangers”. The same sentence was presented for each of the nine people, with the exception of each person having a different name (counterbalanced between the conditions to avoid any name association effects) and probability word that indicated how likely they were to behave extrovertedly. Descriptions of people were presented on a single A4 sheet and participants were asked to read these carefully, spending as long as they wished doing so. After they had finished, they separately rated each of the nine people on the extroversion sub-scale of the Mini International Personality Item Pool (Mini-IPIP) scale (Donnellan et al., 2006), which provides four items assessing extroversion which are rated on a 1 (“very inaccurate”) to 5 (“very accurate”) scale. This scale was developed as a shorter version of the IPIP-FFM from which the extroversion characteristics used in the descriptions were drawn (we ensured there was no direct overlap between the IPIP-FFM descriptions used in this study and those used to developed the Mini-IPIP scale). The use of these two scales was designed to ensure that both the descriptions and the questions were assessing the same construct; empirically this is the case with the IPIP-FFM and Mini-IPIP correlating at r = .97 (Donnellan et al., 2006). The Mini-IPIP extroversion scale has good properties, with 3-week test-retest reliabilities of r = .87 and high convergence with other common extroversion measures (e.g., r = .81 with the Big Five Inventory [John & Srivastava, 1999])(Donnellan et al., 2006). The Mini-IPIP extroversion scale was coded according to normal instructions so that each participant had provided a single extroversion rating for each of the nine people (9 scores in total).
Experimental Manipulation
The probability words used to describe each of the nine people differed between the two groups and constituted the experimental manipulation. Previous work has established the percentages that people naturally with certain probability words. Specifically, “no chance” is associated with 0%, small chance with 14.43%, even chance with 50%, very likely with 81.53%, and definitely with 100% (Stewart et al., 2006). On this basis, we were able to create two groups of people that varied in how extroverted members were; technically the groups respectively had unimodal or bimodal distributions of extroverted behavior. These groups were designed to manipulate the rank position of certain “target” persons who appeared in both groups. This procedure follows the standard RFT paradigm (e.g., Brown et al., 2008; Wood et al., in press).