1
Subjective measures of unconsciousness
Running head: SUBJECTIVE MEASURES OF UNCONSCIOUSNESS
Subjective measures of unconscious knowledge
of concepts
Eleni Ziori
University of Ioannina
and
Zoltán Dienes
University of Sussex
Correspondence concerning this article should be addressed to Eleni Ziori (Department of Psychology, Faculty of Philosophy, Education & Psychology, School of Philosophy, University of Ioannina, Dourouti, 451 10, Ioannina, Greece). Electronic mail may be sent to: . Tel: +30-26510-95761. This work was completed in partial fulfillment for the requirements of a D.Phil thesis at the University of Sussex, and was funded by a Scholarship from the Greek State Scholarships Foundation (IKY).
Abstract
This paper considers different subjective measures of conscious and unconscious knowledge in a concept formation paradigm. In particular, free verbal reports are compared with two subjective measures – the zero-correlation and the guessing criteria – based on trial-by-trial confidence ratings (a type of on-line verbal report). Despite the fact that free verbal reports are frequently dismissed as being insensitive measures of conscious knowledge, a considerable bulk of research on implicit learning has traditionally relied on this measure of consciousness, because it is widely regarded as almost self-evident that the content of any conscious state that is intentional and conceptual can be expressed verbally. However, we found that the most recently developed subjective measures based on trial-by-trial confidence ratings provided a more sensitive measure of conscious and unconscious knowledge than free verbal reports. In a complimentary way, the qualitative pattern of the free report and the confidence measures were similar, providing further evidence for the validity of the latter.
Subjective measures of unconscious knowledge of concepts
Many studies have shown that implicit learning results in knowledge that is difficult to express verbally. It is sometimes concluded that therefore the knowledge is unconscious (e.g. Berry & Broadbent, 1984;Lewicki, Hill & Biziot, 1988; Reber, 1967). However, free verbal reports have been thought of as an unreliable source of evidence of conscious or explicit knowledge (e.g. Berry & Dienes, 1993; Shanks & St. John, 1994). This research attempts to shed more light on this controversy as well as on the usefulness of subjective measures as measures of conscious awareness by comparing free verbal reports with two other subjective measures of consciousness that are based on confidence ratings. To do so, the present research uses one of the tasks that researchers have used to study implicit learning, namely a category learning task.
Implicit category learning is assumed to occur when learning proceeds in the possible absence of any intention to learn and in such a way that people acquire knowledge they are not fully conscious of and thus cannot express verbally (e.g. Frick & Lee, 1995; Posner & Keele, 1968; Reber, 1967, 1976). The term “implicit” is commonly used to refer to implicit memory (e.g. Bowers & Schacter, 1990; Schacter, 1987; Schacter, Bowers, & Booker, 1989). According to Graf and Schacter (1985, p. 501), “implicit memory is revealed when performance on a task is facilitated in the absence of conscious recollection of previous experiences”. Thus, implicit memory refers to the influence that a previous event has on performance without one being consciously aware of the influential event. By contrast, implicit learning refers to the acquisition of knowledge about the structural relations among stimuli, without being conscious of that knowledge (see Berry & Dienes, 1991; Dienes & Perner, 1999; and Seger, 1994 for relations between the two research areas). The term “implicit learning” was introduced by Reber (1967). He asked subjects to memorize strings of letters, where, unbeknownst to subjects, the order of letters within the string was constrained by a complex set of rules (i.e. an artificial grammar). After a few minutes of memorizing strings, the subjects were told about the existence of the rules (but not what they were) and asked to classify new strings as obeying the rules or not. Reber (see Reber, 1993, for a review of his work) found that subjects could classify new strings 60-70% correctly on average, while finding it difficult to say what the rules were that guided their performance. He argued the knowledge was unconscious.
Over the years, there has been considerable debate about the unconsciousness or implicitness of knowledge with some researchers even questioning the usefulness of an explicit/implicit distinction (e.g. Dulany, 2003; Shanks & St. John, 1994; Tunney & Shanks, 2003). In an attempt to resolve this controversy, researchers have proposed several different criteria of the unconsciousness of knowledge.
One such criterion is the inaccessibility of knowledge to free report. However, starting with Dulany, Carlson, and Dewey (1984), critics have been unhappy with free report as an indicator of unconscious knowledge. Free report gives the subject the option of not stating some knowledge if they choose not to (by virtue of not being certain enough of it); and if the free report is requested some time after the decision, the subject might momentarily forget some of the bits of knowledge they brought to bear on the task (Berry & Dienes, 1993). Even if people cannot recall a piece of knowledge in a given period of time, they may be able to recall it later on, if they are given a second chance (Erdelyi & Becker, 1974). Similarly, Shanks and St. John (1994) argued that participants’ inability to report the rules of an artificial grammar, for example, is not evidence of implicit or unconscious knowledge. Instead, it may be that knowledge is accessible to consciousness, but has to be specifically asked for to be elicited. What the subject freely reports depends on what sort of response the subject thinks the experimenter wants. For example, if the subject classified on the basis of similarity to memorized exemplars, but thinks the experimenter wants to hear about rules, then free reports may not be very informative accounts of the subject’s conscious knowledge. That is, a test must tap the knowledge that was in fact responsible for any changes in performance (the information criterion of Shanks and St John, and the problem of correlated hypotheses highlighted by Dulany, 1968). One way around the information criterion is to use confidence ratings, because then the experimenter does not need to know exactly what the knowledge is that participants use. Any knowledge the participant is conscious of using, no matter what its content, should be reflected in the participant’s confidence. This is a major benefit of the use of confidence measures of conscious knowledge.
Chan (1992) elicited a confidence rating in each classification decision, and showed subjects were no more confident in correct than incorrect decisions. Dienes, Altmann, Kwan, and Goode (1995), Dienes and Altmann (1997), Allwood, Granhag and Johansson (2000), Channon et al (2002), Tunney and Altmann (2001), and Dienes and Perner (2003) replicated these results, finding some conditions under which there was no within-subject relationship between confidence and accuracy. Subjects could not discriminate between mental states providing knowledge and those just corresponding to guessing; hence, the mental states were unconscious. Kelly, Burton, Kato, and Akamatsu (2001), and Newell and Bright (2002) used the same lack of relationship between confidence and accuracy to argue for the use of unconscious knowledge in other learning paradigms. The method has an advantage over free report in that low confidence is no longer a means by which relevant conscious knowledge is excluded from measurement; rather the confidence itself becomes the object of study and can be directly assessed on every trial.
Dienes and Berry (1997) urged the use of trial-by-trial confidence ratings in measuring conscious and unconscious knowledge. Such measures, along with free report, are called subjective measures because they measure what states of knowledge the subject thinks he or she is in. By these measures, people’s knowledge is said to be unconscious when they lack metaknowledge. That is, unconscious knowledge is defined as being in an occurrent knowledge state one does not know one is in (Cheesman & Merikle, 1984; Pierce & Jastrow, 1884).
If one knew one was in a certain knowledge state one could express that knowledge on a forced choice test on the content of that knowledge (a so-called direct test). So sometimes people measure conscious knowledge by the use of a forced choice test about the state of affairs in the world that the knowledge is about. Such tests are called objective tests; they are about objective worldly affairs. Failure on an objective test indicates the person does not have conscious knowledge (so if in addition an indirect test, e.g. a liking rating, indicated the person had knowledge, we could conclude that the knowledge was unconscious, cf Kuhn & Dienes, submitted). However, passing a test about states of affairs in the world can be achieved not only by conscious knowledge, but also by unconscious knowledge about the world. Indeed, it is sometimes difficult to see why unconscious knowledge should not apply when objective tests are used. Subjective tests are tests about the subjective state the participant is in; that is, they directly test whether the participant is aware of the knowledge state they may be in. Subjective tests assess the presence or absence of conscious knowledge more directly than objective tests do, and can be used when objective tests indicate the participant has some (conscious or unconscious) knowledge.
In the present research, we will compare the free verbal report with two other subjective criteria: The zero-correlation and guessing criteria. The “zero correlation criterion” is a short hand expression for the “zero confidence-accuracy relationship criterion” (Dienes & Berry, 1997). When the subject makes a judgement, ask the subject to distinguish between guessing and different degrees of knowing. If the judgment expresses conscious knowledge - on those cases when it is knowledge and not guessing - then the subject should give a higher confidence rating when she actually knows the answer and a lower confidence rating when she is just guessing. In other words, conscious knowledge would prima facie be revealed by a relationship or correlation between confidence and accuracy, and unconscious knowledge by no correlation (the person does not know when she is guessing and when she is applying knowledge). The other subjective measure of unconscious knowledge we will use is the guessing criterion (Dienes et al., 1995; see also Dienes & Berry, 1997). In order to estimate unconscious knowledge in terms of the guessing criterion, we take all the cases where a person says they are guessing, and examine whether they are actually demonstrating the acquisition of some knowledge, that is whether the percentage of guesses that are correct is greater than a chance level. This is the criterion that is satisfied in cases of blindsight (Weiskrantz, 1986, 1997). The person insists they are just guessing, but they can be discriminating up to 90-100%correct. So, based on the guessing criterion, the knowledge in blindsight is unconscious.
It is very difficult to obtain measures that are sensitive only to conscious or only to unconscious knowledge (Merikle & Reingold, 1991). An advantage of the combination of the two metaknowledge criteria, the zero correlation and guessing criteria, is that they allow for a measured mix of implicit and explicit knowledge in any one experimental condition (cf Jacoby, 1991). In particular, a percentage of correct guesses that is found to be reliably greater than chance in the guessing criterion analysis provides evidence of implicit knowledge without, however, excluding the possibility that explicit knowledge might exist on other trials. By contrast, a zero-correlation criterion analysis that results in reliably greater confidence for correct than for incorrect responses indicates the presence of some explicit knowledge without ruling out the possibility that some implicit knowledge might exist on the same trials. A lack of metaknowledge may be related to people’s inability to report verbally what they have learned, because people who lack metaknowledge may not know what specific questions to ask themselves to elicit their own knowledge (Dienes & Berry, 1997). We will investigate whether the zero correlation and guessing criteria are more sensitive measures of conscious and unconscious knowledge than verbal reports, and thus whether they alleviate the insensitivity problem of free reports. Free reports have sufficient face validity as measures of conscious knowledge, that for a long time they were the only measure of conscious knowledge, from Smoke (1932) to Reber (e.g. 1967) and Lewicki (e.g. 1986). Given the face validity of free reports, we will explore whether the other subjective measures (i.e., the metaknowledge measures) produce qualitatively similar patterns of results as free reports, which would provide converging evidence for the metaknowledge measures’ validity.
One might argue that the guessing criterion faces the same response bias problem as do verbal reports. For instance, a person may say they are guessing when they actually have some awareness of their knowledge (e.g. Merikle & Reingold, 1992). It should be noted that, in the present study, participants were clearly instructed that a guess meant their response was based on no information whatsoever. However, a direct way of testing the sensitivity of the guessing criterion is to show that the measure provides the pattern of results expected by a theory of conscious and unconscious knowledge. According to the well-known theory that the acquisition of conscious knowledge requires the use of working memory, a working memory secondary task would be expected to interfere with the acquisition of conscious knowledge while leaving the acquisition of unconscious knowledge unaffected. Such a finding would provide evidence for the validity of the subjective measures (Dienes, in press). Therefore, we will use the dual-task methodology as a means of testing the validity of the subjective measures.
Implicit learning of concepts has mostly been studied using meaningless and highly artificial material, such as dot patterns, or artificial grammars. However, as Whittlesea (1987) argues, the use of highly artificial and arbitrary stimuli in concept formation studies is not informative about the formation of natural concepts. Thus, the present research used stimuli that were more similar to real categories and therefore more appropriate for studying natural category learning than highly arbitrary stimuli devoid of any meaning. Moreover, the present stimuli allow one to test the interaction of prior knowledge with empirical learning, which, as the recent view of concepts argues, characterizes the learning of concepts in the real world (see e.g. Medin, 1989; Murphy & Medin, 1985; Heit, 1994, 1997). Further, as Mathews and Cochran (1998) have pointed out, a disadvantage of implicit tasks that use meaningless and highly artificial stimuli is that they are rather boring, tiresome and detached from people’s interests.
We will use data on concept learning from Ziori and Dienes (submitted). In their experiments, half the participants learned the categories under a dual-task condition thought to discourage explicit learning (see e.g. Roberts & MacLeod, 1995; Jiménez & Méndez, 1999; Waldron & Ashby, 2001; contrast Shanks & Channon, 2002), whereas the other half of participants learned the categories under a single-task condition meant to favour explicit learning.
Ziori and Dienes (submitted) used Murphy and Allopenna’s (1994) concept learning paradigm, which revealed a strong facilitative effect of prior knowledge on concept learning. The main aim of Ziori and Dienes’ study was to investigate the relationship of this effect with implicit and explicit concept learning. During training, participants classified category exemplars with feedback. All exemplars consisted of descriptions taken from familiar everyday domains (i.e., animals, vehicles, and buildings). However, in one condition, the features were meaningful but completely unrelated to each other, whereas in the other condition, the same features were combined such that they were interrelated and integrated by prior knowledge. In a test phase that followed training, all participants had to categorize only the individual features of the category exemplars without feedback, and with confidence ratings given on each trial. Finally, participants had to freely report which features went with which category.
The aim of this article was to compare the guessing and zero correlation criteria with free report. With respect to free report, a high correlation between the knowledge expressed in these reports and participants’ knowledge in the test phase would provide evidence of explicit knowledge, since both estimates would be measures of the same knowledge. By contrast, a lack of relationship between the knowledge expressed in the free reports and the knowledge measured in the test phase would indicate the presence of implicit knowledge. As mentioned above, the secondary task was used to test the validity of the subjective measures. If the current subjective measures were valid measures of unconscious knowledge, then we would expect the concurrent task to interfere only with explicit and not with implicit knowledge.
With respect to the effect of prior knowledge on implicit and explicit knowledge, we could not make any clear predictions as to whether prior knowledge would affect one or both types of knowledge. Some researchers (e.g. Hayes & Broadbent, 1988) argue that implicit learning is an unselective and passive process of learning with no room for any interpretive processes based on declarative knowledge. On the other hand, it has been suggested (e.g. Frick & Lee, 1995; Pothos, in press; Sun, Merrill & Peterson, 2001; Sun, 2000) that implicit learning may interact with prior knowledge and expectations. For example, Pothos (in press, Experiment 2) showed that explicit expectations about stimulus structure facilitated the acquisition of implicit knowledge when the types of information on which the expectations and the knowledge acquired in an implicit learning task (i.e., an AGL task) were likely to be the same (see also e.g. Cleeremans, 1993; Dienes & Fahey, 1995; Mathews et al. 1989 for evidence of a synergy between explicit and implicit knowledge). People may direct their attention towards the features or themes highlighted by prior knowledge (cf. e.g. Murphy & Wisniewski, 1989; Pazzani, 1991; Wisniewski, 1995) facilitating the acquisition of both explicit knowledge (through analytic, rule-based learning) and implicit knowledge (through e.g. exemplar-based or associative learning).