Chapter 4
Chapter 4.Conscious contents and nonconceptual representation
4.1The role of conscious experience in the study of perception and cognition
4.2What subjective experience reveals about psychological processes
4.3The special case of the experience of space
4.3.1The status of our phenomenal experience of space
4.3.2The argument for nonconceptual representation of space
4.4The phenomenal experience of mental imagery
Chapter 4.Conscious contents and nonconceptual representation
In the last chapter I raised the issue of nonconceptual representation as a form of representation that does not involve concepts and therefore does not enter into beliefs and thoughts about the perceived world. Nonconceptual representations have been widely discussed in philosophical circles for a number of reasons. One reason is the basic problem that we have encountered in several places earlier in this book: The need for a way to get information from its distal causes through proximal effects (e.g., the retinal image) to perceptual beliefs, which are by definition thoroughly conceptual. The interface is thought to involve a type of information-bearing state that is more complete and finely structured than beliefs, but which nonetheless qualifies as being a representation. The primary evidence for such a form of representation is the disparity between appearances and beliefs (which is the hallmark of a perceptual illusion) or between the concrete, quantitative, panoramic, detailed and fine-grained nature of our conscious experiences (particularly our visual experiences) and the relative coarseness and abstractness of our thoughts, beliefs and recollections. When we look at an ambiguous figure, such as a Necker cube, something changes over time as we watch, and that something is neither the physical stimulus nor our beliefs about what we see. It is what we refer to as the appearance of the figure or how we consciously experience it. Since the content of our experience seems to be something distinct from what we believe about the figure and from the actual projection of the figure on our retinas it would seem that we need a theoretical vehicle for that content, and such a vehicle would be some sort of a representation. Similarly when we examine a scene, what we experience is very different from, on the one hand, what we independently know to be the information that enters the brain and, on the other hand, from what we think we could capture in terms of the vocabulary of concepts we are likely to have. In the case of the former discrepancy the data are very clear that the incoming information is highly incomplete and has a narrow scope compared with how we experience it (this point will be discussed later in connection with the special case of the experience of space, and is illustrated in Figure 41). The argument from the richness of experience compared with the relative poverty of our conceptual resources depends both on how we characterize experience and what we think are the conceptual resources of the mind. But even without considering the fine points of what are reasonable bounds on our conceptual apparatus, it seems clear that we are unlikely to have as many distinct concepts for, say, colors, as there are colors that we can discriminate. Certainly if we consider the number of color terms in a language we find that they are highly limited [about 11 monoleximic words, see \Berlin, 1969 #1785]. Yet we can distinguish well over a million different colors[Halsey, 1951 #1784], so we are unlikely to encode each of these as a separate concept or code.[1]
Notwithstanding such plausible arguments for nonconceptual representation, there are several questions that need to be considered and several tacit assumptions that need to be exposed before the hypothesis that there is a nonconceptual representation that corresponds to conscious contents is taken as established. The most contentious of these is the assumption that conscious content corresponds to a level of representation in an information-processing or functional view of the cognitive system. I take up this and other issues in the next few sections.
4.1The role of conscious experience in the study of perception and cognition
Cognitive science, and particularly vision science, has had a deeply ambivalent relation with conscious experience. On one hand, the way things appear or what they look like has always constituted the primary data of the science. When one thing looks bigger in one condition than in another or when something looks to be moving faster under one condition than another, or when colors appear different under one lighting condition than another, these are considered primary data to which theories are expected to respond. On the other hand, the content of experience has also proven to be one of the most misleading sources of evidence because it is not neutral with respect to theories. What explanations appear most natural is determined in part by the way we describe our experiences and conversely, the way we describe our experiences, even to ourselves, depends on what theories we hold. In fact, the way we describe the experience often caries with it the implication that the experience itself explains the data – that the experience of X is explained by alluding to the experience of Y (e.g., the reason it takes longer to report details from a mental image that is experienced as being small is that the details are harder to see). Phenomenological evidence has also tempted people to the view that vision provides a dense manifold of panoramic information. It has suggested that mental images consist of pictures examined by an inner eye in what Dan Dennett has called a Cartesian Theatre [Dennett, 1991 #976].
It also has encouraged direct perception theories which claim that we pick up information about aspects of the world that are prominent in our experience, such as whether the things we see are suitable for certain purposes – from eating to sitting on (suitability is often referred to as an “affordance”). It is true that we do not see patterns of light and shadow and patches of color, we see familiar things such as tables and chairs and people. We also see people on television screens and in movies. We never see just the front surface of objects, we see entire objects and we see them as certain kinds of objects, such as our car or the bicycle we used to ride in our youth. How big something appears or how bright or even what color it is depends on the context how we see other aspects, such as how far away, whether in direct light and so on. These adjustments, called constancies, are made prior to our experience, so what we experience is sometimes thought of as a low-level sensory pattern, but it is in fact more often a high-level reconstruction of the distal objects. The experience we have when we see objects often involves emotional memories; we see them as parts of our past. Yet a theory that takes these sorts of observations as the givens of perception, as the starting point of a theory (as in Gibson’s direct realism theory), fails to take even the first step towards an explanatory theory of the mechanisms involved in vision,a theory that might eventually make contact with neuroscience. Science always needs a way to objectify the problem we want to explain, to make these phenomena strange. One way to do this is to ask what it would take for a computer to demonstratesome of themundane perceptual skills that we exhibit. While this does not guarantee that we will avoid circular or dead-end approaches, it is one of the ways that we can ensure a detachment from the temptations of reifying phenomenal experience. But why do we need to be detached from reifying phenomenal experience? The answer is not a priori, it is just that in the past such experience has proven to be the most misleading source of evidence and inspiration.
As a source of evidence about how perception works, the appearance of the perceived world is misleadingbecause it reflects not only the operation of the perceptual system, but also of our beliefs and expectations and pre-existing folk theories, and it incorporates these to a much more profound degree than generally believed. The phenomenal experience of a taste reflects, in part, what we think we are tasting, the experience of our actions reflect what we believe about the agency of the actions, and so on. Our assumptions about the deliverance of our senses depends greatly on thecontent of our subjective experiences and we have every reason to be skeptical of what this reveals about the information that goes on in the process.[2]An even more serious problem with the use of conscious contents as direct indicators of the content of the outputs of perception is that there is also no room in phenomenology-based theories for the growing evidence of vision-without-awareness, including change blindness, visuomotor control without conscious awareness, blindsight, visual agnosia and disorders of visual-motor coordination and other sources of behavioral and neuroscience data.
The concept of experience in philosophy and psychology as ambiguous. It is used both to refer to the entire corpus of incoming information – as in “experience shapes our perception” – and also to the subjective content of this experience – as in “I experience this patch as red or this line as being shorter than that line”, and so on. But the two are very different. We know this because in the past 50 years or more experimental and clinical studies have revealed that much of what is causally efficacious in our sensory input is not available to conscious experience and much of what is available to our conscious experience is not a cause of our behavior but a product of ouranalysis or our theories of mental life.
4.2What subjective experience reveals about psychological processes
Daniel Wegneretc
At different points in the history of perception research we have oscillated between the two extremes of accepting phenomenology as the basis for a science of perception and writing it off as unreliably or even as epiphenomenal. Although perceptual experience cannot be discounted in the study of perception, neither can one assume that the experience itself is to be taken at face value as an indicator of the nature of the functional mental states that play a role in the explanation of how perception works. How are we to reconcile these differences (which often coexist within individual researchers)?
There is no general solution to this problem. The question of how to interpret a particular type of observation can only be resolved as we build more successful theories – in particular when we have at least a sketch of a theory that scales up from individual laboratory experiments to more general phenomena. The situation we are in is very similar to that which linguists have been in during the last 60 years. Intuitions of grammatical structure led early linguistics astray by focusing on surface phenomena. But as generative linguistics became better able to capture a wide range of generalizations, it found itself relying more, rather than less, on linguistic intuitions. What changed is that the use of the intuitions was now under the control the evolving theories. Even such general questions as whether a particular intuitive judgment was relevant to linguistics became conditioned by the theory itself. Take Chomsky’s famous sentence “Colorless green ideas sleep furiously” which was introduced to show the distinction between grammaticality and acceptability judgments. This example engendered considerable debate because what constitutes grammaticality as opposed to acceptability is not given by intuition but came from the nascent theory itself.
So my view is that as theories of vision formulate general principles, the theories will direct us in the interpretation of evidence from conscious experience. It will show us how to interpret such findings as those of [Wittreich, 1959 #1312]. Wittreich confirmed that when people walked across the floor of the Ames distorting room they appeared to change in size. But he also found that this did not happen when the people were well-known to the observer, e.g., their spouse, even if they were accompanied in the walk by a stranger (show size did change!). Even now I think we are in a fairly good position to be incredulous of the theoretical significance of this finding, given what we know about how reports of conscious experience can be cognitively penetrable, hypnotism being an extreme example of this. Sometimes we can show this fairly directly by comparing measures from which the response bias has been mathematically factored out, as we do when we use the signal detection measure d′ rather than percent correct. But sometimes we make the decision on the grounds that a theory that takes certain observations at face value will simply miss the deeper underlying principles.
4.3The special case of the experience of space[ZwP1]
4.3.1The status of our phenomenal experience of space
Our grasp of space appears to us to be subtle, complex and extremely fine-grained. Our experience of space is all-pervasive; we experience ourselves as being totally immersed in the space around us which remains fixed as we move through it or as objects other than ourselves move through it. Our spatial abilities are remarkable and have resisted a causal explanation despite the efforts of some of the best minds over the past two centuries. For example, we can orient ourselves in space rapidly and effortlessly and can perceive spatial layouts based on extremely partial and ambiguous cues. We can recall spatial relations and recreate spatial properties in our imagination. Animals low on the phylogenetic scale, who may not even have concepts, inasmuch as they arguably (though this is by no means obvious) do not how the power to reason about things that are absent from their immediate perception, exhibit amazing powers of navigation that prove that they have quantitative representations of the space through which they travel and that they update these representations continually as they move through the space or interact with it in various sensory-motor ways. Although perception science is arguably the most developed of the cognitive sciences there are many areas of vision science where it is far from clear that we have posed the problems correctly, and the problem of spatial cognition strikes me as an extremely likely candidate for one of those problems.
The experience of spatial layout is a fundamental datum that presents many constraints on a theory of early visual representation (some of which I will take up later). But it is also problematic because it is the experience of how the world is and not of our mental processes. Because of this it is actually the result of many different mental processes, including the inferences we draw from our beliefs about the location of objects in world before us. Our experience is of a stable panoramic layout of spatial locations, some of which are empty while others are filled with objects, surfaces and features that stand in some spatial relation to one another. This is the very phenomenology that has leads people to postulate an inner replica of the perceived world that constitutes the experiential content of our perceived space – a panoramic display that fills the world around us [Fred Attneave called it "cycloramic" since it appears to cover 360 degrees of view, \Attneave, 1977 #1500]. If we assume that the content of experience must somehow arise from a representation that has that content, and that the representation is constructed from the information we receive through our sensors, then there is a problem about how such a representation could possibly come about, given the poverty of the incoming information. The incoming information consists of a small peephole view from the fovea that jumps about several times a second, during which we are essentially blind, and so on [the information available to the brain has been described in detail and is a familiar story, see e.g., \O'Regan, 1992 #733]. So the gap between our visual experience and the available visual information requires some explanation. While there are many ways to try to fill the gap (some of which, as we will see, appeal to what we call visual indexes or FINSTs) the natural way, given the form of the experience, is to try to build an internal facsimile of the contents of the experience and by postulating a process that takes account of the saccades and constructs an inner picture in synchrony with the eye movements, along the lines depicted in Figure 41 below:
Figure 41. The intuitive view of the content of our experience of seeing[from \Pylyshyn, 2003 #1528]
But as we now know, this theory is patently false – there is no inner replica or picture of any kind in our head, neither literally or in any other non-vacuous sense capable of explaining how we represent spatial information in perception and thought. What has gone wrong that has led so many people to succumb to that story? What has gone wrong is that we are using a particularly natural description of phenomenological experience as the explanandum: we are trying to explain the content of the experience by positing certain intrinsic properties of a medium of representation. But we are not entitled to assume that the content of experience reflects the structure or format or any inherent property of a representation. To do so is to succumb to the well-known intentional fallacy, the fallacy of attributing properties of what is being represented to the representation itself (as if our representation of a square object were itself red and square). Yet so long as we take the content of the perceptual experience as our primary datum this is where it will lead us. Should we, then, discount the experience and start afresh from psychophysical data? I will return to this topic in the next chapter when I consider what a theory of spatial cognition needs to explain.