Transmediation: What Art Affords Our Understanding of Literacy
Jeome C. Harste
Indiana University
What insights into literacy does art afford? This paper argues that the interplay between language and art supports learners, as well as teachers and researchers, in developing a critical stance toward literacy. In making these argument I hope to explicate key processes involved in “transmediation,” the process of taking what one knows in language and representing it in art.
Art as a Social Semiotic
Systemic functional grammar postulates that language did not develop because of one language user, but because of two language users who wanted to communicate (Halliday & Mathiesen, 2013 ). In like fashion, art did not develop because of a single artist, but rather because artistically whatever was created captured something not readily communicated through spoken or written language. The “functional” part of a systemic functional perspective assumes that if two semiotic system did the same thing , one would either never have been created or over time become extinct.
Psychologists tell us that the only thing that confronts the senses are stimuli of various sorts (Bateson, 2000). If we see something as a table or a person or an object, it is because of what the brain did in sorting and organizing these stimuli. Our individual making of meaning is called semiosis (for example, calling some array of stimuli “a table”). The process by which we convince others to call this same array of stimuli the same thing we are calling it is called semiotics. Semiotics, then, is the study of how groups of people come to make and share meaning (Eco, 1976).
In making sense of this bombardment of stimuli, the brain uses sign systems (language, art, music, math, dance) to placehold the meaning that was made as well as share that meaning with others. This process of “naming” our world for purposes of making and sharing meaning is what literacy and the study of literacy are all about. The signs we create – be they language, art, music, mathematics, dance or drama -- allow us to communicate with each other as well as make sense of our world.
As both an artist and a literacy scholar I’m interested in understanding what art has to offer educators in terms of a deeper understanding of literacy and learning. To that end, I’m going to use the major benchmarks in my career, given the books that I have published, to share: (1) some language stories and literacy lessons, (2) some instructional strategies involving art that were created to support teachers in creating classrooms for authors and inquirers, (3) some instructional engagements created to support teachers in creating critical classrooms, and (4) some things two literacy researcher colleagues and I learned about art by taking an indepth look at ourselves as practicing artists.
Language Stories and Literacy Lessons
One of the tasks in our research study on what young children know about reading and writing prior to going to school (Harste, Woodward, & Burke, 1984) asked 3, 4, 5, and 6-year olds to write their name and draw a picture of themselves. While the 4, 5, and 6-year olds could do this very easily, the 3-year olds had more trouble. Nonetheless, by the age of 3 they were beginning to make distinctions between art and writing, although they clearly had not compartmentalized the sign systems as we adults have. We found 3-year olds, with surprising consistency, used circles to placehold their art and up and down strokes to placehold their drawing, or vice versa. We say “vice versa” because one of the surprising things we found was that if their name started with a curved letter (like the ‘s’ in Shannon), they tended (91 percent of the time) to use circles or curved letters to placehold their writing, and up and down strokes to placehold their drawings. The opposite was true for children whose name started with an up and down stroke (like Thomas). We concluded that a child’s name is the child’s first learning laboratory for experimenting with and sorting semiotic systems.
Another task in our research study of what young children know about reading and writing asked children to select three props from a box of props we had put together, tell us a story, and write it down as best they could. Three-year olds had a difficult time making sense of this task so they simply turned it into a task they could understand by using the paper and pencil we gave them as props for the story they were telling. The pencil became the rabbit. The marks on the paper served as a trail of the hops the bunny was taking. The paper itself was the stage for the play that the child is putting on. Pragmatically, children negotiated our research task to one that made sense to them. It was surprising how often they moved to communication systems, like art and drama. These systems of communication obviously made more sense to them than did written language literacy.
Children made several other interesting moves during our data collection. We showed young children pieces of environmental print and asked them what they said. When we showed them the Coca Cola logo pasted on a 5”x8” index card several of the children initially said “Coke” or “Coca Cola.” Virginia Woodward, who was working with the children while Carolyn Burke and I were videotaping, occasionally thought they mumbled and so followed up their initial response with, “What did you say?” or “Could you say that again?” Inevitably, when this happened, the child being questioned responded, “Pepsi.” Feeling vulnerable they began to explore other options that not only fell in the semantic ball park but shared many of the same artistic features of the logos we were showing them.
It became obvious that our research procedures were influencing the data we were collecting. While we had designed a very careful experimental study --largely because of me having taken 48 hours of statistics in my doctoral program -- the very procedures we were using were restricting the data we were collecting. To self-correct we began a series of more naturalistic studies of young children in more naturalistic settings, like homes.
It was about this time that the “research wars” started in our profession. I made myself rather unpopular within the Literacy Research Association community by arguing that not all research methods were equal if the goal was to study language or language learning. I still contend this is true.
I mention the research wars to suggest that maybe one of the first contributions that art made to the field of literacy is that it disrupted the dominant research paradigm that was being used to conduct language research. In an article entitled “Paradigmatic Diversity Within the Reading Research Community” published in Journal of Reading Behavior in 1989, Patrick Shannon had this to say about the shift in our thinking:
Harste, Woodward, and Burke's Language Stories and Literacy Lessons (1984) may be unique because it frankly discusses the authors' metamorphosis from E/A [experimental /analytic science] to symbolic science in their efforts to understand the perspective of young language users. Early in their work, the authors initiated controlled experiments to determine how young children did or did not use written symbols in meaningful ways. After several trials, the researchers noticed that the experiments actually restricted the children in their attempts to relate what they knew about written language. Their study soon switched to an investigation of how children negotiated adult's language requests in order to make sense of the task at hand within particular social settings. Using semiotic analyses, Harste, Woodward, and Burke interpreted the signs, symbols, and signifiers that their child informants offered when using literacy appropriately for specific contexts. This book may be as valuable as a record of paradigm shift as it is a record of what young children can do with written language (p.101).
Several of the graduate assistants working with us during these early literacy studies had young children at home whom they were raising, including myself. As a result we initiated a series of parent-research studies. One of my favorite pieces of data, collected from my daughter Alison at age 6, was her multimodal representation of a telephone conversation she had with her friend Jennifer (see Figure 1). After church Jennifer was going to bring her tutu, slippers, and hair ribbon in a bag over to Alison’s house and Alison was going to get her tutu, slippers, and hair ribbon from the dresser in her room. Together they were going “to play ballerina” (Alison’s words).
PLACE FIGURE 1 ABOUT HERE
There are three things that fascinated me about this note. The first was the sheer economy of the art itself. It captures the subject and sets the tone. The second thing was that Alison uses language (letters), art and mathematics (the plus sign) to record her message. She very freely moved across sign systems in an attempt to mean. The third thing that fascinated me was the sheer elegance of the note itself. There is no clutter. We can only wish that our telephone conversations ended in as tidy a presentation as what Alison at age 6 was able to record in just seconds after getting off the phone.
Creating Classrooms for Authors and Inquirers
Despite the fact that I am now known as a researcher and an artist, I have always considered myself first and foremost a teacher. I think I can safely say the same for my research partners, Drs. Carolyn Burke and Virginia Woodward, as well as the graduate students whom I have worked over the years. Our interest in early literacy was really an interest in how to set up more supportive environments for language learning in schools.
To that end, after Language Stories and Literacy Lessons, we began working in classrooms with teachers attempting to take what we had learned and apply it to practice. The result was a couple of methods books (Harste, Short, with Burke, 1988; Short, Harste with Burke, 1996), several videotape series (Harste & Juerwicz, 1985, 1990-1992), and the creation of a school called the Center for Inquiry in Indianapolis. The curriculum in the Center for Inquiry uses the learner as informant but features children’s literature, process reading and writing, multiple ways of knowing, inquiry-based learning, and more recently critical literacy (Harste, 1992; Harste, 1994; Harste, Leland, Schmidt, Vasquez, & Ociepka, 2004).
Sketch-to-Stretch is a specific example of how we -- and in this particular instance I mean Karen Feathers, Marjorie Siegel, Carolyn Burke, and myself -- went about taking our research findings to create more supportive classroom environments for learning. Building off of the finding that children move very freely across communication systems in an effort to mean Sketch-to-Stretch featured transmediation (Suhor, 1984) or what semioticians see as the process of re-mediating information from one sign system to another.
In practice, Sketch-to-Stretch involves reading a story to children and afterward asking them (often working in groups) to “symbolize” what the story meanx to them in art. We encouraged participants to sketch rather than to use language. Afterward we play Save the Last Word for the Artist, another strategy we developed, in which the artist holds up their sketch, everyone generates hypotheses as to what they think the artists wished to say, and then the artists themselves get the last word.
Marjorie Siegel (1984, 1995) and later Phyllis Whitin (1996), and Megan McBride (2013) made Sketch-to-Stretch the focus of their dissertations; and in McBride’s case, the subject of her masters thesis. They found the same phenomenon that we had found in Alison’s telephone Sketch-to-Stretch and that was a cognitive elegance that simply astounded. Figure 2 shows Matt’s Sketch-to-Stretch from Marjorie Siegel’s dissertation. Matt had read Ira Sleeps Over (Waber, 1972) about two boys having their first sleep over. Matt’s sister heckles Matt by asking him, “How will you feel sleeping without your teddy bear for the very first time? Hummmmmmm?” “What do you think your friend is going to say when he finds out the name of your teddy bear is Foo-Foo, Hummmmmmm?”
When Matt was asked to talk about his sketch and specifically the formula he had constructed in the left hand corner, he said, “A boy plus a teddy bear plus another boy plus a teddy bear equals two good friends.” Now if you know Ira Sleeps Over, this has to be one of the most elegant summaries of a book that anyone might possibly construct. Like Alison’s sketch, the meaning or significance of the event has not only been captured but the presentation is both uncluttered and elegant.
PLACE FIGURE 2 ABOUT HERE
Over the years we have continued to study Sketch-to-Stretch as an instructional strategy. Rather than have students transmediate their readings as sketches, we have had them transmediate what they made of a text into clay, music, drama, dance, and even more recently into video productions.
In a presentation given by Marjorie Siegel (2006) at Carolyn Burke’s and my retirement conference at Indiana University, she argued that while we had made a strong case for transmediation as a generative cognitive process, we needed to begin to study transmediation from a socio-cultural perspective. Art, she argued, positions the user differently in the world and in so doing alters the social practices that surround both the “reading” of the sign and the “reading” of the sign maker vis-à-vis others.
Siegel’s thinking about literacy and literacy learning, like my own, had been influence by critical theorists. Luke and Freebody (1997) argued, for example, that what literacies are valued in a community or culture is a function of the social practices that are in place. Rather than focus on a particular semiotic system (like reading, writing, art) in order to understand literacy, from a socio-cultural perspective the profession needed to focus on the social practices that are operating in a particular context of situation. To change what literacy practices are valued or not valued it is necessary to change the social practices that are in place.