Cognitive Study Proposal

Kihyun Ryoo

Paula Wellings

Peter Worth

July 15, 2003

ED 333A

Learning Environment

Macromedia Flash is atool for developing highly visual, interactive content and applications. Since its first version in 1997, Flash has evolved from a simple animation tool into a powerful, complex, and highly-integrated multimedia design environment. The software consists of a number of conceptual models which enable users to create a wide variety of artifacts. As Flash has adapted to meet the needs of interactive designers, programmers, and animators, one side-effect of this evolution has been the accumulation of redundant or even conflicting models for artifact creation. This complexity poses significant challenges to novice users, not only because of the large collection of functions and procedures, but also because users can create the same effect by using different methods.

Recognizing the power and utility of Flash despite these difficulties, many teachers in the SunshineCoastSchool District would like to learn to use the program. Many would like to become expert facilitators with the ability to teach their students to use Flash. Others would like to create multimedia artifacts for the classroom. Some would simply like a functional understanding of how Flash works. The teachers come from a variety of backgrounds and levels of technical literacy, but share a strong desire to become proficient in Flash. She commissioned the Teach Flash study as a way to find an effective way for teachers to learn Flash.

After doing some preliminary research on available Flash training programs, the Assistant Superintendent for Technology at SunshineCoast, Julie Bunda, found that a number of tutorials already exist. She noticed that some involve having the user deconstruct and/or recreate a completed Flash artifact and others give an explanation of the available tools and how they are used. Through experience and knowledge of studies in the area of learning software, she knew that people find different instructional methods useful.

The number of ways in which people become proficient in software programs indicates that the “Teach Flash” study of how best to instruct people in the use of Flash will be broad-based. Studying just online tutorials or university classes would place unnecessary and misleading constraints on the study. The learning environment in this case is not in any one location, but in the program Flash itself, and in the interaction between learner and program. It is in the construction of knowledge and the building and connecting of schema that we will find learning, and this will be reflected in our methods. Thus, we need not study a particular Flash class, but rather seek out expert models of Flash understanding. Obviously, issues of practicality require that we limit our study somewhat. To that end, the subjects of our study will be advanced or expert Flash users, and those teachers with a novice practice of Flash use.

Learning Problem

Existing Flash tutorials usually fall into two categories: one is to provide learners some completed Flash artifacts and have learners deconstruct/reconstruct them; another one is systematic introduction of Flash functions and design techniques. However, these tutorials are designed based on certain expert mental models, which are not necessarily suitable for each novice learner. Each expert mental model is related to the expert user’s background, including previous experience with similar design/develop software in particular, and previous experience with the world in general. Therefore, it is difficult for novice learners to learn Flash effectively. To be more specific, novice learners can not easily construct their own mental model of Flash tutorials and training programs.

Learning Goal

Learning is a process in which learners construct new ideas or concepts by combining their current/prior knowledge and experiences with new concepts and structures through active engagement in problem-based environments.From this cognitive perspective, we identify Flash experts as those who “have obtained a level of knowledge and experience that enables them to have meaningfully organized cognitive activity, principled and relevant representation of problems, an efficient and informed strategy, ongoing and flexible self-monitoring, and principled and coherent explanations of their work.”(Bransford et al.,2000) This more evolved or integrated conceptual understanding is what, in this study, distinguishes an expert from a novice.

Therefore, novice users of Flash face a number of challenges.The complexity of Flash requiresthe formation of a new cognitive hierarchy to support the effective learningand use of the tool by all levels of learners.It is thus difficultfor the novice learner to easily access an expert mental model within which toachieve their desired functions.

In order to solve this problem, it is important to understand how experts think about the interface and to learn their mental models in order to facilitate Flash. Our learning goal is to allow novice Flash learners to access particular expert mental models, which best correspondto their knowledge and background, and construct their own mental model with efficiency and effectiveness.

Design Principles

The Teach Flash study will examine the Flash learning process from a cognitive perspective. Because the focus is on how individuals, regardless of situative or physical environmental factors, construct their understanding of Flash, the study is grounded in three principles of cognitive learning as outlined by Greeno, Collins, and Resnick (Greeno, et al., 1996). Following principles reflect our assumption that we will be able to design an effective means of teaching Flash to new users by understanding the conceptual models of expert users.

Cognitive design principle 1 (c1) asserts that learning occurs in an environment that encourages people to construct an understanding through interesting problem-solving tasks. The Flash environment provides continuous problem-solving and reasoning tasks for the novice, or indeed the expert. Their ability to successfully use the program is contingent on their developing a strategic knowledge of how and when to use the extensive array of tools and features to create a desired effect, and the difficulty of gaining such strategic knowledge is a fundamental component of the learning problem.

This principle of one’s understanding being constructed through interaction with an environment will guide this study in that it will focus on how expert and novice Flash users develop, use, and perceive their own strategic knowledge, and how newcomers to Flash have developed problem-solving strategies for other working with other programs. The goal of this facet of the study is to make explicit to new users the tacit procedural knowledge used by experts, as discussed by Gott in her study of the LISP tutorial (Gott, 1989). This is complicated by the fact that, with Flash’s evolution from a number of conceptual models, the sheer variety of ways of achieving the same task means that there will be several sets of procedural knowledge. While this may create some confusion, it also accommodates a greater variety of learning styles and learner backgrounds.

Cognitive principle 2 (c2) addresses just this issue. Learners proceed through “sequences of cognitive development” (Greeno, et al., 1996), in which new learning is acquired in the context of what is already known. Thus, understanding and activating the learner’s prior knowledge and frame of reference is essential in helping them to build connections to new information and skills.

In the Teach Flash learning problem, this is a crucial issue. The teachers who will be learning Flash have varied backgrounds and experiences in interacting with computer software. Developing an effective instructional method will require a clear understanding of how those backgrounds and experiences are linked to the creation of conceptual models of how Flash works. The study phase will examine how proficient Flash users have created such models based on their use, and how those models have changed over time. The tool and manipulation paradigms within the Flash interface have been designed to relate to people’s previous experience with other similar development environments, including previous version of Flash. How these conceptual models can be utilized for different learners will be an area addressed in this study.

Cognitive principle 3 (c3) addresses the practical use of Flash by the teachers. C3 emphasizes that learning can be demonstrated and applied when learners are able to generalize information within a broader context. This principle works on two levels in this study. Within the realm of acquiring proficiency in the use of Flash, it will be important to understand how users see their individual actions and commands in the context of the wider Flash environment. While this is related to c2 in that it requires examination of Flash users’ conceptual models of the program, it is distinct in that the focus here is on the how the device model makes explicit the user’s understanding of the overall concept of Flash, while c2 focused on how concept models of other familiar programs could be utilized as prior knowledge in the teaching of a new program.

The other level on which the generalization principle will be a factor deals with how the mastery of Flash, once attained, will be utilized. Our proposed solution will include the assumption that, with limited resources of budget and time, teachers and the school districts sponsoring their training will want to know how what they are learning will be implemented in the classroom. The efficacy of the training will necessitate the explicit generalization not only of how the individual functions relate to the overall program of Flash, but how those functions will be useful to the students whom the teachers will ultimately be teaching.

Proposed Study

The “TeachFlash” study is based on the cognitive design principles explained above, particularly that people create mental models representing their understanding, and that understanding and making these conceptual frameworks explicit, and structuring instruction on these frameworks represents an effective means of teaching new skills and concepts. To that end, the study consists of six steps designed to identify experts and understand how their conceptual frameworks of Flash differ from those of novice users. The first step uses a focused questionnaire aimed at assembling an appropriate Flash-user group. Steps two and three involve observing the Flash users creating artifacts and reflecting on their process. In step four we analyze the data from steps two and three. Step five involves the expert reconstruction of a novice’s artifact, while the final step is a comparison of the two construction models of the same artifact.

Step One: Initial Questionnaire

The goal of the initial questionnaire is to identify potential study participants in the SunshineCoastSchool District. The questionnaire will be distributed through staff development seminars as well as being available in professional development resource centers, both in schools and at the district office. In addition to asking the questions that follow, the questionnaire will be explicit with regards to opportunities and compensations associated with participating in our study.

Based on the responses to this questionnaire, we are initially interested in following up with teachers that have experience with Flash. We assume that this group will be relatively small as Flash is a complex and expensive program and is not typicallypart of the business productivity software installed on school computers. In addition, while a number of interesting studies in the UK[1] have suggested that Flash might be the new and improved LOGO, there has not been a significant effort in the US to position Flash as a constructivist learning environment. Our hope with this initial sorting questionnaire is to identify twelve study participantswho have experience in using Flash personally and with their students, as this population will likely have the knowledge and problem solving skill in Flash that we are hoping to target, explore, and leverage in future design solutions.

Step Two: Observation of Artifact Creation Process

In this step of our study, we are interested in observing how study participants with an existing knowledge of Flash create artifacts. We are also interested in understanding how participants support their use of the software with existing resources such as manuals, online tutorials, and discussion boards.To support these objectives, we will be observing the artifact creation process within the environment in which the participants usually create artifacts—be that the home, computer lab, or other location. The participants will be asked to create an artifact in Flash that will be used in some common context, such as a classroom lesson, a personal website, orpreparation for a classroom activity where students will use Flash.

To capture this activity, we will set up a recording device that video captures the participant while they are creating the artifact, as well as a recording device that captureson-screen activities. A researcher will not be present during the actual artifact creation. This choice is based on the desire to enable the participant to create the artifact on a normal time schedule and also to decrease the impact of the researcher on the participant’s typical creation and troubleshooting strategies. However, the recording software will document any recording irregularities to ensure data accuracy.

Step Three: Reflections on Observation Videos

In this step, a researcher will review the video of the artifact creation process with the participant in a session that is also video recorded. The researcher and the participant can see both a video image of the participant working on their artifact and a synchronized view of the computer monitor, and can hear any associated audio. Depending on the length of the recording, the researcher may identify specific areas. The selection of these areas is based on identifying moments where the participant appears to be engaging in activities that reveal elements of their structuring of knowledge.The participants will be asked to describe what they were thinking during the process of creating the artifact and to describe their process when they encountered any challenges.

The observation video from Step Two acts as a prompt to the participant and facilitates their reflection on their own cognitive processes. Of particular importance to the researchers will be observing the participant’s methods of explanation and any accompanying gestures, which may signify the manipulation of mental models or other representative schema. Additionally, researchers will probe the participants to learn more about how they acquired their current level of knowledge and ability.

Step Four: Categorization of Participant Activities Observed in StepsTwo & Three

In this step, researchers will utilize a model premised on Baxter and Glaser’s model of Cognitive Activity and Structure Knowledge to categorize the artifact creation practices of the participants. While working from this general framework, specific categories of activities and cognitive strategies will be articulated that relate to the activities encountered in the use of Flash.

TABLE 6.1 Cognitive Activity and Structure of Knowledge
Structure of Knowledge
Identified Novices / Identified Experts
Organized Cognitive Activity / Fragmented / Meaningful
Problem Representation / Surface features and shallow understanding / Underlying principles and relevant concepts
Strategy Use / Undirected trial-and-error problem solving / Efficient, informative, and goal oriented
Self-Monitoring / Minimal and sporadic / Ongoing and flexible
Explanation / Single statement of fact of description of superficial factors / Principled and coherent

From How People Learn: Brain, Mind, Experience, and School (Expanded Edition) by John Bransford (Editor), Ann L. Brown (Editor), Rodney R. Cocking (Editor), 2000, National Research Council, Washington, D.C.

Based on this analysis, we hope to identify participants with activity and knowledge structures that are representative of people in both columns of Glaser and Baxter’s table.Although Flash users that we might locate on the “Identified Novice” of the above chart are able to create an effective Flash artifact, their methods indicate not an intentional path, but an almost accidental trial and error or “stumbling into the creation of a completed artifact. In contrast, the teachers in the “Identified Expert” column of the above chart will have created Flash artifacts using a clear, goal-oriented strategy. Those strategies may vary among the individual users, but the fact that they have a coherent conceptual structure for a variety of activities teachers might perform with Flash, and are able to implement that structure in the creation of an artifact, is of interest to us here.”

Step Five: The Expert Reconstructs a Novice’s Artifact

In this step, we present an identified expert with a final product created by an identified novice. Without access to any source files or materials, the expert is asked to construct a Flash file that has a similar functionality and visual quality to that created by the novice. Similar to steps Two and Three above, the expert is observed creating the artifact and then reflecting on the creation process with the researcher. It is hoped that through this process we will see differences in how the experts organize and create these Flash artifacts, and how they reflect on the process of doing so.

Step Six: Comparison of expert and novice practices in creating artifacts with Flash

As a final step in this study, wewill comparethe data gathered from the novices and expertsto a gain a better understanding of how cognitive activities and structures of knowledgeimpact the practice of creating artifacts with Flash. Comparisons that we think are valuable to make include:

Comparison of Flash working documents
Flash is organized around the principle of having a working file that is then exported to a final file upon completion of the artifact. As Flash enables artifacts to be created in a number ways, the working file becomes an embodiment of how a participant understands Flash’s functions and capabilities. Comparing these working files will enable us to better understand the differences and similarities in participants’ conceptual understandings of the software.