Exploring the Effects of Causal Mapping Procedures on Causal Understanding

Allan Jeong

Instructional Systems Program
Educational Psychology & Learning Systems
Florida State University

Woon Jee Lee

Instructional Systems Program
Educational Psychology & Learning Systems
Florida State University

Descriptors: Concept mapping; causal understanding

Exploring the Effects of Causal Mapping Procedures on Causal Understanding

Allan Jeong & Woon Jee Lee

Abstract

This study examined three structural attributes of causal maps (total links, temporal flow, outcomes node position) and their relationship to map accuracy (ratio of correct/incorrect links) to identify effective mapping processes and strategies. The findings suggest that limiting the number of links can increase accuracy where asincreasing temporal flow was not found to increase accuracy in students’ causal maps and causal understanding. The implications of these findings on how to setup causal mapping tasks and directions for future research are discussed.

Introduction

To identify root causes to complex problems, causal diagrams can be used to elicit, articulate, refine, and improve understanding and analysis of complex problems. Casual diagrams, a network of events/nodes and casual relationships/links, have been used to assess systemic understanding of complex problems. However, students’ diagrams canvary widely in accuracy if students use different processes while constructing their diagrams.Research is needed to determine which processes help students produce more accurate causal maps.

A previous study (Jeong & Lee, in press) examined how number of links in a map, temporal flow, and lateral distance of the outcome node from the left edge of the map (nodedistance) werecorrelated withmap accuracy. Map accuracy was measured in terms of the ratio of correctly/incorrectly identified root causes and the total number of times each student correctly identified the links (root links) connecting root causes to mediating causes. The study produced data to suggest that increasing temporal flow can increase the number of correctly identified root causes, and placing limits on the number of links can significantly increase the number of correctly identified root cause links.The correlation between distance of the outcome node and causal understanding was not statistically significant. However, these findings were based on the maps of only 16 students.

The purpose of this study was to determine if similar findings could be replicated in a different course, in a different lesson, with a larger numbers of students, and with a simpler measure of causal understanding based on the ratio of correctly to incorrectly identified links between variables. Like the previous study, regression analysis was used to determine to what extent the three variables (number of links, temporal flow, and lateral position of outcome node) predicted the accuracy of students’ diagrams following online discussions of the cause-effect relationships between variables. The research questions in this study were:

  1. What are relationships between mapthese map attributes and students’ causal understandings (ratio of correct/incorrect links)?
  2. What is the relative magnitude of each attribute’s impact on causal understanding?

Method

The participants were 30 graduate students enrolled in an online course on the topic of Introduction to Distance Education at a large university in the southeast region of the U.S.Studentswere presented with a case study that required them to: a) analyze and discuss the possible cause-effect relationships between the six variables/conditions that determine how well an organization is able adopt new technology; and b) identify which of the variables might be considered root causes to problems with technology adoption. Prior to discussing and debating over the cause-effects relationships between variables in an online threaded discussion forum, each student downloaded a map template (figure 1) to individually construct a causal map that conveyed their views on how the variablesare causally inter-related. Students then debated over the merits and logic behind proposed causal links in the discussion forum. Based on what they learned from the discussions, students revised and submitted a second and final causal map.

Figure 1. Map template preloaded with 6 conditions and outcomes.

Data Analysis

Causal understanding was measured in terms ofthe ratio of correctly to incorrectly identified links (matching scores) based on a direct comparison to the instructor’s map.The matching score for each student was computed using the jMAP software (Jeong, 2010). Number of causal links was measured by counting all links in each map.Ratio of temporal flow was computed by dividing the number of links pointing in the general direction of the final outcome node (see green node in figure 2) by total number of link. Node distance was measured by taking the absolute distance between root node (starting point of cause-effect relation) and final node (outcome position).

Figure 2. A student’s map illustrating right-to-left temporal flow.

Results

Table 1 shows that with the maps produced prior to discussion, no significant correlations were found between attributes and causal understanding. With maps produced following discussion, total number of links was negatively correlated with students’ causal understanding or matching scores (r = -.425, p = .027).

The regression analysis of the maps producedpriorto discussions (Table 2) did not produce a statistically significant model. The maps following discussions produced a statistically significant model (F(3,26) = 3.043, p = .049) that explained 28.4 % of the variance (R Square = .284) with a power of 0.75. In this model, total number of links had the largest and a negative associationwith students’ causal understanding (β = -.504, p = .021) after controlling for the other two attributes. This finding was consistent with that found in the previous study. As a result, the findings altogether suggest that students should be instructed to minimize and limit the number of links in their causal diagrams to create more accurate and parsimonious causal maps.

Although not statistically significant, the model also revealed that temporal flow had a negative association with causal understanding. This finding contradicted findings from the previous study. One explanation for this difference is that in the previous study the initial position of each node in the map template were positioned along the left edge of the template. Hence, students were essentially required to move the nodes to make adequate space between nodes so that links could be inserted between nodes. In this study, many of the students did not change the position of the nodes most likely because all the nodes were initially placed in the center portion of the template with the final outcome node encircled by all other nodes.

The distance of the final outcome node was found to have no significant impact on students’ causal understanding which was consistent with the finding from the previous study. These combined findings suggest that positioning the final outcome to one particular side or edge of the map (even though it is correlated with temporal flow) is not necessary a critical part or step in the map construction process.

Discussion

One main finding in this study was consistent with findings from the previous study. A negative correlation was found between total number of links and students’ causal understandings. The instructional implications of this finding is that students should be encouraged to minimize and limit the number of links in their causal maps in order to help them create more accurate parsimonious causal maps and to increase their level of causal understanding. Another alternative is to simply place a limit on the total number of links students can insert into their causal maps.

In contrast to findings from the previous study, the data showed that temporal flow had no significant association with causal understanding. Some plausible explanations for the differences in findings are the following: a) in this study, initial position of nodes were placed in the center of the map (not at the far left edge of the map), and as a result, many students did not change position of nodes; b) the outcome node in this study was placed in the center of all other nodes; and c) students has to consider only six variables instead of 15 variables in the earlier study. These findings suggest that: a) the nodes should be placed to the far edges of the map in order to encourage students to consider the temporal flow of events; and b) temporal flow should be encouraged especially when students have to examine the relationships between large numbers of variables.

Given that this was a case-study, the findings reported here are not conclusive because the data cannot be used to determine cause-effect relationships. To determine cause-effect relationships between the various map attributes and students’ causal understanding, future studies can conduct controlled experiments on each individual attribute. Future studies can also be conducted in the following manner to address some of the limitations of this study: a) control individual differences in content knowledge & mapping skills, or increase sample size; b) replicate earlier study using identical node placements and outcome measures; c) conduct thorough identification of relevant variables to facilitate more systematic research; and d) consider other outcome measures (event chains, nodes with largest impact).

References

Jeong, A. & Lee, W.J. (In press). Developing causal understanding with causal maps: The impact of total links, temporal flow, and lateral position of outcome nodes. To appear in Educational Technology, Research & Development.

Jeong, A. (2010). jMAP. Retrieved November 11, 2011 at