Measuring Conceptual Development Gains for High School Chemistry Students with and without Scientific Visualisations
David Geelan
School of Education
University of Queensland
Brisbane, Australia
Peter Mahaffy
Department of Chemistry
The Kings University College
Edmonton, Canada
The use of visualisations in science education has received a lot of interest over the past decade, and chemistry education has in many ways been a leader in this field. There is a well-attended biennial Gordon Research Conference on Visualisation in Science and Education and a very large number of different visualisation projects at all levels of education.
Visualisations include everything from student diagrams and drawings to web-based animations and simulations, and the project discussed in this article focuses on the latter. There are many studies in the literature that describe the development and educational use of visualisations, and these report satisfaction from students and teachers, as well as enhanced student engagement.
There are very few studies, however, that provide quantitative information about the effectiveness of visualisations for students’ learning and conceptual development.
We wanted to be able to measure whether teaching with visualisations yields better, worse or similar gains in understanding of the key chemistry concepts on the part of students compared to teaching without visualisations. The focus was on conceptual development rather than success on a standardised test, both because we think this provides a better measure of chemistry understanding and because there is no external chemistry examination in Queensland.
It is difficult in quasi-experimental studies in education to create perfectly matched, randomly selected experimental and control groups. This difficulty is compounded when the research is conducted in school classrooms, since students cannot usually be removed from their classes and randomly arranged. Different teachers and different groups of students have a wide variety of different characteristics, which makes comparisons very difficult.
One approach to dealing with this issue is the use of a ‘crossover’ (Ratkowsky, Evans & Alldredge, 1993) research design. In the version of this design used in this study the same group of students with the same teacher did both the ‘experimental’ (teaching with visualisations) and ‘control’ (teaching without visualisations) condition, at different times. In that way, each class (teacher + students) served as its own control group, meaning that the groups were ‘perfectly matched’. The ‘crossover’ part of the study comes because the order in which the treatments are given might be important. For that reason, half of the 12 classes participating in the study did the visualisation teaching first and then the no-visualisation teaching, and the other half did the two trials in the reverse order. While from an experimental perspective it would be ideal to teach exactly the same topic in each trial, from an educational perspective this is impossible, so concepts of similar difficulty were chosen.
The situation was made slightly more complex by the fact that the Queensland chemistry curriculum is not very prescriptive in terms of which topics should be covered in Grade 11 and Grade 12, so different schools cover the course in different orders. The initial intention was to choose two different topics of similar difficulty and conduct a simple crossover study as described above, however it soon became clear that more than two topics in total were needed if each school was to be able to work with two topics taught in Grade 11 (where the study was focused). We ended up choosing three topics, which makes the analysis more complex but will still allow quite robust comparisons to be made.
The three concepts chosen were Le Chatelier’s Principle (and dynamic chemical equilibria more broadly), Intermolecular Forces (and other interparticle forces) and Thermochemistry. These were linked to teaching sequences intended to take three to four lessons, or about one week of normal Grade 11 chemistry lessons. One or more web-based visualisations was chosen for each concept – links to the visualisations are included below.
Le Chatelier’s Principle
Intermolecular Forces
Thermochemistry
We chose to use existing resources that were available on the net. This may have led to less directly comparable visualisations in terms of approach and style, but we felt that it allowed us to model more closely what really happens in school classrooms.
The Force Concept Inventory (FCI)(Hestenes, Wells & Swackhamer, 1992) has been influential in physics education. It includes 29 multiple-choice items, each with answers from A to E (i.e. 5 in total per question). The correct answer for each item is the correct scientific concept in relation to forces. The other four answers are common student misconceptions drawn from the literature. Mulford and Robinson (2002) developed the Chemical Concepts Inventory, which is more broadly focused on a whole university semester-worth of chemistry concepts rather than the narrowly focused Force inventory but takes a similar approach. The conceptual tests used in this study followed a similar approach. A parallel physics study used some of the items from the FCI, but the three tests for this chemistry study were original. Each test contained 12 items, each with four options (the correct concept and three common misconceptions), and was used as both a pre- and post-test. Here are a few sample items:
Le Chatelier’s Principle
Question 10 relates to the reversible reaction of iron (III) ions, Fe3+, with thiocyanate ions, SCN- to produce iron thiocyanate, FeSCN2+, ions in accordance with the equation:
Fe3+(aq) (pale yellow) + SCN-(aq) (colourless) ⇌FeSCN2+(aq) (red)
10. If colourless solid potassium thiocyanate, KSCN(s), is added to the solution, it will dissolve producing thiocyanate, SCN-(aq), ions according to the reaction KSCN(s) K+(aq) + SCN-(aq). As it comes to its new equilibrium the colour of the solution will:
a. become more red
b. become paler
c. stay the same
d. there is not enough information to tell
Intermolecular Forces
9. Although the water molecule has no overall electric charge (it is neutral), a stream of water will be attracted to a charged rod. This attraction is due to:
a. an induced dipole in the water molecule
b. the water molecules separating into charged H+ and OH- ions
c. the existing dipole (charge separation) between the O and H atoms in water molecules
d. electrons being removed from the water by the charged rod to create H2O+ ions
Thermochemistry
- The reaction between octane and air is very exothermic, and yet an open container of octane can be left at room temperature for several days without catching fire (i.e. reacting) (although it will evaporate). This is because:
- octane is naturally in a liquid state
- energy must be supplied to start the reaction
- there is not enough oxygen in the air to start the reaction
- energy must be removed from the system to break the bonds in the octane before it can react
Some of the teachers were concerned that the students would be disadvantaged in their learning if they learned a particular concept without visualisations (many of the teachers already routinely used scientific visualisations in their teaching), however the short teaching sequence meant that after the post-test (in the non-visualisation teaching sequence) the students could then revise the concept using the visualisation.
We had realised that it is often difficult in schools to use web-based resources, due to restrictions such as limited bandwidth, very strict content filters, malfunctioning computers and lack of plugins such as Flash and Java, and in fact these difficulties arose, to a greater extent than we had expected. As far as possible we moved the materials offline, and a class set of laptops was bought in order to be taken into schools where it was impossible to run the visualisations. A combination of strategies made it possible to complete the research study, but it is clear that there are still significant challenges in allowing all teachers to have the option to use visualisations, even if the evidence suggests that it enhances learning.
One extra layer of analysis was added in order to further explore the ways in which visualisations support learning: data were analysed by students’ sex, learning style (using an adapted version of the Visual-Auditory-Kinesthetic (VAK) test) and academic ability (ranked by the teacher as to whether each student was in the top, middle or bottom third of the class). This enabled us to explore whether visualisations were supportive generally, and whether they were more supportive for some students than others.
Now comes the slightly disappointing part of this discussion: we are still in the process of analysing the data! Initial analysis suggests that in general visualisations are yielding slightly better learning gains (post-test minus pre-test) for each of the three concepts, however we have not yet been able to determine whether these differences are statistically significant, and to measure effect sizes and conduct some of the more detailed tests, because the final data are still being collected from some schools. We hope to be able to add an addendum in December or January with a full analysis of the study data. We hope the ending of this story was not too much of a cliff-hanger.
References
Hestenes, D., Wells, M. & Swackhamer, G. (1992). Force Concept Inventory. The Physics Teacher, 30, 141-151.
Mulford, D. R. & Robinson, W. R. (2002). An inventory for alternate conceptions among first- semester General Chemistry students. Journal of Chemical Education, 79, 739-744.
Ratkowsky, D.A., Evans, M.A. & Alldredge, J.R. (1993). Cross-over experiments: design, analysis and application. New York: Marcel Dekker.