College of Science, Engineering and Health,RMIT University
Scheme for Teaching and Learning Research (STeLR)
2012 Final Report
Due Date: Wednesday 19 December 2012
Send via email attachment to
(Please identify attachment as Project Leaders Surname_STeLR12_FR, i.e. Smith__STeLR12_FR)
Project title: / Development and evaluation of active learning packages for improving student understandingProject leader:
Title and school / Dr Richard Guy
Team members:
Identify team members school (or institution) if other than above /
Dr Bruce Byrne and Associate Professor Marian Dobos (School of Medical Sciences), Mr Christopher Van Der Craats (Senior Web Developer – Since resigned from RMIT)
Abstract:50 words / The Biggs study process questionnaire was used to determine three subgroups of students - deep (D), intermediate (I) and surface (S), within a large first year cohort. Students in groups S and D who used online active learning video “concept clips” achieved better marks than those who did not. Group I students relied more heavily on passive learning resources.
Introduction:
500 – 1000 words / First year students often find it difficult to develop the learning approaches required for success in tertiary education and as a result may not achieve satisfactory academic outcomes. This in turn may lead to a reduction in student confidence and motivation. (Chandler and Potter, 2012). To assist students in this area one needs to establish what approaches are being used and then to use appropriate interventions to support learning development. However it has proved difficult to achieve this with large classes and for this reason there are two main aspects of the study. Firstly to determine whether it is possible to usefully subdivide a large group of students on the basis of their learning approaches and, if so, to establish the learning characteristics of each group. Secondly to determine whether the use of online active learning video “concept” clips can provide a useful learning resource, and to establish this for different student subgroups.
In the current study the revised Biggs study process questionnaire (rSPQ) was used to assess deep and surface learning approaches. This is not intended as a “labelling” exercise (Coffield et al 2004), but as a basis for diagnosing effective and ineffective strategiesfor learning. Many studies approach larger classes as a single cohort and conclusions on learning approaches and the effectiveness of interventions is based on results from the class as a whole. Although these studies have provided an important base for further work it is possible that the needs of individual students are not being met and the effects of interventions not effectively evaluated. For this reason the student cohort was subdivided using cluster analysis of deep and surface learning approaches to improve the precision of diagnosis and to develop an improved basis for determining the effectiveness of interventions. Learning approaches were determined at four points during the study over the course of two semesters.
Accommodation of diverse student learning approaches and maintenance of good academic outcomes are often difficult to achieve in large class university courses. Particularly in situations where academic support is limited, it becomes difficult to create an environment where students feel that their personal learning needs are being supported and that they are being recognized as individuals rather than an anonymous member of a large cohort. These issues become even more significant when one is dealing with first year students in science courses with high levels of factual and conceptual content. One approach in this situation is to use online resources that provide learning support that both caters for individuals and also attempts to generate a student-centered environment. This can be achieved, for example, by using computer-based adaptive tutors that support individual student learning by selecting appropriate problems to be solved, allowing alternative solution strategies and providing timely feedback and hints. In the current study we have used a different approach by generating active learning video “concept” clips as an optional resource for a large class human anatomy and physiology course. The concept clips are designed to provide a reduced cognitive load, the impression of a “personal” communication to the student and an active learning situation that encourages a student-centered approach to learning. The potential effectiveness of this approach is evaluated by comparing the academic outcomes of those students who accessed the concept clips compared to those who did not. This is regarded as a first step in an investigation of the properties and application of these learning resources. Our approach is a very simple one, and combines an ongoing student interest in short videos (e.g. YouTube) with a pedagogically sound basis. The use of these "active" resources is also compared with student use of more passive online resources such as whole lecture recordings (Lectopia) and short lecture clips (without an active learning component). The results of student use of these online resources were compared to both their learning approaches and to their academic outcomes.
Summary of literature: / Learning approaches are best considered within the context of metacognitive abilities e.g. accuracy in describing one’s own thinking, self-regulation and beliefs and intuitions. Self-regulation encompasses learning approaches, supervision and monitoring of these approaches, and motivation (Salamonson et al, 2013; Trigwell et al, 2012). In this study we focus on one facet of learning approach, the distinction between deep and surface learning. Learning can be described in terms of a complex interplay between student characteristics, learning context and outcomes e.g the Biggs 3P model – presage, process and product (Biggs, 1982; 2001). Students with deep learning approaches tend to have well-developed self-regulatory approaches (Vermetten, Lodewijks, and Vermunt 2001), and can reason and explain aspects of a task. Profiling of student subgroups on the basis of learning approaches is becoming a more common topic in the literature in recent years (Vanthournout et al, 2013). However, care must be taken not to regard this approach as a finite classification (Coffield et al 2004).
The use of short videos in learning has been reviewed by Kay (2012). The original technical problems have now been largely overcome and there are several recent publications that investigate different forms of presentation and that report on the utility and usefulness of this learning approach (Kay, 2012; Narual et al, 2012; Pinder-Grover et al, 2011). The cognitive load aspects of learning materials have been addressed by van Merrienboer and Sweller, (2010) and Kirschner et al (2011). Replacement of lectures by other learning resources has been discussed by Mazur (2009)
1.Biggs JB. Student Motivation and Study Strategies in University and College of Advanced Education Populations. Higher Education Research & Development 1: 33-55, 1982.
2.Biggs JB, Kember D, and Leung DYP. The revised two-factor Study Process Questionnaire: R-SPQ-2F. British Journal of Educational Technology 71: 133-149, 2001.
3.Coffield F, Moseley D, Hall E, and Ecclestone K. Should we be using learning styles? What research has to say to practice. edited by Centre LaSR2004, p. 1-77.
4.Kay RH. Exploring the use of video podcasts in education: A comprehensive review of the literature. Computers in Human Behavior 28: 820-831, 2012.
5.Kirschner PA, Ayres P, and Chandler P. Contemporary cognitive load theory research: The good, the bad and the ugly. . Computers in Human Behavior 27: 99-105, 2011.
6.Mazur E. Farewell, Lecture? Science 323: 50-51, 2009.
7.Narula N, Ahmed L, and Rudkowski J. An evaluation of the '5 Minute Medicine' video podcast series compared to conventional medical resources for the internal medicine clerkship. Med Teach 34: e751-755, 2012.
8.Pinder-Grover T, Green KR, and Millunchick JM. The efficacy of screencasts to address the diverse academic needs of students in a large lecture course. Advances in Engineering Education 1-28, 2011.
9.Salamonson Y, Weaver R, Chang S, Koch J, Bhathal R, Khoo C, and Wilson I. Learning approaches as predictors of academic performance in first year health and science students. Nurse Educ Today 2013.
10.Trigwell K, Ellis RA, and Han F. Relations between students' approaches to learning, experienced emotions and outcomes of learning. Studies in Higher Education 37: 811-824, 2012.
11.van Merrienboer JJ, and Sweller J. Cognitive load theory in health professional education: design principles and strategies. Med Educ 44: 85-93, 2010.
12.Vanthournout G, Coertjens L, Gijbels D, Donche V, and Van Petegem P. Assessing students’ development in learning approaches according to initial learning profiles: A person-oriented perspective. Studies in Educational Evaluation 39: 33-40, 2013.
13.Vermetten YJ, Lodewijks HG, and Vermunt JD. The Role of Personality Traits and Goal Orientations in Strategy Use. Contemp Educ Psychol 26: 149-170, 2001.
.
Methodology:
500 words / Methods
The test cohort consisted of 105 first year, first semester, nursing students taking two sequential one-semester anatomy and physiology subjects. This cohort represented 57.1% of the total class. All participating students were provided with a plain language statement and signed an informed consent form. An individual student was able to opt out of any component of the study at any time. The project was approved by the RMIT Human Ethics Committee (College Level) and assessed as low risk.
The students were studied in two consecutive subjects, Biomedical and Physical Sciences 1 (first semester) and Biomedical and Physical Sciences 2 (second semester). The study commenced during week 6 of a 12-week first semester subject and continued through to the end of the subsequent second semester subject. Students were surveyed using the revised study process questionnaire (rSPQ) during week 6. The rSPQ was repeated in week 11 of the first semester and then during weeks 2 and week 11 of the second semester.
Student access to passive and active learning resources was measured during the course of the study. In both semesters student access to recordings of the lectures (Lectopia) was assessed. The recordings were of presented material (e.g. PowerPoint slides) and the voice of the lecturer. Access to video clips provided in first semester was also assessed. In the second semester access to short video “concept clips” was measured. The final exam mark for each student for both subjects was also recorded.
Online Learning Resources
All lectures were recorded using the Lectopia system and made available online. Video clips (33) were made available online between semester week 7 and week 12 during the first semester subject. These video clips were short sections of anatomy and physiology lectures that had been recorded during a previous year. Active learning video “concept clips” were provided online during second semester. In total 39 “concept clips” were generated to complement several of the anatomy and physiology topics covered during the semester. Each “concept clip” was made using the video recording capabilities of an Apple Mac. Each clip consisted of a title slide, a short (approximately 1 - 2 minutes) introduction showing the lecturer, followed by approximately 8-10 minutes of content. The introduction requested the student to “do something” with the content they were about to view. The content was followed by a multiple-choice question and a request for the viewer to pause the video while they considered the answer. The answer was then provided together with an explanation of the incorrect options (1-3 minutes). The “concept clips” ranged in time between 10 and 15 minutes.
The rSPQ was used to track student learning approaches. At the beginning of the project the student deep and surface learning data was divided into three clusters using k-cluster analysis (SPSS). The members of each cluster were identified and then tracked with respect to their deep and surface learning and other study measures.
At the end of semester 2 students were given a 16-question survey (5-point Likert scale). One question dealt with the use of first semester video clips. Six questions related to the second semester concept clips and five questions related to the study in general. Statistical analysis used Pearson correlation, paired T-tests and ANOVA. The intraclass correlation coefficient was also calculated for different components of the end of project questionnaire.
Discussion of results:
500 words / For all groups (except group S first semester) there were significant decreases in deep learning during each semester. Howeverall groups showed a significant increase in deep learning during the non-teaching period between semesters. The effect of the decreases during semester 1 and semester 2 and the increases between semesters generated a final difference for each group between the start and end of the study. For group D there was no change in deep learning over the duration of the study (24 weeks) but there was a significant increase in deep learning, for groups I and S. One might be encouraged by these results in the sense that group D started with a high level deep learning approach, whereas the other two groups had lower deep learning approaches which improved over time. However, the deep learning increases were due to increases in the non-teaching period. The decreases during the teaching periods may have been related to the context of the two subjects where much of the assessment is multiple-choice based and where a large amount of factual material is presented (for both human anatomy and physiology). However, as discussed below, there was also significant engagement with active learning resources (which can be regarded as a deeper mode of learning) to the extent that significant differences in outcomes were found. It may be that the balance between the “surface” context of the course overall and the “active learning” components was such that the former dominated and generated the learning approach changes observed.
The increases in surface learning observed in a previous study (for first semester) were not observed in this study. In fact group D showed a decrease in surface learning in semester 1 and overall showed a decrease in surface learning. The other two groups did not demonstrate a change in surface learning over the entire duration of the study. This result is encouraging in that the various interventions introduced into the subjects between the previous study (Guy et al 2013a) and the current study may have, at least, provided enough assistance to prevent what is regarded as an unhelpful response to the teaching and learning environment. It may be that the decrease (or lack of change) in surface approach may provide an indicator of beneficial interventions. It is interesting that in a previous study (Guy et al 2013a) the group with the dominant surface approach did not show any changes in their surface approach. This lack of response was carried on in the current study over the 24 week period i.e. no changes in surface approach during semesters 1 or 2 or between semesters. The lack of responsiveness of this group may have important implications when these students are exposed to subjects with a much deeper learning context.
Resource / Group S / Group I / Group D
Lectopia (s1 mark) / 81.2% vs 71.1%, p=0.0002 (a)
Lectopia (s2 mark) / 70.5% vs 61.8%, p=0.025
Video Clips (s1 mark) / (a) / (a)
Concept Clips (s2 mark) / 68%, n=17 vs 59.5%, n=16, p=0.012 (a) / (a) / 69.8%, n=15 vs 58.7%,n=14; p=0.019
Age (s1 mark) / (a) / (78.4% vs 70.6%, p=0.005
Age (s2 mark) / (a) / 70.1 vs 61.3, p=0.014 (a) / 72.4% vs 60.9%, p=0.024 (a)
Table 1 Summary of relationships between use of active and passive learning resources, age and final exam marks (semester 1 and semester 2). (a) Represents those cases where frequency of use of the resource was significantly correlated with the exam mark.
It is clear that the groups differ in whether or not use of the various resources impacted on their final marks the subjects in semester 1 and semester 2 (Table 1). For group S use of Lectopia (for both semesters) and the first semester video clips was not related to the final exam marks. All of these are regarded as passive learning resources as they do not require the student to do anything other than view the material. However, this group did show significant improvement in the semester 2 final mark if they accessed the concept clips. In contrast group I demonstrated a relationship between use of Lectopia (both semesters) and the first semester video clips (passive learning) and their final marks. Their use of the concept clips did not improve their mark (users vs non users), although there was a correlation between concept clip use and final mark. For group D there was no impact of the Lectopia recordings, but a significant effect of use of the concept clips.
For group S age was not a major factor in their final exam marks for either semester (although there were correlations between age and final mark. Apart from the passive and active learning resources it was found that age played a major role in the final marks for group I (semesters 1 and 2) and for group D (semester 2). The older students performed better than the younger groups with respect to their final marks. In a previous study (Guy et al 2013a) the older students in groups I and SD performed better than the younger students. The older students in these groups performed as well as both the younger and older students in group D. The important conclusion from these results is the age may moderate the differences in learning approach such that equivalent results are achieved by students with quite different learning approaches.