Bridging the feedback divide utilising inclusive technologies
Abstract: The aim of this action based research project is to enhance the student learning experience through the production of vidcasts that focus on the conventions of academic writing in Glasgow Caledonian University’s School of Building and Natural Environment (BNE). Vidcasts (using a blend of narrated audio and on screen images and slides) aim to meet students’ individual academic writing requirements. Four vidcasts have been produced at this stage; the first level focuses on an introduction to academic writing explaining and illustrating how to write a paragraph at micro level and how to apply the conventions of referencing. The second level considers how to plan, construct and write a report including a detailed breakdown of appropriate material for each section of the report. Level three concentrates on features of group reports whilst level four is aimed at executing a complex report applying a case study format. Each vidcast has links to additional support materials that are available on the web.
The objectives were to produce models of good practice on the issues of academic writing conventions at four distinct levels. The vidcasts have been placed on the virtual learning environment (VLE), Blackboard, and feedback was requested from student representatives studying across the programmes in BNE.
We are also exploring the effectiveness of three different approaches that have been applied to obtain feedback; these included the use of online feedback on the Effective Learning Service’s website through Google analytics and Poll Everywhere, an interactive electronic voting tool and tracking students’ use via the university VLE. An integral aspect of this project is the involvement of students and BNE staff in the evaluation of the different approaches. We recognise that the feedback, like the vidcasts, should be context based and acknowledge that one size does not fit all. Thus, we acknowledge that the feedback mechanisms will need to be inclusive and tailored to different ability levels as well as for different cohorts of students.
An interesting outcome from this project has been that despite students noting the importance of receiving timely feedback from staff; they appear unwilling to complete surveys giving feedback formally, although willing to express their views anecdotally. One reason could be that students are constantly being asked to complete evaluation from individual end of module forms to National Student Survey leading to survey fatigue. This suggests that using immediate feedback formats such as Poll Everywhere has a role in encouraging students to respond although it is acknowledged that there could be insufficient time for students to reflect.
Keywords: academic writing conventions, feedback, vidcasts, student learning,
The aim of Glasgow Caledonian University’s Caledonian Scholars and Associates initiative’sbased research project is to enhance the student learning experience through the production of innovative feedback practices by the medium of 'vidcasts' in Glasgow Caledonian University’s (GCU) School of Building and Natural Environment (BNE). This project commenced in October 2010 and is aligned to GCU's Learning, Teaching and Assessment Strategy (2008-2015). The project was funded by GCU’s Caledonian Scholar and Caledonian Associate Initiative, the purpose of which is ‘… to contribute to a distributive leadership model of innovation in learning and teaching across the University’ (Caledonian Academy, 2010). By supporting staff in this initiative the intention is ‘…to contribute to the enhancement of learning and teaching practice and the quality of the student experience’ (Caledonian Academy, 2010). The focus was directed towards meeting Objective 1 of GCU's Learning, Teaching and Assessment Strategy (2008-2015), a summary of which is outlined as follows: ‘ …To equip students with the knowledge, skills and attributes to operate as flexible, independent lifelong learners. This will be through the application of incorporating innovative approaches to teaching and learning.’ (Glasgow Caledonian University Quality Office, 2008)
As Cohen et al., (2007) reiterate, action based research is an approach that enables the practitioners to pose questions, carry out actions and justify activities. Once evidence has been gathered in collaboration with other educators in the work place, there follows the process of reflection enabling social transformation and a critique of practice. All takes place against the theoretical background of action based research (Kemmis & McTaggart, 1992, cited in Cohen et al., 2007).
The original objectives of the project were to produce and evaluate models of good practice on the issues of academic writing conventions. Within higher education, students are required to engage with Advanced Academic Literacy (AAL) and as such, write at an accepted complex level as required by higher education. Indeed, Tardy (2005, p326) observed that ‘… students need to learn ways of thinking about, interacting with, and constructing knowledge of disciplinary communities and content’. All has to occur within the discipline of the individual academic department within the higher education institute; this methodology is generally defined as being an academic literacies approach (Lea & Street,2010).
The Effective Learning Service
In Glasgow Caledonian University, the Effective Learning Service (ELS) first commenced in 2000. At that time, academic support was regarded as being necessary in order to develop the personal and study skills learners needed to meet the demands of Higher Education (Gerrard et al., 2005).This could be argued as being the traditional study skills method, a deficit approach (Lea & Street, 2010). However, since its conception, the ELS approach has developed to one that recognises that irrespective of the student’s background and knowledge, entering or re-entering higher educationis a transitional process which is unique to each individual. This has underpinned the development of the ELS in GCU, such that it aims to develop the academic literacies that are required in higher education (Kern & Schultz, 2005). This approach is one that that has been integrated on an institution-wide basis, is inclusive and student focused, emphasising that all students can improve learning, rather than a remedial few (McAllister & Shapiro, 2004). Moreover, this developmental approach recognises that learners derive from different backgrounds, have different motivations and learning needs (Illeris, 2006). Thus, the pedagogical approach that the ELS offers is one that seeks to encourage the students to take on responsibility for their own learning in that they identify what is important, rather than being directed by the lecturer (Illeris, 2006).
We believe that students at GCU are comfortable with accessing technology, indeed, over the academic year 2009/2010, over 13,000 visitors accessed the Effective Learning Service (ELS) website, of whom 17% accessed the online guides (Google Analytics, 2011). At the same time, over 5,000 students attended over 200 workshops that were run by the ELS over the academic year 2009/2010. However, although the students attending the workshops indicated their usefulness, in feedback many commented that they would also have liked to have the opportunity to re-access the material at a later date. Moreover, every workshop also generates students requesting individual follow up appointments or requests for additional support material. This suggested that using vidcasts in Higher Education would be helpful for students in their academic journey and aid their progression towards independent learning.
Using inclusive technologies
A Vidcast or vodcast contains audio and images, either moving or fixed (Traxler, 2008). It is widely acknowledged that students have different learning styles (visual, reflective,auditory, and kinaesthetic) but at times some of these learning styles are not accommodated fully in Higher Education, although Fox and Ronkowski (1997) acknowledge that some lecturers accommodate these differences in their teaching methods. This project will use vidcasts to incorporate the students’ preferred pedagogical preferences in an online environment. Podcasting, for example, will clearly appeal to auditory learners; with the addition of visual prompts the vidcast will also support dyslexic students in retaining the information (Edirisingha et al., 2008).Essentially by students having to concentrate and apply active listening when using podcasts for educational pursuits, this means that they are cognitively involved rather than merely listening, as one does to music.
In addition, users of vidcasts with visual impairments will be able to select to listen solely to the audio element. Students can also choose when and where they wish to engage with the vidcasts and this approach enables students to revisit the material at their own paceand can download the material on to their mobile device (Informal Mobile Podcasting and Learning Adaptation (IMPALA) 2006;Gribbins, 2007). Moreover, students at university have a wealth of experience, irrespective of age and if the learning is purposeful, then the adult learner is more likely to become engaged in the process(Rogers &Uddin, 2005).
Investigation Stage
Two workshopsapproximately 45 minutes in length were recorded using a fixed video camera and subsequently digitised and placed on the ELS website along with a description of the workshop. Initial feedback from students suggested this approach offered little value as it was not easy to find the relevant information as they had to scroll though 45 minutes of video to find the relevant part. Further feedback also suggested it was difficult to see the PowerPoint slides in the background and that seeing the presenter offered little value. (Figure 1).
Figure 1: Project journey.
Evaluation of first year of funded project
The initial evaluation approach that was selected, asked users to evaluate the usefulness of the two completed vidcasts on essay writing and writing final year reports through feedback from the Survey Monkey on-line questionnaire service. Producing vidcasts to enhance students’ learning is only a worthwhile activity if students use the vidcasts. It was important, therefore, to know what students thought about this type of delivery and whether they felt more engaged in the student learning experience. Accordingly, the questionnaire was placed on the ELS website in November 2009. However, feedback was slow and only 50 questionnaires were completed. Nevertheless, the responses in the main were positive, and users (both students and staff) gave recommendations for improvement. At the same time statistics were collected from Google Analytics which indicated that over 1,000 users had accessed the vidcasts, which clearly indicated that they were being utilised.
The respondents to the Survey Monkey online survey stated that they preferred the vidcast to be delivered in shorter sections;that they wanted brief summaries of the content and suggested improvements in navigation. These changes were subsequently actioned; shorter chapters consisting of no more than five minutes of content were added along with detailed chapter descriptions.Improvements were made to address the navigation issues by upgrading the web video player to allow users to control navigation. In response from feedback from post-graduate level students, an additional vidcast on writing literature reviews was produced. (Figure 1).
Evaluation of second year of funded project- The four new vidcasts
In January 2011, an additional four vidcasts were placed within GCU’s institutional virtual learning environment (VLE).This new approach was selected to allow us to target this particular academic school (BNE) and engage the students in their own learning environment rather than placing the content on the ELS website, which was used previously.
The four new vidcasts are also set within the national context of exploring new assessment and feedback practices, the purpose of which is to enhance the student experience, and more specifically to support the development of self-regulated learners (Nicol & Macfarlane-Dick, 2006). We wanted to discover the following:
- Can increased dialogue make for more effective feedback as perceived by our students?
- Can the use of new learning technologies enhance such dialogue and thereby support the academic literacies model through an online process. (Figure 1).
Conventions of academic writing and feedback
Primarily the foundation for placing the second year aspect of the Scholar and Associate Project was based on Nicol and Macfarlane-Dick's (2006) principles which focused on the importance of giving students appropriate and timely feedback that encourages and helps the students learn and progress. Feedback is essential to student learning. Providing feedback in different media reinforces the message and gives students a more personal learning experience. The first principle recommends helping students to clarify the criteria for good performance (outcomes, performance criteria and expected standards).Our response was to produce exemplars demonstrating acceptable academic writing conventions using the vidcastsfor students studying in the School of Built and Natural
Environment at academic levels 1-4 (Figure 2).
Figure 2: Extract from introduction to academic writing vidcast.
The first level vidcast focuses on an introduction to academic writing conventions by explaining and illustrating how to write a paragraph at micro level including the conventions of referencing. Animated sequences highlighted the various errors throughout the paragraph. This approach blends the narration and the visual representation shown on screen to emphasise the point to the viewer. The second level vidcast explains how to plan, construct and write a report including a detailed breakdown of appropriate material for each section of the report. The level three vidcast concentrates on features of group reports and examines some of the issues that may occur when individuals are working together on one task.The level four vidcast is aimed at post-graduate students and demonstrates how to execute a complex report, applying a case study format. Each vidcast has related links to additional reputable support materials that are available on the internet. (figure 2).
The next principle influenced by Nicol & Macfarlane-Dick (2006) indicated the importance of facilitating the development of self assessment (reflection) in learning. We decided to send emails requesting feedback from selected students and staff in the School of BNE and carry out feedback using Poll Everywhere with the first year cohort following the use of the first level vidcast. Poll Everywhere is an interactive audience response system that uses mobile phones, twitter, and the web. The responses are displayed in real-time. The polls can be multiple choice based questions or open-ended questions thatcan be created to create conversations with the participants. The advantage of Poll Everywhere is that immediate feedback may be obtained. We thought that since a high percentage of the students in BNE first level were young males, that they would have a mobile phone and be comfortable with using technology.
Figure 3 illustrates an example of results from a sample poll using Poll Everywhere. Students were shown a vidcast as part of their lecture and then asked questions to test their understanding about conventions of academic writing. However each group of students revealed an unforeseen result in that students preferred to either vocalise their thoughts, or put their hand up rather than using their mobile. This may have been for several reasons; firstly, students had to use a long number to send their results back (Figure 3) and secondly, some students were concerned that responding would incur cost (despite the fact that most had call plans with free texts). The lecturer took time to become familiar with using Poll Everywhere and in the initial polls students could guess the correct answer as the results were revealed as student responses were sent. In the later polls, the lecturer learnt to hide the results so they did not appear instantaneously. Despite these challenges, students enjoyed the interaction; however, as this took place very quickly after the learning experience, there was no time for reflection by the learner.
Figure 3: Sample student response using Poll Everywhere.
Following this, we distributed a brief questionnaire requesting comments from selected staff and students in the school of BNE. To date, although we know the vidcasts are being used regularly, based on Google analytics web tracking (1583 people accessed the vidcasts in 2010), only a handful of BNE staff and students have responded to the email.However anecdotal feedback has been heartening with users stating that they found the vidcasts easy to follow and liked the fact they had the option of being able to navigatethrough the chapters.
The issue of survey fatigue has been suggested in studies, with respondents in these studies stating that reasons for non completion included: too many surveys, lack of time and that many of the surveys appear to be irrelevant (Porter et al., 2005). Other research highlighted that the culture of the department may affect the level of responses in conjunction with the personality and attributes of the respondent (Porter & Whitcomb, 2005).
Conclusion
The results indicated that students liked to respond using their mobile phones, but some were unwilling to engage with the feedback process, either because of security or cost, although interestingly students were happy to respond by putting their ‘hands up’ when asked the same questions that were on the Poll Everywhere slide.This suggested to us that students do not feel uncomfortable responding infront of their peers. However, it is acknowledged that this approach of feedback does not allow time for reflection.Additionally, Clouder (1998) suggests that‘… students are not all at similar cognitive levels; therefore their evaluations can only be a product of their expectations for their level of cognitive development.’ Indeed in a study by Arthur (2009) it was noted that some lecturers believed that they are the professionals and thus they are the only ones who are sufficiently competent to evaluate lecturers’ work. We would disagree but recognise that care needs to be taken when distributing questionnaires. However we believe that while students and staff may not formally respond to questionnaires,by accessing the site repeatedly and spending time using the vidcasts and the related links this partially indicates student and staff approval.