Ken Newton Has Been Looking at E-Assessment and Feedback in SAM Recently. He Notes
January Eat Newsletter 2010
Feedback
Because electronic feedback has been a hot topic in strategic targets over the last year or so, we thought it was about time that it deserved a newsletter to itself. So this edition you will find see some examples from a School context and a quick guide to the batch feedback tool in Blackboard if you’ve never tried it. We’ve also included some of the advice given to students in the Feedback campaign which took place at the end of last term – think about how to turn them round to apply them to your teaching!
Feedback can encompass many things, including ‘feed-forward’ as well as feedback, and more than one medium – not just electronic versions of text feedback but audio or videoed comments on submitted work too. One really interesting project presented at ALT-C in September was by Sue Rodway-Dyer, Elisabeth Dunne
and Matthew Newcombe from Exeter University on “Audio and screen visual feedback to support student learning”, and whose final research paper can be read at
Ken Newton has been looking at e-assessment and feedback in SAM recently. He notes:
In order to get a more qualitative assessment of SAM modules and perhaps a better understanding of the root of the quantitative information, I looked at all modules in the school in order to see how different staff created them. It was clear that in most modules, most of the information one would expect to find as a student, was there. The overarching issue is how easily can students find stuff - the balance of menu items to content within them - given
that most of this sat inside a module guide, usually provided as a linked file.
In the process of doing this work I realised that my own modules were not as well constructed as I had imagined. By looking at modules that I had no connection with it I became sensitised quite quickly to how some felt to welcome and support the viewer and others were quite impenetrable, and why. When looking at my own modules I realised that there were simple changes which would really help students and which had just not occurred to me. In my case I have relatively high levels of contact with students and pass a lot of information verbally on a day to day basis. The problem is that not all the students are necessarily there, they don’t necessarily pass on information , and if they do then it sometimes get lost in translation. “That’s not what I heard” “ “you
said” “so and so said that you said”. So for me, I have made the key information that students want to know accessible through the Menu Items, organise materials into more useful folder descriptions and am trying to make more information directly accessible on screen when possible.
E-assessment
This is an introduction by Liam O’Hare, E-learning Co-ordinator in SSE, to a project on electronic submission and electronic
feedback. It is based on three main sources of information.
• e-feedback project: CLQE and the SST (now SSE) L&T Committee funded a project to investigate the application of new technologies to improve feedback. This project is still ongoing, but the first round of findings are included here.
• Survey of Subject Groups within SSE. Each subject group in SSE was visited to solicit views on electronic submission and feedback.
• Survey of e-Learning Co-ordinators. E-learning Co-ordinators from other schools were asked to provide examples of their experience of electronic feedback. Most of the examples given are drawn from different forms of electronic feedback given on assignments submitted to e@t (Blackboard), although some others are included in the project.
Advantages of e-assessment
There are some commonly agreed advantages to e-assessment, which might be best considered alongside the disadvantages of paper assessment.
• 24hr remote hand-in. It is a real advantage to students, especially non-traditional students, to be able to submit a piece of work out of hours, from home. This is not possible on paper.
• Storage costs. We have to store student work in case it is required for review. If this is done on paper it represents a significant cost in space and a serious administrative burden. Electronic storage is much cheaper, and tracing a particular piece of work is much easier.
• Tension between feedback and quality assurance. In some cases the desire to retain an unmarked paper copy to pass on to second markers and externals has meant that staff are prohibited from writing on the student’s submission; many feel that this inevitably detracts from the quality of feedback provided. In other cases staff do write on the submission, but then have to choose between returning it to the student for feedback purposes or keeping it for the second
marker and the external. In such cases, marked work may even be returned to students in class for brief inspection, and then collected in again for the
module box! Electronic submission means that there is always a pristine copy in the database, which can be consulted, annotated and marked independently and simultaneously by 1st and 2nd markers, and also passed on to the external.
• Secure return to individual students. Returning marked paper assessments and feedback to students is not always possible in class (for example, when
an assessment comes at the end of a topic or block) and so can be difficult in some cases. Thus, in the past it was not uncommon to see boxes of marked work or folders of feedback sheets waiting for collection outside staff offices, and occasionally it still happens. However, by some readings of the data protection act, it is illegal, so various hand back arrangements have been organised in different schools. To some extent all of these are inconvenient to staff and students and costly to the school. All of this is overcome if electronic submission is accompanied by electronic feedback.
• Better reflection. It is useful for students to build a portfolio of their assessments and feedback so that they can reflect on them and learn from
them rather than just stumble from one to the next. This is much more convenient when the assessments and feedback are stored for them in a
single VLE.
• Potential for richer assessments. As access to technology improves, there must be potential to use other assessment types, perhaps incorporating more collaborative work, more effectively structured self and peer assessment, better assessment of process, better alignment of assessment and learning, more engaging media. There is also potential for more flexible assessment arrangements, even “assessment ondemand”.
• Quicker, more individual, feedback. There remains the vision of quicker results. Objective assessment which already allows instant feedback as well as diagnostic information on a learner’s performance may not always be applicable, but perhaps we should think harder about designing other assessment types on which the computer can help us provide more rapid individual and personal feedback.
• Plagiarism detection. Just as cut and paste plagiarism is easier to do electronically, so is it easier to detect.
Staff Perceptions of e-assessment
The information gathered supports the view already formed during the survey of subject groups in SSE; the details of assessment practice vary from subject
group to subject group, from academic to academic and from assignment to assignment. I have characterised the wide range of current practice into broad
groups.
1. Some staff use the quiz tool in Blackboard for objective assessment, mainly formative but also some summative assessment. Other similar tools are rarely used.
2. Many staff have moved completely to electronic submission (where this is possible – see below) and, building on this, have adopted a range of strategies for provision of electronic feedback.
3. Some staff are collecting some of their coursework electronically, but are not providing electronic feedback. Generally these staff are printing the
work, and providing hand written feedback.
4. Some staff are providing electronic feedback on work that has been submitted in paper form. Some of these are providing feedback by e-mail, while others are using the “comments” field in Blackboard.
5. Some staff are continuing to collect paper assignments and providing paper feedback as before.
6. Many examples of assessments that are not amenable to e-assessment have been identified.
It isn’t clear how many staff fall into each group. Indeed, it seems likely that very few staff can be fitted exclusively into any one group as there is a tendency to adopt different strategies for different assignments. Since the initial survey, the University has made a major push to require electronic submission (except where this is impossible) and electronic feedback, and as a result groups 1 and 2 have grown.
The evidence suggests that much of the variation is well founded in the motivation to provide students with an engaging and varied experience of assessment. Many examples were provided of interesting assessments and valuable approaches to rapid, personal, feedback that did not have an obvious electronic equivalent. There remains, however, a view that the majority of assessments are based on documents that could be submitted electronically and that most feedback was either in the form of text or as the spoken word. Most agreed that, for these assessments, electronic submission and feedback offers many of
the potential advantages mentioned but there were still concerns that, even for these assessments, there were technical barriers to universal acceptance. Some of these are dealt with in the larger report, but many of them are due to a lack of confidence in the academic merit of new technologies or as a result of custom and practice.
Liam O’Hare - E-Learning Co-ordinator – SSE
Feedback on assessment in SAM
Electronic feedback was one of the targets for completion during academic year 2008-09.
The School Policy is that students receive feedback within 20 working days, an element of which must be delivered electronically.
Within the School there are a wide range of disciplines using a variety of assessed material. This ranges from text based documents to three dimensional sculpture and dance performance.
In all cases the feedback material will be word based, either in the form of text or as the spoken word.
Much assessed material receives immediate formative and summative feedback in the form of comments by staff responsible for the assessment. This may have to happen as a result of material not being available at a later date for reference, or as a result of custom and practice – the critique system used in Art and Design for instance.
The initiative to encourage electronic feedback and now submission has meant that staff are looking at possibilities to use different mechanisms which are more effective and timely.
Staff in English have developed an A4 Word document with drop down options for each assessment criterion. This allows a quick way of covering the major points leaving more time to provide richer and particular commentary in a space allocated for it. A number of their colleagues are using this template, customising it for particular material.
Staff in Design are using Audacity on a USB stick to record feedback on work submitted digitally. In most cases the material is largely image based so the whole of the screen needs to be used to see enough detail. Using a voice recording means that none of the screen is required for text editors, and is much quicker. Using Audacity from a USB stick means that software does not need to be installed which generally requires administrator rights. This approach has also been used to record feedback after a student presentation, when feedback was given verbally at the time. Clearly there is duplication so recording the critique will be attempted next time.
Staff in Design are developing expertise in using screenr to create video of on screen material with voice over. This offers the opportunity to move the cursor over the work to identify areas being talked about.
Staff in Design use a matrix of learning outcomes against marks with each cell having a description of how the level of achievement meets the outcome. Cells are emboldened to provide a visual map of overall achievement underpinning the associated mark.
Using two screens is a powerful tool for creating typed feedback on Electronic submissions. The assessed material is held immovably in place on the second screen while other activities such as creating feedback and the sending of it can take place on the other.
Ken Newton | E-Learning Co-ordinator | SAM
Batch Feedback
Giving feedback for an assignment in a module with a large number of enrolled students can be a time consuming task. Normally this has to be done on an
individual basis via the Grade Centre in Blackboard, because of this we have created the Batch Feedback tool.
Using the Batch Feedback tool you are able to download all of the submissions made by your students for a particular assignment, create a file containing
individual templates for you to enter feedback and once you have edited the feedback files, upload it all back to Blackboard in one go.
The Batch Feedback tool has been designed to be flexible, letting you work with groups you have previously created in Blackboard and also use feedback
templates you might already have in your school.
It doesn’t just have to be used for giving feedback as documents, why not try using the Batch Feedback tool to create MP3 audio files as templates, the possibilities are endless! Contact the E-learning team () for information on how to enhance your Blackboard modules.
“Have you had your feed….”
The Feedback Campaign took place over a week in December2009. It included an online survey, Big Brother Diary Room, and tablecloths in many buildings with questions for open comments. The Students Union has analysed the data collected, and offers some recommendations from the students. Thanks to Jules Pringle, Education and Welfare Officer for the following and for some of the student feedback feedback!
Recommendations
The experience of the respondents is quite varied with descriptions varying from very good experiences to those dissatisfied with the feedback. The majority are positive comments. This is promoted but the commitment to improve is included with this.
There are many very positive descriptions of well delivered feedback and how students have benefitted from it. These are used in an aspirational description to academic staff for feedback.
There are some negative experiences which can be used to highlight the characteristics of poor feedback. That the minority of shortcomings are
acknowledged with the actions being taken.
Absent or late feedback is the biggest issue, this is borne out by NSS scoring in timely feedback. Management action is taken to ensure the 4 week standard is met, including monitoring KPI’s.
Variation in the detail and specificity/relevance of the feedback is another issue. A standard form and guidance/training to ensure the student gets specific and constructive criticism and some general comments to support ongoing development.
There are comments about the way (spoken, written) feedback is given. The methods used in the well received feedback is considered in the new feedback standard.
The feedback campaign was led by the Students’ Union (SU) and supported by the University’s ‘Feedback and Assessment for Learning’ (FAL) working group - taking place over one week from the 30th November 2009. It aimed to enhance students’ experiences with feedback by raising student’s awareness of different types of feedback and advising students on how to get the most out of their feedback. It also encouraged current students to give their opinions on feedback
provision across the University. Ten key messages about feedback were circulated widely to students on bookmarks and leaflets, and there were stalls
and stands around campus with tablecloths where students could give their views. A survey about feedback was posted on the SU web pages for on and off-campus students.
Vicki Hill - Research Assistant - CLQE
1. Feedback feeds learning. Feedback is not simply something you receive after your assessments have taken place but is a continuous part of your learning experience and comes in many forms.
2. All opinions count. Feedback can come from many different people, including: module leaders, classroom tutors, fellow students, employers, or you can even give feedback to yourself through reflection. All can be valuable to you.
3. Feedback has different forms. You will receive feedback in many different forms, which include: written comments about your work; verbal comments from your tutors in class or on a one-to-one basis; discussions with peers in the classroom or outside it; electronic discussion; emails; feedback grids and generic feedback proformas. Welcome all feedback and use it to learn.
4. Help feedback to work for you. Use your feedback to develop knowledge about your strengths and weaknesses. Reflect on the feedback you receive and think about what you have done well and how you could improve. For example, you could draw up an action plan based on your feedback.