The Higher Education Academy
Feedback relating to Department workshop at Royal Holloway University of London, 13 February 2013
Lynn Vos – Discipline Lead Marketing, interim DL Finance and
Richard Atfield - Discipline Lead – Business and
1.
Issues related to marking, assessment and feedback. / Marking issues – should marking be completely anonymous (first marker not known to student) / Downsides of anonymous marking:
- Creates distance between the learner and the teacher. One of the most powerful influences on student achievement (Hattie, 2009) is a positive, trusting and open relationship between student and teacher. Feedback from a known teacher has a greater developmental influence on the student.
- Weaker students may not come for supervision if they do not know who will be marking their work.
- Does not necessarily remove bias in marking. Bias in marking is made greater when there are no generalised marking criteria that have been discussed and agreed upon by staff. Bias occurs when markers place emphasis on different aspects of the dissertation (for one, it might be the appropriateness and depth of discussion of the methodology; for another it might be the data analysis section and recommendations), so prior discussion amongst all markers as to key criteria is helpful.
To what degree do markers make value judgements particularly with their own students? / As long as we use evaluative forms of assessment, there will always be room for some degree of subjectivity in marking and some value judgements. This can never be removed entirely.
Tutors who have seen strong commitment and desire to improve from their students are likely to give some marks – consciously or unconsciously – for these characteristics. Part of the development opportunities provided by tutors to students is that of encouragement and recognition of a student’s intellectual and personal growth through positive feedback. To reduce bias, subjectivity in marking and value judgements as much as possible, use marking criteria that have been discussed and agreed to by all dissertation markers on a programme. However, continue to encourage good relationships between supervisors and students for the positive benefits this brings.
Marking issues – how do we deal with markers having different issues and preferences, thus leading to large gaps in grades given. / As noted above, carefully laid out and agreed upon criteria can help reduce gaps in first and second marks. Use third markers where the gap remains large.
Set marking criteria with a holistic component (for internal consistency, balance, argumentation, etc. (see) and criteria for each section. The holistic component should be given the balance of the marks.
Marking issues – should we be using vivas? / Vivas are a useful means of:
- Helping to determine if the student has done the work;
- Determining how well a student can develop and verbalise a sustained argument;
- Providing further evidence of dissertation quality for both first and second marker.
If tutors are losing confidence in whether students are submitting their own work (rising plagiarism), then other solutions may be needed: a company project rather than a traditional dissertation; stricter student/supervisor meeting requirements where student bring drafts of their work and are able to discuss it – for example.
Marking issues – different types of dissertations are offered on our programmes, and yet our marking criteria is the same for all / Clear, meaningful marking criteria should be set in advance for each programme and discussed with the students to ensure that they understand (or understand over time) how they will be assessed. Hattie (2009) noted that one of the most powerful influences on student achievement is knowing what the learning intentions are of a particular assignment and having clear success criteria to work towards.
Marking issues – if we use clear marking criteria that students are made aware of, might this not lead to grade inflation? / Good teaching = making the learning intentions clear + providing clear success criteria+regular opportunities for the student to practice and get feedback+ high expectations + willingness of teacher to get feedback from the student on what they (the teacher) could do to improve. (Hattie, 2009; Gibbs, 2010)
If this leads to grades increasing, then a tutor is demonstrating good/quality teaching.
We should however make it clear that we want our students to succeed and do better. Grades will likely remain in a normal distribution, but with a higher mean.
Marking issues – are we allocating sufficient time to marking and moderation? / Probably not – an institutional issue
To establish academic understanding and application of the dissertation marking criteria
(there are other versions of marking criteria in the booklets which should be made consistent throughout for the benefit of staff and students) / Using survey software like Survey Monkey, set up the criteria used in the marking guide and ask all academic staff to indicate what percentage of marks they allocate to each criterion. This would give an anonymised indication of marking consistency which is likely to also influence how they offer feedback to drafts, at supervisory discussions and in final feedback.
This could also be set up for students to gain insights into their current understanding of each section.
2. Issues related to student preparedness and commitment/ issues in the student/ supervisor relationship. / Regular meetings – how do we encourage students to come to meetings and to stay in touch over the summer break? / Consider using a student/tutor contract of some type. Establish clear expectations of what is expected up front – from both the student and the tutor. Make a record of the contract, refer to regularly to see that both parties are keeping their end of the bargain, and set minimum guidelines. For example, if the student has not met with the tutor three times face-to-face (or via Skype), then the dissertation will not be marked.
How can we improve student engagement with the dissertation process? / Start from week one of the degree.
Identify the stages (and times) when engagement wains. Determine the reasons why this is occurring (cognitive issues; affective; time-related/social (other assignments, students having to work to support themselves (see Vos, 2013). Put in interventions where needed.
Reconsider how the dissertation is structured; change to another form of assessment.
Use collaborative cohort groups – provide guidance in how to get the most out of these groups; set regular meetings with groups; set clear criteria for the group – helps to share information and provide support for students.
Developing student independence and agency – what do we need to do to help students without “spoon feeding” them? What should be the balance between detailed guidance from the supervisor and independent work by the student?
How can we move students from being passive in the process to more active and engaged? / Identify stages in the dissertation where students are most challenged/least motivated – develop specific interventions that can be delivered via collaborative cohort groups or action sets.
Provide clear set of expectations up front as to what the supervisor will and will not do. (e.g.: Mark drafts of a few chapters only) Try to get consistency amongst colleagues.
Ensure students have chosen a project that they are truly interested in and that is manageabl and provide specific interventions for topic selection and focusing.
Provide more specific support and/or training upfront.
International students – many international students do not know how to write assignments (UG in particular) / Consider pre-enrolment training for those with English as a second language. Ensure training is discipline specific (Hattie, 2009 notes that additional training seminars work better when they are specific to the discipline and not simply general for all international students). If pre-enrolment training not possible, work with your Learning Development Unit to put on additional training during the academic year.
3. Challenges with particular stages in the dissertation process. / How well are we preparing students for the dissertation (e.g.: research methods training)?
Do students really see the relationships between what we are teaching in research methods and their dissertation?
How can we improve transfer of knowledge from the research methods course to the dissertation? / As noted above, set clear criteria, agreed by all and provide up front. Ensure students understand what each criteria means. Provide exemplars of each criterion and use in dissertation training sessions.
Ensure that what you expect students to do in a dissertation is being addressed in the training (e.g.: we want the student to be able to “critically reflect” on the literature and to develop an “argument” as well as select and defend an appropriate method, analyses findings, etc. – are you providing training in critical reflection and argumentation?)
As noted above, identify typical areas where students are weak in their dissertations or have particular challenges – address with specific interventions.
How can we help students choose appropriate topics for the dissertation? / Use collaborative cohort groups at early stages by supervisor. Have each student bring their topic idea – discuss each student’s proposal. Have other students provide suggestions and critiques.
4. Plagiarism and academic dishonesty / How can we reduce plagiarism and the possibilities of students’ purchasing their dissertation or getting external help that goes beyond simple editing of their work? / Change from a traditional dissertation to a more focussed and specific project.
5. General issues / Is it time to consider an alternative to the traditional dissertation? / Review grades, plagiarism, quality of dissertations over the past 3 years. Are students really meeting the learning outcomes set out for the dissertation? If not, why not? If not, can another project be substituted that still allows students to achieve the key learning outcomes but is more suited to the time scale, the needs of employers, and the available time for training and supervision.
Are we asking too much of students in the short time scale given for researching and writing up the dissertation? / Yes, probably. See above.
How to deal with large variations in student numbers between supervisors? / Institutional/department level issue of fairness in work load.
How can we improve feedback from students on their experiences with supervision and the dissertation? / Clear criteria shared with students from the beginning; student-supervisor contracts or agreed terms of reference; additional training and support.
Gibbs, G. (2010). Dimensions of Quality. York: Higher Education Academy
Hattie, J. (2009).Visible Learning: A Synthesis of over 800 meta-analyses relating to achievement. London: Routledge.
Vos, L. (2013). Dissertation study at the postgraduate level: A review of the literature. York: Higher Education Academy