How to interpret the ADI scores

Contextual items (3)These are items which featured in our early research analyses[i], but colleagues may well want to construct their own contextual items depending on their focus of interest. Each item scores a minimum of 1 and a maximum of 5, but there is no overall ‘contextual score.’

7. There is a separation between my assessment philosophy and my assessment practices.
13. My experience on the University teaching programme has changed my views on assessment practice.
15. New assessment methods are needed to improve current practice.

Desirable practice factor (7 items: minimum score 7 – maximum score 35)

3. In my practice I emphasise assessment for learning rather than assessment of learning.
4. I involve students in an ongoing dialogue about assessment criteria.
5. I design my assessments to help students take responsibility for their own learning progress.
9. I allow students the freedom to choose how their assessments are shaped (e.g. choice of topic, method, criteria).
14. Involving students in the assessment design would encourage them to engage in the assessment task.
16. I underpin my assessment practice through reading the literature on learning and teaching.
17. My assessments are designed to be an authentic reflection of real world tasks.

Constraints factor (8 items: minimum score 8 – maximum score 40)

1. If external constraints (e.g. QAA, subject benchmarks) were removed, I would be more willing to change my assessment practice.
2. It is possible for students to ‘go through the motions’ to satisfy assessment requirements without learning anything.
6. Innovation in assessment is not well received by students.
8. Changes to my assessment design are hindered by external factors (e.g. cost, high student numbers, time).
10. QAA requirements seldom allow leeway in assessment design.
11. I find it a challenge to design pedagogically sound assessment because of student focus on grades.
12. There is little incentive for lecturers to innovate in their assessment practice.
18. In order to introduce innovative assessment I would consider avoiding official procedures.

Scoring key: Strongly Agree (SA) = 5; Agree (A) = 4; Disagree (D) = 2; Strongly Disagree (SD) = 1; Uncertain (U) = 3.

Interpreting the ADI scores:

  • A profile of higher scoring than the mean(21) on the total desirable practicescore and lower scoring than the mean (< 24) on the total constraintsscore would indicate an individual who does not feel constrained when putting into practice ‘good’ assessment design.
  • A profile of lower scoring than the mean (21) on the total desirable practicescore and higher scoring than the mean (24) on the total constraintsscores would indicate an individual who feels hindered when putting into practice ‘good’ assessment design.
  • More detailed comparisons can be made with the data to be published in a forthcoming paper by contacting Professor Lin Norton, LiverpoolHopeUniversity,

References

1

[i]Norton, L (2010) ‘New lecturers’ views of assessment: Can they put into practice what they learn on a university teaching programme?’ Three day workshop on the impact of training for teachers in higher education funded by the European Science Foundation in Bratislava, Slovak Republic, 18-20 March.

Norton,L., Norton, B. & Shannon, L. ( 2010) ‘The Assessment Design Inventory:
A tool for research & faculty development’. Paper presented at the 6th international conference of the International Society for the Scholarship of Teaching and Learning (ISSOTL10), Arena and Convention Centre, Liverpool, UK 19-22 October 2010.

Norton, L., Norton, B., Shannon, L. & Phillips, F. (2009) ‘What do lecturers feel about assessment?’ Assessment in Higher Education conference, University of Cumbria in Carlisle, July 8th, 2009