Rubrics to Promote and Assess Student Learning

Rubrics to Promote and Assess Student Learning

Center for Teaching Excellence – DuquesneUniversity

(Adapted from 02/10/15workshop led by Ryan Luchs, Marketing faculty member & Laurel Willingham-McLain, CTE)

Definitions and Purposes

What is a rubric?

A rubric is a rating scale that makes explicit the criteria and standards for judging students’ work on discussion, a paper, performance, product, show-the-work problem, portfolio, presentation, essay question—any student work that involves an evaluation of quality.

Rubrics serve to inform students of expectations while they are learning. These tools also enable teachers to grade efficiently, communicate readily with each student, and rate students’ work against a standard rather than against one another.

Some rubrics rate each criterion separately. Others group several criteria in a holistic scoring guide.

What is a checklist?

The development of a rubric is usually a “work in progress,” something to be improved with experience. Often a first step is simply to list out the criteria in a checklist. In simple assignments, a checklist is all you need for giving feedback and helping students assess their learning. Students might complete a checklist indicating that they have completed all the steps in the assignment, and submit it with the assignment.

Purposes Served by Rubrics

• Increasing reliability and validity of assessment. Reliability may be enhanced when a scoring rubric with well-defined criteria and levels of achievement is used. Two different raters are more likely to agree in their judgments of quality in this situation. Also, a single rater is more apt to be consistent when grading many papers at a particular time (minimizing the “fatigue” factor) or when responding to students who differ in the quality of their previous work (minimizing the “halo” effect).

If the instructor develops a rubric in which the students’ tasks directly draw upon the intended knowledge and skills, then content-related validityis enhanced.

• Providing rich feedback to students on their performance. In a problem-solving task, compare the information conveyed by a score (e.g., 85%) to that of a rubric, which identifies areas of misunderstanding or omissions in the problem-solving process.

Or, compare the information conveyed by a grade on an essay (e.g., “C”) to that provided by a rubric that rates performance in areas such as content, quality of argument, style, and grammatical correctness. The scores and grades tell the learners the instructor’s overall rating of their performance and, perhaps, how they performed relative to other students in the class, but scores alone don’t offer guidance on how to improve.

• Assisting students in performing to the best of their ability. Often faculty observe that some students “just don’t get it” in understanding what faculty expect an assignment to look like. “What should be included? If the content of my paper is correct, why didn’t I receive full credit?”

Students who exceleasily have often developed a tacit understanding of what is expected, even though the instructor does not explain this in detail. Less prepared students may lack this tacit knowledge, but would perform better if the criteria and standards by which they would be evaluated were made explicit. If the goal is for students to learn how to do good work, faculty may help them by initially providing the structure of a rubric. This is especially pertinent to complex tasks.

• Are we spoon feeding students? When students encounter new kinds of complex tasks, scoring guides are useful in clearly setting forth the expectations. As students become familiar with the conventions of writing or performance in your field , the instructions and feedback should be less structured and detailed.

Involve students in writing scoring guides, especially when this simulates a task they will face in their career (e.g., work performance reviews, analysis of team work, revision of their own or others’ writing, evaluation of a product, blind reviews of articles, search committees). If time is short, give them a skeletal draft to fill in based on sample work they review.

Require students to review their peers’ work. Rubrics help structure this. Choose those areas where students can legitimately evaluate peer work and help one another.

• Effort and “desirable difficulties” along the way Research points out that deep learning requires effort and the challenge of “desirable difficulties.” Learning that is easy, where the concepts feel immediately familiar, doesn’t tend to “stick” (Brown, Roediger, & McDaniel, 2014). Rubrics are guides, but they are not intended to assure students get it right the first time; challenge, effort, and risk taking are crucial to deep learning (Dweck, 2006).

• What if students don’t pay attention to the feedback? Ask the students to respond to how their work was assessed in a brief note to the instructor and to themselves. For example, have them list strengths in their assignment, areas for improvement, and specific steps they will take prior to handing in the next assignment.

• Guiding faculty in planning instruction. Once the faculty member has developed well-specified criteria and expected levels of achievement for a task, it may become evident that students will need guidance and practice in various subtasks. For example, critical thinking in sociology may require understanding a political or economic context; and developing a persuasive argument about a historical issue may require understanding how primary sources can be used as evidence. Some tasks we set for students are fairly complex and we need to assure that they know how to carry out elements of the task before orchestrating these elements into a finished product.

Rubrics help faculty do a quick analysis of student work to see patterns of strength and weakness, of development and progress.

Examples of four types of grading guides

CHECKLIST

From

HOLISTIC RUBRIC

Brief written essay (10 points total)

9-10 points: The thesis of the paper is clear. Many relevant details from the readings and films are given to support the thesis. Information is presented logically and compellingly. Word choice, spelling, and sentence structure are consistently correct.

8 points: The thesis is quite clear and is supported by several relevant details from the readings and films. Information is presented in a logical manner that is easily followed. Word choice, spelling, and sentence structure are generally correct (very few errors)

7 points: The thesis is somewhat difficult to interpret, and only a few relevant details from the readings and films are given to support it. There are places where it is difficult to follow the flow of the paper. There are several errors of word choice, spelling, and sentence structure, and these get in the way of understanding the paper.

4-6 points: There is an attempted thesis, but it is unclear. The relevant details from the readings and films are either unconnected to the thesis or too few to support the thesis. The paper is hard to follow. There are many errors of word choice, spelling, and sentence structure, that stand in the way of understanding the paper.

RATING SCALE - excerpt

English Essay

Quality of Argument / Content

Does the paper indicate mature understanding of its topic and reflect originality, college-level thought, and effort of the writer? Does it use effective examples and mature reasoning to support its overall analysis? Are directions for the paper given by the instructor followed appropriately?

(no)12345 (yes)

Thesis Statement

Does your paper contain a clear, perceptive, arguable sentence (or group of sentences) which articulates its central point? Does the thesis explain how and/or why that central point is significant? (no) 1 2 3 4 5 (yes)

Excerpted with permission of Amy Criniti Phillips, Wheeling Jesuit University (Ph.D. Duquesne University)

DESCRIPTIVE RUBRIC - excerpt

Excerpted with permission of Corina Wack, Chowan University (Ph.D. Duquesne University)

What purpose, advantages, and disadvantages do you see for each type of grading guide?

BlackBoard Tools

The Duquesne University Educational Technology office supports the use of grading tools within Blackboard. These can save faculty a lot of time.

The AAC&U VALUE Rubrics

• These rubrics are part of the Association for American Universities & Colleges larger project, Liberal Education and America’s Promise (LEAP)
• Context: legislators accusing faculty of not knowing how to assess learning; exerting enormous pressure on regional accrediting agencies and on universities; distrust of universities; and threat of creating federal college standards.
• AAC&U led national initiative to articulate agreed upon principles and outcomes related to undergraduate education: “The LEAP Essential Learning Outcomes represent a consensus among educators and employers about the kinds of learning students need as preparation for successful participation in civic life and the global economy” (Rhodes & Finley, 2013, p. v).
• They prepared 16 descriptive (or analytic) rubrics, known as VALUE rubrics, for “Valid Assessment of Learning in Undergraduate Education.” There is one for each of the Essential Learning Outcomes. These are “meta” rubrics, compiled from many institutions and contexts, outlining components considered essential across many contexts.
• These rubrics help us address complex kinds of learning, such as:
• Inquiry & analysis
• Critical thinking
• Creative thinking
• Civic knowledge & engagement
• Ethical reasoning & action
• As meta or high level rubrics, they give excellent insights on what we value nationwide, and will need to be re-contextualized to our local educational purposes and settings. They also need to be adapted to the developmental level and scope of learning in our context (e.g., unit, course, degree, core curriculum, honors college levels).

Steps to creating your own rubric

1. Choose a specific assignment. Jot down a list of what you’re looking for in this assignment.

1. From that list, determine some grading criteria. Write these in terms of what you expect students to KNOW (content) and DO (skills)?

1. Use this wording: “students will…”

1. Write these in the left-hand column of the chart under criteria.

1. Assign points. Is each criterion worth the same number of points?

1. Now describe what top-level student work might look like for these criteria. Write this in the “A” column. Write what you imagine a “D” assignment might look like. Jot down some preliminary notes for the “B” and “C” columns, as appropriate.

Later when you are using the rubric, flesh it out based on what you see students doing. It’s hard to do this in the abstract.

As you use the rubric, you may also find yourself revising the way you word the assignment.

Creating a Rubric: Practical Questions to Consider

1. How might a scoring guide enhance your students learning? Focus not on techniques and tools, for their sake alone, but on communication and meaning.

1. Do you have actual student work you can use in order to develop the wording for levels of achievement or do you need to think about this in the abstract and refine it later? If the latter, consider starting with the highest level of work you expect and then moving from there down.

1. How new or complex is the task you’re asking students to do? To what extent does the feedback need to be structured and detailed?

1. How will you encourage students to develop their ability to approach complex assignments on their own (in order to prepare for a world where reality rarely fits preset criteria neatly).

1. How can students take more responsibility for their learning? In what ways might it be relevant to your course goals to involve students in the drafting or editing of a scoring guide. Would it help them learn if they used the scoring guide for self or peer evaluation?

1. What is the degree of creativity and flexibility in the assignment? If it’s high, the scoring criteria will need to be broader.

1. How well are you introducing students to the language of your field? Have you defined terms or provided models for conventions that may be unfamiliar to them?

1. What kinds of names will you use for the levels of achievement, if any? A,B,C, etc? Distinguished, proficient, basic, unsatisfactory, no evidence? Novice, apprentice, practitioner, expert? What terms make sense in your field?

1. How many levels of achievement do you need to distinguish? Is each criterion of equal importance or do you need to weight them differently? Are the descriptions of each level clear to others?

1. Have you left room for the unexpected? How would you distinguish creative, inspired work from that which meets the criteria in a flat, uninspired way (Suskie, 2004, p. 135)?

Rubrics to Promote and Assess Student Learning, CTE, August, 2015, page 1

Grading Rubric Worksheet

Course Title:______Name of Student: ______

Assignment Title: ______

Brief Description of Assignment:

Rubrics to Promote and Assess Student Learning, CTE, August, 2015, page 1

Selected references

Association of American Colleges & Universities. (2009). Valid Assessment of Learning in Undergraduate Education (VALUE) , Requires you to give some information to log in, but they are freely available. Provides rubrics for the 16 Essential Learning Outcomes Faculty are encouraged to adapt these to their course and program contexts.

Brown, P.C., Roediger III, H.L., McDaniel, M.A. (2014). Make it stick: The science of successful learning. Cambridge, MA: Belknap of Harvard UP. Importance of making mistakes and effort in learning. Need for frequent (self) testing – to find out gaps in learning and to practice retrieving what you have learned to consolidate memory.

Doyle, T., & Zakrajsek, T. (2013). The new science of learning: How to learn in harmony with your brain. Sterling, VA: Stylus. Written for learners and teachers. Practical, evidence-based approaches to deep learning that lasts.

Dweck, C. S. (2006). Mindset: The new psychology of success. New York: Ballantine. Importance of risk taking and persisting in the face of learning setbacks. Mistakes and effort are central to growth mindsets.

Kuhs, T. M., Johnson, R.L., Agruso, S. A., & Monrad, D.M. (2001). Put to the test. Portsmouth, NJ: Heinemann. Chapter four: Guides to scoring student work: Checklists and rubrics. K-12 focus, but useful examples.

Rhodes, T. L., & Finley, A. (2013). Using the VALUE rubrics for improvement of learning and authentic assessment. Washington, DC: American Association of Universities & Colleges.

Stevens, D. D., & Levi A. J. (2005). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning. Sterling, VA: Stylus. Guide to developing rubrics, including with groups of faculty.

Suskie, L. (2009). Assessing student learning: A common sense guide. San Francisco: Jossey Bass. Part one gives excellent foundation for understanding assessment. Chapter 9 introduces scoring guides and rubrics.

Walvoord, B. E., & Anderson, V. J. (2010). Effective grading: A tool for learning and assessment (2nd. Ed.). San Francisco: Jossey-Bass. This book provides practical tips and tools for both grading at the course level and examining learning across the curriculum of an academic program. There is a lengthy discussion and many examples of rubrics. Also, chapters 6 & 7 are excellent on managing one’s time and grading efficiently so that students truly learn.

Rubrics to Promote and Assess Student Learning, CTE, August, 2015, page 1