AMTA 2012 Massage Therapy Schools Summit

Designing Rubrics

AMTA 2012 Massage Therapy Schools Summit

Chicago, IL

February 17, 2012

This is a 2 hour course for instructors on how to create and implement rubrics. Although they can be applied to written essay questions, research papers, projects and oral exams, our focus will be on rubrics for practical exams in Massage Therapy.

Learning Outcomes

At the end of this presentation, participants should be able to:

List three uses for a rubric in a Massage Therapy Program
Identify the parts of a rubric grid
Determine the hierarchy of skill description
Create a rating scale for a rubric
Construct a rubric for a written, oral, and practical exam

Presented by:

Kathy Paholsky, PhD, NCTMB

Owner, Optimum Health Center, LLC.

Massage Therapy Program Coordinator

Schoolcraft College, Radcliff

Garden City, MI 48135

 2012 Optimum Health Center LLC; Kathleen M Paholsky

This material is solely designed for seminar attendees. It may not be reproduced or distributed without written permission of the presenter. Note: the rubric samples provided in this handout can be used, modified and shared without my written permission.

Designing Rubrics

February 2012

Page 1 of 11

Rubrics

Rubrics are scoring tools for assessments.

Rubrics are standardized scoring tools. The word rubric means red and originally referred to portions of text that were printed in red ink to highlight information. When applied in the educational setting, the rubric is a scoring tool for assessments.

The simplest rubrics have ___pass/fail implications______.

Either the student performs the task or does not.

Rubrics can increase in complexity based on quality, quantity and skill of the work performed; they identify levels of potential achievement for each criterion.

At no time should the rubric complicate your assessment. In all cases it should add to the understanding of what needs to be done by both the student and the instructor.

Rubrics are typically given to students ___before the exam______.

Use them as working guides. It doesn’t serve the student or instructor if the exact requirements of an assessment are not clear.

Rubrics

let the instructor identify the _____skills and performance______necessary
let __students______know what is expected and how to be successful; they can prepare accordingly
allow the assessment to be more ______objective______
evaluate student on the full range of criteria instead of a single score
are authentic; they measure according to real-life criteria
vary expectations based on skill level: ex: beginners vs. advanced

Who?

What?

When?

Where?

Why?

Benefits of using rubrics

Challenges of using rubrics

The Value of Rubrics

Rubrics standardize assessment grading.

Once the criteria are identified, rubrics allow for:

consistency in _____assessment______
alignment with ____learning objectives______
less test bias: reflection before reaction
evaluate subjective material in the most objective and measurable way
improved student motivation______
repeatability with multiple evaluators
objective information to provide for students continued progress
evaluation of how the item was taught; course corrections needed

Rubrics are usually expressed in __a grid______that contains measurable/observable criteria in incremental designations. It also contains the values for each of those criteria. This allows for consistency and repeatability in assessments by more than one evaluator. A quality rubric will replace vague criteria with specifics.

Although it can be applied to a simple pass/fail exam, a rubric allows for expansion and explanations.

Rubrics:

show students their _strengths and weaknesses______in each assignment.
show instructors and course designers the specific areas of strength and weakness in student understanding and performance.
provide a start for ______communication______when examining these areas.

Designing Rubrics

Rubrics are recyclable!

Rubrics need to be__useful______and ____totally meaningful______at many levels for assessment.

Rubrics align the evaluation criteria to the curriculum and learning outcomes.

Rubrics initially take a while to create, yet they are one of the best investments of instructor time. Include them in your planning and preparation!

Rubrics___organize information______that allows for checks-and-balances not only of student skills but of the course itself. Although they can be created by one person, using other faculty ensures a better tool. Discussions with peers when creating rubrics bring to light differences in how we each identify the individual processes, the value we see in each item, and how a subject is taught. In some cases, students can provide valuable input to the design of a scoring rubric.

When designing a rubric, start simple then identify any important factors that may be missing. They can be specific or broad, detailed or generalized.

The value assigned to any of the elements allows for prioritization of each element. _You_____ determine the values assigned.

Here are some steps that can help when Designing a Rubric

Identify the skill (or item) to be evaluated.
Brainstorm all qualities and components; make a checklist
Find 3 to 5 main criteria for each skill; categorizeand create descriptors
Create a rating scale (Q). It may be letters, numbers, or terms of value
Create a chart. Criteria on left side and rating scale along top
Describe the levels, from excellent to poor

Criterion 1 / Criterion 2 / Criterion 3
5 points
3 points
1 point

Helpful Hints:

Describe the ideal or model response first

Use absolute language: be specific, don’t say ‘always, seldom, good, appropriate

Review them with students before the test.

Designing Rubrics

February 2012

Page 1 of 11

SAMPLE RUBRICS

Rubric for Lubricant Use

Here is a simple rubric that provides different points for skill with the lubricant.

Simple Rubric for lubricant use

3 / Student uses sufficient amount to apply strokes with ease and no excessive slippage
2 / Student needed to reapply or remove lubricant in more than one area
1 / Student frequently uses excessive or insufficient amounts of lubricant

Notice that the number of points is in a small range. If the use of lubricant is an important factor in your assessments, the numbers can be increased and additional criteria added.

Rubric for Draping

Since each of us may have a different idea of what is modest and what is comfortable, a rubric would provide clarity for students who are having difficulty performing one or both of these tasks consistently during an exam.

Example of designing a rubric for draping.

What are students expected to perform?

Draping for modesty
Draping for warmth

How are they expected to perform?

At beginning or advanced student skill level, or just draping?

Draping is tentative and therapist hesitant when client is changing positions
Student drapes for modesty and warmth at least 70% of time
Student does not expose client at any time (rewrite as a positive)

Designing a rubric for draping would include draping for both modesty and comfort:

Points / Modestly draped / Points / Comfort draping / Points / Skill level
5 / 100% / 5 / 100% / 5 / In command of materials
3 / 75% / 3 / 75% / 3 / Hesitant
1 / 60% / 1 / 60% / 1 / Very tentative
0 / 60% / 0 / 60% / 0 / Unskilled

What do you think?

Where would you add “exposed client once during session”
Where would you add “client expressed concern about exposure”

In addition to a grid, a few comment lines can be made available below the grid or on the student exam form for any specifics to individual student performance.

Example: Let’s say a student keeps the client well covered most of the time yet gets the sheets twisted up or lets them slide around when the client rolls over. In addition to the points given, provide feedback for corrective measures.

Comments: Student drapes for modesty and warmth at least 75% of time. Needed reminder from client to cover exposed leg.

Draping is awkward and therapist hesitant when client is changing positions

More columns could be added that address the student’s ability to use the draping material as a barrier for hygienic purpose, to add additional draping materials for a client in side-lying.

Rubric for Draping Material

Are the students graded on the draping materials themselves? If so, that can be separate from the rubric, possibly as a check list.

Draping materials are:

Clean
Fragrance free
Opaque
Natural materials (or hypo-allergenic)
Size appropriate for client and table
Sufficient draping material for warmth and modesty (extra towels, linens, etc)

Items can be added or removed as needed

What do you think?

Would you create a grid for this item?

Rubrics for Technique Assessment

During a practical exam, students are assessed on multiple techniques. Imagine that each of the strokes/manipulations has a value of 5 points toward the total.

This is an example of a simple rubric designed to assess massage strokes. Additional criteria could be added for hand placement and use as well as the comfort to the client.

Value / Quality / Criteria #1 / Criteria #2
5 / In command of stroke / Effectively accesses and mobilizes tissues, identifies areas of restriction / Performs with skill and ability to change pressure levels and speed
3 / Some hesitation / Effectively accesses tissues with some mobilization; unsure of areas of restriction / Performs mechanically
1 / Tentative / Accesses tissue with no attempt to move or mobilize / Unable to perform without coaching
0 / Unskilled / Did not perform; does not know technique / Did not perform; does not know technique

Here is the same rubric designed to assess petrissage specifically.

Value / Quality / Criteria #1 / Criteria #2
5 / In command of stroke / Lifts and mobilizes tissues, identifies areas of restriction / Performs with skill and ability to change pressure levels and speed
3 / Some hesitation / Lifts tissues with some mobilization; unsure of areas of restriction / Performs mechanically
1 / Tentative / Pushes or rolls tissue with no attempt to lift or mobilize / Unable to perform without coaching
0 / Unskilled / Did not perform; does not know technique / Did not perform; does not know technique

Designing Rubrics

February 2012

Page 1 of 11

Bibliography

BOOKS:

Burke, Kay. From Standards to Rubrics in Six Steps/ 3rd Edition. Corwin Press: October 2010

ISBN: 9781412987011

Gross Davis, Barbara. Tools for Teaching/2nd Edition. Wiley, John & Sons: February 2009 ISBN: 9780787965679

McKeachie, Wilbert. McKeachie’s Teaching Tips: 13th Edition. Cengage Learning: January 2010

ISBN: 9780495809296

WEBSITES:

The following Websites were accessed multiple times in December 2011 and January 2012.

How to Create a Rubric

Kolar, Rachel. How to Design an Assessment Rubric. July 09, 2011

Langley, Michelle. How to Design Scoring Rubrics for your Classroom

Discovery Education “Kathy Schrock’s Guide for Educators”

Designing Rubrics

February 2012

Page 1 of 11