HANDOUT #3.1
Module 3.1: Develop Assessment Scoring Keys and Rubrics
A scoring key provides the correct answer and point value for a selected response item.
3.1.1 Workflow: Develop Assessment Scoring Keys (SR Items)
3.1.1 Procedural Steps: Develop Assessment Scoring Keys (SR Items)
- Enter the assessment information at the top of the Scoring Key Template.
- Record the item number, item tag (optional), item type, and point value.
- Record the MC answers in the Answer column.
- Repeat Steps 1-3 until all items on the test specifications are reflected within the Scoring Key Template.
- Validate that each question-to-answer relationship is recorded correctly.
3.1.1 Scoring Key-Social Studies Grade 4 (Example)
Assessment Name / Grade/Course / Administration / Total Possible PointsSocial Studies Final / Elementary Social Studies-Grade 4 / Post-test / 50
Item # / Item Tag / Item Type / Point Value / Answer
1 / 0001.SOS.GR4.POST.MC-LV1-GEO2.1 / MC / 1 / B
2 / 0002.SOS.GR4.POST.MC-LV1-GEO2.4 / MC / 1 / A
3.1.1 Scoring Key: QA Checklist
Task / TaskQuestionArticulation / Are all items articulated in the test specifications represented within the Scoring Key?
Validation / Is there only one correct answer for every item?
Queuing / Are the answers arranged in a way that does not create a discernible pattern?
Balance / Are the answers “balanced” among the possible options?
Revalidation / Are the answers revalidated after the assessment has been reviewed?
3.1.2 Scoring Rubric Examples
- Holistic/Single-dimension rubrics:
- are used when performance criteria cannot be separated clearly;
- combine all performance criteria for simultaneous evaluation; and,
- requirethe scorer to enter only a singular point value based on the student’s performance as a whole.
- Analytic/Multi-dimension rubrics:
- are used for more complex tasks that measure many skills at once;
- separate performance criteria into separate dimensions and each criterion is scored individually; and,
- provideevery scoring level of each different criterion with its own statement that specifies guidelines for attaining that specific level of achievement.
3.1.2 SCR Holistic Rubric-Reading Grade 6 (Example)
Sample Response for: “In two complete sentences, explain why people should help save the rainforests.”2 points / The student’s response is written in complete sentences and contains two valid reasons for saving the rainforest.
“People must save the rainforest to save the animals’ homes. People need to save the rainforest because we get ingredients for many medicines from there.”
1 point / The student’s response contains only one reason.
“People should save the rainforest because it is important and because people and animals need it.”
0 points / There is no response, or the response is completely incorrect or irrelevant.
3.1.2 ECR Analytic Rubric-Art (Example)
Dimension / Advanced (4 pts.) / Proficient (3 pts.) / Basic (2 pts.) / Below Basic (1 pt.)Art Elements and Principles
- Sketch a Face / The drawing of the face shows a breakdown of the proportions of the facial features with a lot of attention paid to details, such as strands of hair, details in eyes. The facial expression establishes a mood. / The drawing of the face shows a breakdown of the face with facial features placed in proportion to one another. The portrait lacks facial expression, thus no mood is shown in the drawing. / The drawing of the face shows some breakdown of the face but following through with other visual cues, such as eyebrows lining up with tops of the ears, are not evident. No attention is paid to facial expression to establish mood. / The drawing of the face is not broken down into proportional segments, so the placement of the facial features is random and abstracts the realism of the portrait. No attention is paid to details, such as strands of hair, eyebrows, eyelashes, etc.
Art Elements and Principles
- Color Scheme / Background collage uses either a monochromatic or analogous color scheme with all ends of the pictures glued down. The arrangement of the collage is unique using either space of transition of color in a planned way. The color mixing and transition of the paint in the portrait shows a broad range of tones, and the application contours the shape of the face. / Background collage uses either a monochromatic or analogous color scheme with some ends of the pictures sticking up or small portions of the oak tag showing through. The color mixing with the paint in the portrait shows a range of either a monochromatic or analogous color scheme, but no awareness of contour color application. / Background collage has at least 60% of the paper fitting into either a monochromatic or analogous color scheme. Color mixing in the portrait is blotchy and not blended well; each color is separate with no attempt to transition one color to the next. / Background collage and color mixing show no identification to either a monochromatic or an analogous color scheme.
Art Elements and Principles
Point of View
- Original Work / Portrait and background show a full understanding of monochromatic and analogous color schemes. Paint mixing is gradual using a range of hues that have more than eight tones, and the tones are applied enhancing the contour and roundness of the face. / Portrait and background show understanding of monochromatic and analogous color schemes. Some edges of the collage are not glued down. The color portrait is painted with a range of six or more hues of a monochromatic or analogous color scheme. The transitions of color are smooth and gradual. / Portrait and background show some understanding of monochromatic and analogous color schemes with only one or two colors that do not fit into the selected schemes. Half of the color mixing with paint shows gradual transitions of color. / Portrait and background show no knowledge of monochromatic or analogous color schemes. There is no range of color mixing within either color scheme with no gradual transitions of color.
ECR Holistic Rubric-TDA (Example)
Item #_____ Sample Response for: ______4 points /
- Student addresses the prompt accurately and completely and gives a central claim/argument
- Response has a clear beginning, middle, and end that all support and articulate the student’s original claim
- Student supports claim with relevant evidence from the text
- Student uses clear language that is appropriate for the task, and demonstrates proper use of grammar and conventions throughout response
3 points / Student response is missing one of the following:
- A complete claim that addresses the prompt accurately
- A clear beginning, middle and end that are in line with the central claim
- Relevant use of textual evidence to support the central claim
- Correct and appropriate use of language and conventions throughout response
2 points / Student response is missing two of the following:
- A complete claim that addresses the prompt accurately
- A clear beginning, middle and end that are in line with the central claim
- Relevant use of textual evidence to support the central claim
- Correct and appropriate use of language and conventions throughout response
1 point / Student response is missing three of the following criteria:
- A complete claim that addresses the prompt accurately
- A clear beginning, middle and end that are in line with the central claim
- Relevant use of textual evidence to support the central claim
- Correct and appropriate use of language and conventions throughout response
0 points / Student does not display any effort that is relevant to the task
ECR Analytic Rubric-TDA (Example)
Item #1 Sample Response for: In the letter, Adams informs her daughter about the experience of moving to a new city and living in a new home. Write an essay analyzing how Adams responds to her new surroundings. Use evidence from the letter to support your response.Dimension / Advanced
(4 points) / Proficient
(3 points) / Basic
(2 points) / Below Basic
(1 points)
Response to Prompt
(4 points) / Central claim is clearly articulated and completely addresses the writing prompt / Central claim is clearly articulated and mostly addresses the writing prompt / Central claim is ambiguous/unclear and partially addresses the writing prompt / An effort was made to create a central claim, but it does not address the writing prompt
Textual Evidence
(4 points) / Student uses sufficient (i.e., at least one instance per body paragraph) and relevant textual evidence to support the claim / Student uses relevant evidence to support the claim, but does not address all parts of the prompt with supporting textual evidence (i.e., one or more body paragraphs are missing textual evidence) / Student attempts to use sufficient (i.e., at least one instance per body paragraph) evidence to support the claim, but the evidence presented is irrelevant to the claim / Student attempts to use insufficient (i.e., one or more body paragraphs are missing textual evidence) evidence to support the claim, and the evidence presented is irrelevant to the claim
Structure
(4 points) / Response contains a clear beginning, middle, and end, and uses transitions to ensure these parts flow together and stay on-topic; textual evidence is integrated smoothly and its relevance is justified in the response / Student uses relevant evidence to support the claim, but does not address all parts of the prompt with supporting textual evidence (i.e., one or more body paragraphs are missing textual evidence) / Student attempts to use sufficient (i.e., at least one instance per body paragraph) evidence to support the claim, but the evidence presented is irrelevant to the claim / Student attempts to use insufficient (i.e., one or more body paragraphs are missing textual evidence) evidence to support the claim, and the evidence presented is irrelevant to the claim
Conventions
(4 points) / Student’s language is clear and appropriate for the task; there are up to two minor errors in spelling, grammar, and/or usage / Student’s language is clear and appropriate for the task; there is a major error andother minor errors in spelling, grammar, and/or usage / Student’s language is either clear OR appropriate for the task; there are up to three major errors in spelling, grammar, and/or usage / Student’s language is neither clear nor appropriate; there are up to five major errors in spelling, grammar, and/or usage
3.1.2 Workflow: Develop an Assessment Scoring Rubric(SCR/ECR/PT items/tasks)
3.1.2 Procedural Steps: Develop Assessment Scoring Rubrics (SCR/ECR/PT items/tasks)
A scoring rubric is a tool that is used to measure and evaluate students’ performance of a task.
- Review the Item Framework and the criteria articulated in the item’s stem/directions.
- Select a rubric structure based upon scoring criteria and the number of dimensions being measured.
- Holistic/Single-dimension rubrics
- Analytic/Multi-dimension rubrics
- Modify the rubric language, using the specific criteria expected in the response needed to award the maximum number of points. Ensure that the criteria/guidelines articulated in the rubric are clear enough for students and other teachers to understand.
- Determine how much the response can deviate from “fully correct” in order to earn the next (lower) point value. [Continue until the full range of possible scores is described.] For an analytic rubric, this step must be completed for each dimension/criterion being measured.
- During the quality assurance review, ensure the response expectation, scoring rubric, and test specifications are fully aligned.
3.1.2 Scoring Rubric: QA Checklist
Task / Task Question
Targeted Content Standards / Does the rubric reflect a performance continuum?
Developmentally Appropriate / Is the rubric clear and concise?
Aligned to Task / Does the rubric provide all dimensions (components) of the task?
Criteria / Does the rubric include expectations for a “fully correct” response?
Potential Bias / Does the rubric omit non-cognitive attributes (e.g., motivation, timeliness)?
Editing / Have editorial correctness and Universal Design principles been applied?
HANDOUT #3.2
Module 3.2: Develop Scoring Guides
An assessment’s scoring guide provides the scorer with clear guidelines on how to score items.
3.2.1 Workflow: Develop a Scoring Guide for Selected Response (SR) Stand-Alone Items
3.2.2 Workflow: Develop a Scoring Guide for Selected Response (SR) Passage-Based Items
3.2.13Workflow: Develop a Scoring Guide for Selected Response (SR) Evidence-Based Items
3.2.1 Procedural Steps: Develop a Scoring Guide for Selected Response (SR) Stand-Alone Items
3.2.2 Procedural Steps: Develop a Scoring Guide for Selected Response (SR) Passage-Based Items
3.2.3 Procedural Steps: Develop a Scoring Guide for Selected Response (SR) Evidence-Based Items
1. Review the Item Framework and apply the Quality Control checklist
2. Provide correct answers for each item
3. Check the scoring key to ensure it matches the answers provided in the Item Task Framework.
4. Give further instructions for calculating final scores of items (e.g., multiply the raw score by 2 for the final score) when applicable. If there are special conditions for scoring or awarding point values (e.g., when to award partial credit/half of a point for a certain performance criteria), ensure that they are included by the key and are easy for the scorer to see and comprehend.
5. Review the item and answer options for accuracy and grammatical soundness.
3.2.4 Workflow: Developing Guidelines for Scoring Rubrics for Short Constructed Response (SCR) Stand-Alone Items
3.2.5 Workflow: Developing Guidelines for Scoring Rubrics for Short Constructed Response (SCR) Passage BasedItems
3.2.6 Workflow: Developing Guidelines for Scoring Rubrics for Extended Constructed Response(ECR) Stand-AloneItems
3.2.7 Workflow: Developing Guidelines for Scoring Rubrics for Extended Constructed Response(ECR) Text Dependent Analysis Items
3.2.8 Workflow: Developing Guidelines for Scoring Rubrics for Performance Task (PT) Multi-Day Task Items
3.2.4 Scoring Guide Procedural Guidelines: Scoring Rubrics for Short Constructed Response (SCR) Stand-Alone Items
3.2.5 Scoring Guide Procedural Guidelines: Developing Guidelines for Scoring Rubrics for Short Constructed Response (SCR) Passage BasedItems
3.2.6 Scoring Guide Procedural Guidelines: Developing Guidelines for Scoring Rubrics for Extended Constructed Response (ECR) Stand-AloneItems
3.2.7 Scoring Guide Procedural Guidelines: Developing Guidelines for Scoring Rubrics for Extended Constructed Response (ECR) Text Dependent Analysis Items
3.2.8 Scoring Guide Procedural Guidelines: Developing Guidelines for Scoring Rubrics for Performance Task (PT) Multi-Day Task Items
1. Review the Item, TDA or Performance Task Framework and apply the Quality Control checklist.
2. a. Develop a clear statement that articulates specific criteria for the test-taker to provide.
b. Develop a clear statement for each subordinate task that articulates specific criteria for the test-taker to provide.
c. Develop a clear statement for the final product and for each subordinate task that articulates specific criteria for the test-taker to provide.
d. Select and develop an appropriate rubric type.
SCR Stand-Alone / SCR HolisticSCR Passage Based / SCR Holistic
ECR Stand-Alone / ECR Holistic, ECR Analytic
ECR Text Dependent Analysis / ECR Analytic TDA
Performance Task / ECR Analytic)
3.Give clear, detailed instructions about how the item is to be scored through use of the rubric.
Include sample responses/exemplars.
4.Give further instructions for calculating final scores of items (e.g., multiply the raw score by 2 for the final score) when applicable. If there are special conditions for scoring or awarding point values (e.g., when to award partial credit/half of a point for a certain performance criteria), ensure that they are included in the scoring instruction and are easy for the scorer to see and comprehend.
5. Review the item and answer options for grammatical soundness.
3.2.1 Science Grade 7 Selected Response (SR) Stand-Alone Item (Example)
1. Manuel tested the effects of temperature on the solubility of sugar in water. He measured the maximum amount of sucrose (in grams) he could dissolve in 100g of water. Manuel repeated the test at five different temperatures. The graph of his results is included below.Based on his observations, which of the following is a general hypothesis Manuel might form?
A. / The solubility of a solid increases as the temperature increases.
B. / The decreasing solubility of a solid raises the temperature of water
C. / The increasing solubility of a solid raises the temperature of water.
D. / The solubility of a solid decreases as the temperature increases.
3.2.1 Scoring Guide and Scoring Key
Assessment Name / Grade/Course / Administration / Total Possible PointsScience / Grade 7 / Post-test / 50
Item # / Item Tag / Item Type / Point Value / Answer
1 / 0001.SCI.GR7.POST.SR-LV2-S7.A.3.2.1 / SR / 1 / A
3.2.2 Reading Grade 6
3.2.2 Selected Response (SR) Passage-Based Item (Example)
Danger ZoneHotshots have one of the most dangerous jobs in the world. At any second, a gust of wind can change the direction of a fire and block escape routes. In June 2013, tragedy struck when 19 hotshots died while fighting a wildfire in Yarnell, Arizona.
Despite the danger, Moore insists that he never gets scared on the job. He says his training makes him feel safe.
“We have a tremendous amount of respect for the force we’re dealing with,” he explains. “As long as we’re doing things the right way, fear isn’t a factor.”
--by Joe Bub
1. How does the author support the claim that hotshots have one of the most dangerous jobs in the world?
A. / The author titles the passage “Danger Zone”.
B. / The author cites an example that reflects the potential danger of being a hotshot.
C. / The author gives a specific date and time of a tragic situation involving hotshots.
D. / The author uses a quote from a hotshot in the article.
3.2.2 Scoring Guide and Scoring Key