District-Determined Measure Example
Social-Emotional Learning Skills for SISP Educators
Content Area and Grade Range: Social-Emotional Learning Skills, K-2
DDM Summary: This DDM is a direct measure of K-grade 2 students’ learning of social and emotional skills; specifically, their ability to identify and name a range of emotions and to explain how others might be feeling in a given set of situations. These social/emotional skills are essential for building positive relationships with peers.
Developed by: Gerri Quigley, School Adjustment Counselor (East Bridgewater); Robin Harris, Speech-Language Pathologist (Avon)
Reviewed by: Sonya Meiran (ESE), Matt Hollaway (ESE), and Craig Waterman (ESE)
Pilot Districts: East Bridgewater Public Schools, Avon Public Schools
Date updated: June 2015
Table of Contents
Introduction2
Instrument4
Administration Protocol5
Scoring Guide8
Measuring Growth and Setting Parameters9
Piloting10
Assessment Blueprint12
Introduction
Description & Rationale
This DDM is a direct measure of Grades K–2 students’ learning of social and emotional skills. By SISP, we mean school counselors, social workers, speech pathologists, school psychologists, and related staff who work directly with students to develop their social and emotional skills. Specifically, this DDM is designed to measure growth in social awareness and interpersonal skills for maintaining positive relationships with peers. These skills are essential in school and life and require the ability to recognize the thoughts, feelings, and perspectives of others, including those different from one’s own. This measure, which consists of a set of pictured “feeling” and “scenario” cards and an Observation/Scoring Rubric, is based on professional observation of a student’s ability to identify and explain particular social-emotional skills in a one-to-one therapeutic setting. Social-emotional skills assessed by this DDM fall within the following two categories: (1) labeling pictured feelings, and (2) explaining the possible feelings and perspectives of others based on pictured scenarios (empathy).
Target Audience
Students participating in this assessment are those who have been identified by teachers, special education staff, and parents, or via the referral process of a Child Study Team as presenting with a weakness in social/emotional learning and/or language. The student must be on an IEP or 504 Accommodation plan, or being serviced by a school counselor or psychologist for the purpose of developing social emotional skills. This is necessary if this DDM is to reflect the effectiveness of the treatment being provided by the SISP. The minimum number of students for this DDM should be five in order to collect enough data to demonstrate growth in the area of social-emotional learning.
The number of students should be limited to a maximum of 15 in order to effectively monitor and measure social-emotional growth throughout the year. If there are more than 15 students, other district SISPs who work directly with students on developing social-emotional learning may implement this DDM with the additional students. Since this measure looks at growth over the course of the school year, then students should be identified in September or very early in the school year. Students may be selected by first separating into grade levels if the SISP works with multiple grades, then choosing the first, third, fifth, etc. students from each grade until 15 have been chosen.
Alignment
This measure is aligned to the following Core Course Objective (CCO): Students demonstrate emotional regulation and expected verbal and non-verbal social/emotional communication skills in order to function in the school environment.
A CCO is a statement that describes core, essential, or high priority content (knowledge, skills, or abilities), identified by those who designed the assessment, which is drawn, synthesized, or composed from a larger set of curriculum or professional standards. This CCO draws from Standard 5, Mental Health, from the MA Comprehensive Health Curriculum Framework, which states that by the end of grade 5, students will have acquired knowledge about emotions, management of emotions, personality and character development, and social awareness. They will also have learned skills to promote self-acceptance, make decisions, and cope with stress. This is a grade-span standard, meaning that it applies to students in K to grade 5. This content was selected for this DDM because elementary SISP staff often work directly with students who have been identified for services—they already show difficulty with the skills described in this standard and require sustained support for the development of appropriate social/emotional skills in the school context.
This CCO and associated standards are also reflected in the Massachusetts Guidelines on Implementing Social and Emotional Learning Curricula. In addition, these standards are aligned with the professional responsibilities of school guidance counselors, as described by Specialized Instructional Support Personnel indicators I-A-1 and II-B-2.
Content (Standard) / WeightStudents label pictured feelings.
Mental Health Standard 5.1: By the end of grade 5, students Identify the various feelings that most people experience and describe the physical and emotional reactions of the body to intensive positive and negative feelings. (MA Comprehensive Health Curriculum Framework) / 40% of the measure
Students label the pictured feelings and perspective of others (empathy) and provide a plausible explanation.
Mental Health Standard 5.1: By the end of grade 5, students Identify the various feelings that most people experience and describe the physical and emotional reactions of the body to intensive positive and negative feelings. (MA Comprehensive Health Curriculum Framework) / 60% of the measure
100%
Description of the DDM Development Process
This DDM was developed during October 2014 – June 2015 under a DDM Leadership Grant (FC-217) awarded to the North River Collaborative by the Massachusetts Department of Elementary and Secondary Education (ESE). In partnership with the Learning Innovations Program at WestEd (Woburn, MA), the Collaborative convened 12 staff from the Collaborative’s member districts, including PK-12 school guidance counselors, adjustment counselors, school psychologists, and school social workers, as well as speech, language and hearing specialists. Participants worked in small teams to strengthen and apply their assessment literacy toward the development of four direct and indirect measures of student growth related to students’ social/emotional development.
Participants grew their expertise over seven sessions by engaging in a guided DDM development process framed by a series of questions, including:
1. What is most important to measure?
2. How shall we measure what’s most important?
3. How can we strengthen and refine our measure?
4. How can we prepare our measure for broader use?
5. What do we want to gain from the pilot?
6. What did we learn from the pilot?
Throughout, participants engaged in large group discussion and critique, as well as team collaboration and problem solving. In addition to refinements made during these sessions, each measure was also strengthened based on feedback from an ESE review team. Measures were then piloted from April-June 2015. Finally, the group analyzed data collected during the pilot phase, which informed final revisions, as described in the closing pages of this document.
Next Steps
Districts in and beyond the Collaborative now have the opportunity to decide if they would like to implement or modify the attached assessment for use as a DDM for Specialized Instructional Support Personnel (SISP). Because this is a newly developed measure, it is important that districts engage SISP, including school counselors, social workers, speech-language pathologists, and school psychologists, as well as related staff, in examining results from the first year of implementation. Over time, staff should identify any revisions or refinements that may further strengthen the quality of the assessment, scoring tools, administration protocol, and/or growth parameters to suit the circumstances and realities of the district’s local context.
Instrument
The Instrument consists of a one-to-one student assessment and scoring rubric where the SISP evaluates students’ responses in two areas – labeling pictures and labeling the feelings and perspectives of others. To accomplish this, the counselor uses a pre-determined set of feeling and situation cards, as described below.
Sources for Feeling and Situation Cards
Feeling and Situation Cards are available from the following sources:
· Committee For Children, SECOND STEP, K–2 Kits (www.cfchildren.org/second-step.aspx)
May be ordered through Amazon.com
· Webber Photo Cards “Emotions” by Super Duper Publication
· Photo Feelings by Fun Deck Cards
· Photographic Learning Cards Facial Expression by Motivation USA
· Carson Dellosa Key Education Facial Expression Learning Cards by Carson Dellosa
· Language Cards “Photos & Words for Developing Oral & Written” Skills by North Star
After purchase, districts must identify the same set of eight Feeling and Situation Cards that align with the descriptors provided in this DDM for use by all district SISP. Alternatively, a district could opt to identify eight different emotions and modify the Observation/Scoring Rubric to align with the preferred set of cards.
Set 1: Happy, sad, surprised, scared, proud, silly, angry, disgusted
Set 2: Eight additional situation cards showing nonverbal clues of a person’s feelings with these same emotions
Interpreting the Feeling and Situation Cards
The terms defined/identified by the product developer or manufacturer should be used in determining correct responses. In instances where the developer has not identified correct or allowable responses, the SISP team should determine correct/allowable responses prior to administration; all SISP using the assessment must then adhere to the agreed upon correct responses. In each pictured scenario, the team must identify the person who displays the targeted feeling before administering the DDM. They must also point to that person when they administer that portion of the DDM to the student.
It is strongly recommended that the SISP team co-construct a chart showing the picture cards and acceptable responses for each card as a guide to strengthen the consistency of scoring across the team. For example, the team may decide that student responses that are close synonyms for the pictured emotion—such as “mad” for “angry”—are scored as correct. This chart should be modified following the first assessments to reflect the kinds of responses students actually provided and to increase consistency in how SISP professionals score students’ varied responses.
The Observation/Scoring Rubric
The student’s name, grade, and the observation date should be noted at the top of the provided Observation/Scoring Rubric. This rubric describes two social-emotional skill categories, listed down the first, left-hand column of the chart:
I. Identifying/Labeling Pictured Feelings
II. Labeling the Feelings and Perspective of Others from pictured scenario cards
The top row of the rubric describes several performance levels based on the number of correct responses provided by the student. A point value is noted in each cell of the rubric. These scores align with the weighting represented in the Assessment Blueprint. During the assessment, the counselor records the student’s responses, and then circles the point value that is associated with the student’s observed level of performance for each skill within a targeted skill category.
Administration Protocol
The Administration Protocol addresses how the measure is intended to be implemented to best support a common conversation about student growth across classrooms.
When is the measure administered?
Administration occurs during three intervals in the school year: October, January, and May. During each observation period, the SISP professional conducts the assessments in a one-to-one therapeutic setting. The same SISP must administer the assessments throughout the school year. If there is a change in staff and the district wishes to allow the new SISP to use data collected from a prior administrator for use as a baseline, the data for the impact of this SISP on growth would need to be examined with caution. Because this is a repeated measure, the same assessment, materials, and procedures are used each time the measure is administered.
The SISP may administer Sections I and II in two separate sessions if necessary due to time constraints. Each of these sessions, however, must take place no more than one week apart. Each assessment will require a 20-30 minute session. If administered in two sessions, all of Section I should be administered in the first session and all of Section II in the second session.
How is the measure administered?
SISP professionals prepare for observations and subsequent scoring by doing the following:
1) Review the Observation/Scoring Rubric and related administration directions.
2) Prepare copies of the Observation/Scoring Rubric for each student, noting each student’s name and grade at the top of each sheet.
3) Write the correct response on the Observation Scoring Rubric. Prompts should also be included on the scoring rubric to ensure consistency in DDM administration.
4) SISP professionals are strongly encouraged to coordinate calibration sessions with colleagues, as described in the Scoring Guidance section, below.
The Observation Rubric includes two categories: (I) Labeling Pictured Feelings, and (II) Labeling the Feelings and Perspectives of Others (empathy).
Section I: Labeling Pictured Feelings
This section involves the use of the first set of cards picturing people’s feelings. The same picture cards are used throughout the year. (See above for a source listing for these cards.) The SISP professional shows the pictures and says to the student each time, “Tell me how this person feels.” No additional cueing, suggestive voice inflection, or words of enticement or coaching are permitted. If the student does not respond or responds, “I don’t know,” the prompt may be repeated once. If the student still does not respond or responds, “I don’t know” again, the score is 0 (zero).
The SISP professional attends to the student’s ability to recognize and name the pictured emotions and records the student’s response for each card in the space provided. After the eighth card, the SISP circles the point value associated with the demonstrated level of the student’s performance on the provided Observation/Scoring Rubric.
Section II: Labeling the Feelings and Perspectives of Others (empathy)
The SISP professional then shows the second set of eight situational picture cards and prompts the student by saying, “Tell me how this person feels” and “Why does he/she feel that way?” For example, when showing a picture of boy crying at a lunch table, a student might respond, “He feels sad because he left his lunch at home,” or “The boy feels sad because the teacher told him he has to eat something he doesn’t like.” Both responses correctly identify the feeling and provide a plausible explanation that links back to the feeling. A response such as, “He is mad. He is sitting all alone,” does not identify the correct feeling and does not clearly link the explanation back to the feeling, i.e., why would sitting alone make someone feel mad?