District-Determined Measure Example
Mental Health Counselors and Social Workers, K-12
Content Area and Grade Range: Social/Emotional Skills, grades K-12
DDM Summary: This DDM assesses growth in grade K-12 students’ ability to identify social/emotional challenges and their feelings related to these challenges, and to identify, implement, and process appropriate coping strategies.
Developed by: Lawrie Donovan, Social Worker, Secondary (Rockland Public Schools); Deni Howley, Social Worker, Elementary (North River Collaborative); Freea Leahy, Social Worker, Secondary (Rockland Public Schools); Matt Morse, School Adjustment Counselor, Secondary (North River Collaborative)
Reviewed by: Sonya Meiran (ESE), Matt Hollaway (ESE), Craig Waterman (ESE)
Pilot Districts: Rockland Public Schools and North River Collaborative (Rockland, Whitman-Hanson, Hanover and Abington)
Date updated: June 2015
Table of Contents
Introduction2
Instrument5
Administration Protocol7
Scoring Guide10
Measuring Growth and Setting Parameters11
Piloting12
Assessment Blueprint14
Introduction
Description of the Measure
This DDM is a direct measure of students’ growth in identifying and describing social/emotional challenges and identifying, implementing and processing appropriate coping strategies. The measure is both an assessment and a teaching tool. It is based on students’ observable changes in the counseling or therapeutic session regarding their ability to identify and implement successful strategies to manage social/emotional challenges.
Target Audience
Many school counselors have a large caseload of over 50 students. The sample size for this DDM is a minimum of 5-10 students. It is recommended that a sample size of 10-20 be selected in order to improve validity of assessment. The sample will include students who have been referred for individual or small group counseling services related to social/emotional challenges and who work with the school counselor for a minimum of three months. Students should be scored at three months only, and not over the course of a full year. If a counselor has an established caseload at the beginning of the school year, he or she creates an alphabetical listing of the last names of his or her students who meet the stated criteria:
§ Referred by school personnel (rather than by parents); alternatively, high school students may identify themselves as being in need of services.
§ Receive individual or small group counseling (rather than in-class support).
§ Receive service for social/emotional challenges (rather than other needs).
§ Work with the school counselor for at least three months to ensure sufficient time for the counselor’s instruction and support to impact the student’s ability to identify challenges and implement coping strategies.
From this list, the school counselor selects the first 10 students from the list, starting with those names at the beginning of the alphabet. If the counselor has fewer than 10 students on this list, all are selected, with no fewer than five. If the counselor works with fewer than five students who meet the selection criteria, this DDM will not be appropriate without modification.
Typically, when assessing particularly small numbers of students, the amount of data collected needs to be greater to support valid conclusions about the students’ growth in the core content being measured. Therefore, this measure may be modified to increase the number of times the student’s progress is assessed. Alternatively, the user can introduce additional measures, such as an observation rubric, e.g., one that describes the extent to which the student applies social/emotional coping strategies in the classroom or broader school context, rather than solely in the school counselor’s therapeutic setting. The user is encouraged to review pages 6-7 of ESE’s Special Education Implementation Brief for further information about designing measures for small numbers of students.
Description of the DDM Development Process
This DDM was developed during October 2014 – June 2015 under a DDM Leadership Grant (FC-217) awarded to the North River Collaborative by the Massachusetts Department of Elementary and Secondary Education (ESE). In partnership with the Learning Innovations Program at WestEd (Woburn, MA), the Collaborative convened 12 Specialized Instructional Support Personnel (SISP), including PK-12 school guidance counselors, adjustment counselors, school psychologists, school social workers, and speech, language and hearing specialists who serve students in the North River Collaborative member districts. Participants worked in small teams to strengthen and apply their assessment literacy toward the development of four direct and indirect measures of student growth related to students’ social/emotional development.
Participants grew their expertise over seven sessions by engaging in a guided DDM development process framed by a series of questions, including:
1. What is most important to measure?
2. How shall we measure what’s most important?
3. How can we strengthen and refine our measure?
4. How can we prepare our measure for broader use?
5. What do we want to gain from the pilot?
6. What did we learn from the pilot?
Throughout, participants engaged in large group discussion and critique, as well as team collaboration and problem solving.
In addition to refinements made during these sessions, each measure was strengthened based on feedback from an ESE review team. Measures were then piloted from April-May 2015. Finally, the group analyzed the data collected during the pilot phase, which informed final revisions, as described in the closing pages of this document.
Next Steps
Districts in and beyond the Collaborative now have the opportunity to decide if they would like to implement or modify the attached assessment for use as a District-Determined Measure for school adjustment counselors, school psychologists, guidance counselors, and school social workers. Because this is a newly developed measure, it is important that districts engage school counselors and other SISP in examining results from the first year of implementation. Over time, staff should identify revisions or refinements that may strengthen the quality of the assessment, scoring tools, administration protocol, and/or student growth parameters to suit the circumstances and realities of each district’s local context.
Content Alignment
This measure is aligned to the following Core Course Objective (CCO): Students will identify and describe social/emotional challenges that they encounter and identify, implement, and process/reflect on relevant coping strategies.
A CCO is a statement that describes core, essential, or high priority content (knowledge, skills, or abilities), identified by those who designed the assessment. It is drawn, synthesized, or composed from a larger set of curriculum or professional standards. A primary focus of school counselors is to support the development of student attitudes, behaviors, knowledge, and skills that promote time on learning. Helping students learn the skills to manage social/emotional challenges will further that goal. In particular, school counselors often work with students who have been referred to services or are in crisis in part because they have not mastered these skills for coping with emotional stress. As a result, this DDM is aligned with health standards that represent a range of grade and developmental levels.
Most centrally, this CCO draws from Standard 5, Mental Health, from the MA Comprehensive Health Curriculum Framework, which states that students acquire knowledge about emotions, management of emotions, personality and character development, and social awareness; and will learn skills to promote self-acceptance, make decisions, and cope with stress. The relevant sub-standards are noted in the table below.
This standard is also aligned with the Massachusetts Guidelines on Implementing Social Emotional Learning Curricula, the MA Model for Comprehensive School Counseling Programs, and The National Association of Social Workers. In addition, this DDM aligns well with the professional responsibilities of school guidance counselors, as described by Specialized Instructional Support Personnel indicators I-A-1 and II-B-2, school social workers as described in NASW Standard 3, and school psychologists as described in MSPA standard II-D-2. This final professional standard reflects the essence of this DDM, which centers on facilitating the design and delivery of interventions that help students develop skills necessary to become self-regulated, self-motivated, and active learners.
Content (Standard) / WeightStudents identify social/emotional challenges.
5.1 (By the end of grade 5) Identify the various feelings that most people experience and describe the physical and emotional reactions of the body to intense positive and negative feelings
5.7 (By the end of grade 8) Identify and describe the experience of different feelings (such as elation, joy, grief, and rage) and how feelings affect daily functioning (MA Comprehensive Health Curriculum Framework) / 25% of the measure
Students identify a range of coping strategies for a given social/emotional challenge.
5.16 (By the end of grade 8) Describe the signs of destructive behavior, and identify intervention strategies and kinds of professional intervention
5.11 (By the end of grade 12) Analyze healthy ways to express emotions and to cope with feelings, including the common causes of stress, its effects on the body, and managing stress
5.19 (By the end of grade 12) Explain positive techniques for handling difficult decisions (MA Comprehensive Health Curriculum Framework) / 25% of the measure
Students implement a range of coping strategies.
5.2 (By the end of grade 5) Apply methods to accommodate a variety of feelings in a constructive manner in order to promote well being.
5.11 (By the end of grade 12) Analyze healthy ways to express emotions and to cope with feelings, including the common causes of stress, its effects on the body, and managing stress. (MA Comprehensive Health Curriculum Framework) / 25% of the measure
Students will process/reflect on coping strategies
5.6 (By the end of grade 5) Explain how coping skills (such as perceiving situations as opportunities and taking action/exerting control where possible) positively influence self-concept
5.11 (By the end of grade 12) Analyze healthy ways to express emotions and to cope with feelings, including the common causes of stress, its effects on the body, and managing stress (MA Comprehensive Health Curriculum Framework) / 25% of the measure
100%
Instrument
There are two instruments for use in this assessment. The first – the Assessment Question Sheet – is the tool the counselor uses to pose questions to the student in a one-to-one therapeutic setting. In the first column, a series of 10 questions are listed for the counselor to pose to the student. In the second column, simplified versions of these questions are provided for use with English language learners or students with language-based disabilities. The third column provides examples of student responses to each of the 10 questions. The fourth and final column notes the social/emotional skill category for each of the 10 questions; it also notes acceptable expanded response options that the counselor is expected to provide to all students to ensure non-language-based ways in which students may process and express responses to the counselor’s questions. (See Administration Protocol for further information.)
The second instrument – the Mental Health Rubric – is the tool the counselor uses to evaluate students’ responses to the 10 assessment prompts. The first column of the rubric describes the social/emotional skill and the subsequent columns describe a continuum of performance levels with associated scores.
Levels of response include:
§ NO EVIDENCE OF USE, meaning the student provides no evidence of the knowledge or skill requested.
§ INCONSISTENT EVIDENCE OF USE, meaning the student provides some initial evidence of the knowledge or skill requested, but it appears emergent or partial and does not remain stable throughout the assessment.
§ SOMEWHAT CONSISTENT EVIDENCE OF USE, meaning the student provides more predictable, reliable, or stable evidence of the knowledge or skill requested. It is not yet fluent or complete enough, however, to suggest the knowledge and skills are really secure and learned.
§ CONSISTENT EVIDENCE OF USE, meaning the student provides a readily available and complete response that demonstrates predictable, stable, and reliable evidence of the knowledge or skill such that it appears that the student has solid command of the knowledge or skill.
To clarify, the developers provide the following example:
· If the student is unable to describe or identify a situation occurring during the school day that is upsetting or makes it difficult for the student to stay focused on their work, then the school counselor indicates NO EVIDENCE OF USE.
· If the student is able describe or identify only one (we are asking for three examples) situation occurring during the school day that is upsetting or makes it difficult for the student to stay focused on their work, then the school counselor will indicate INCONSISTENT EVIDENCE OF USE.
· If a student is able to describe or identify only two (we are asking for three examples) situations occurring during the school day that are upsetting or make it difficult for student to stay focused on their work, then the school counselor indicates SOMEWHAT CONSISTENT EVIDENCE OF USE.
· If the student is able to describe or identify three or more situations occurring during the school day that is upsetting or makes it difficult for student to stay focused on their work, then the school counselor indicates CONSISTENT EVIDENCE OF USE.
Administration Protocol
The Administration Protocol addresses how the measure is intended to be implemented to best support a common conversation about student growth across classrooms.
When is the measure administered?
This is a repeated measure; the same assessment will be used to measure students’ growth over several administrations during the three-month period. Baseline data will be collected at the student’s first counseling session and then once during the final week of each month for a minimum of three months of counseling services. If, for any reason, the counselor does not meet with the student during the final week of a month, the assessment should be administered during the first week of the subsequent month. Students who do not meet with the school counselor for a minimum of three months should not be included in the final scoring of this DDM. Growth parameters are based upon a three-month period of time. Students who are seen for longer than three months may not be able to have growth assessed using the parameters as they are currently configured.
How is the measure administered?
The school counselor prepares copies of the Assessment Question Sheet and the Mental Health Rubric prior to meeting with the student. The counselor may also provide additional response option supports due to student age and development or recommended through consultation with ELL or SPED providers for students who may need accommodations