MASSACHUSETTS DEPARTMENT OF EDUCATION

MCAS 2007

MCAS Grade 3 Mathematics Standard Setting

(August 15-16, 2007)

Standard Setting Evaluation Form

Please check the most appropriate category or fill in the blank for each of the following statements:

1. I am a 10 Classroom Teacher

4 K-12 Education Administrator

1 University-level Educator

2 Business and/or Community Representative

3 Other

2. I am 4 Male

16 Female

3. Please circle the letter that best describes the extent to which your ratings were based on student responses to multiple-choice and open-response questions. (CIRCLE ONLY ONE)

A.  0 - Overall I relied primarily on open-response questions to determine my ratings.

B.  20 - Overall, I relied equally on open-response and multiple-choice questions to determine my ratings.

C.  0 - Overall, I relied primarily on multiple-choice questions to determine my ratings.

Standard Setting Evaluation Form
DIRECTIONS: Please circle the number indicating the degree to which you agree or disagree with each of the following statements:
1-Strongly Disagree 2-Disagree 3-Agree 4-Strongly Agree
Statements / Strongly Disagree / Disagree / Agree / Strongly Agree
4. The overall environment and accommodations were comfortable and appropriate for standard-setting activities. / 0 / 1 / 5 / 14
5. The background information provided on Wednesday regarding the Curriculum Framework, MCAS Grade 3 Mathematics exam, and the purpose of standard setting improved my ability to set standards. / 0 / 0 / 6 / 14
6. Taking and discussing the MCAS Grade 3 Mathematics exam during my orientation helped me understand the purpose and process of the MCAS Grade 3 Mathematics standard setting. / 0 / 0 / 3 / 17
7. By the end of the calibration training (ranking, discussing, and classifying sets of student work), I could distinguish among MCAS Grade 3 Mathematics Performance Level Descriptors. / 0 / 0 / 6 / 14
8. Overall, I was provided with clear instructions for my standard-setting assignments. / 0 / 0 / 1 / 19
9. The group discussions that took place after the first round of ratings improved my ability to set standards. / 0 / 0 / 4 / 16
10. I am confident that the ratings I provided were consistent with the MCAS Grade 3 Mathematics Performance Level Descriptors. / 0 / 0 / 7 / 13
11. The MCAS Grade 3 Mathematics standard-setting process provided for a reliable classification of student work. / 0 / 0 / 6 / 14
12. The facilitator was effective. / 0 / 0 / 3 / 17
Standard Setting Evaluation Form
DIRECTIONS: Please circle the number indicating your perceptions as to the time allotted for each of the segments of standard setting:
1-Far too short 2-Too short 3-Approximately right 4-Too long 5-Far too long
Segments of Standard Setting / Far too short / Too short / Approximately right / Too long / Far too long
13. Initial background information provided on Wednesday morning. / 0 / 0 / 17 / 3 / 0
14. Taking and discussing the MCAS Grade 3 Mathematics exam. / 0 / 0 / 19 / 1 / 0
15. Learning about and discussing Performance Level Descriptors. / 0 / 2 / 14 / 4 / 0
16. Ranking, discussing, and classifying students work (calibration). / 0 / 0 / 18 / 1 / 1
17. Initial individual classification of student work. / 0 / 0 / 19 / 0 / 1
18. Group discussion regarding initial ratings. / 0 / 0 / 16 / 4 / 0
19. Rating student work for the second time. / 0 / 0 / 18 / 2 / 0
20. Final rating of student work. / 0 / 0 / 19 / 1 / 0

Please provide additional comments on the back of this page.

Thank you for being a part the MCAS grade 3 Mathematics standard-setting panel.

Comments for Mathematics

· 

o Facilities – great

o Staff – great

o This experience was extremely helpful as an educator! I loved being part of this panel!

o Facilitator Sally was exceptional!

·  Overall, it went well. I always take home new ideas in regards to how I teach math skills and content as a result of the insight of my peers on the committee.

· 

o (On question 3) I do not like the word primarily. Multiple choice were worth more – so I relied on them more.

o The facilitator was fabulous and did and amazing job of keeping the process moving

·  The process was fair. I am confident in the cut score recommended by this panel. A great job by DOE/Measured Progress!

· 

o Sheraton Facility was top notch – one of the best.

o Further explanation of “equating” would be nice, this committee had very strong “new” members who should be tapped for future/other committees.

o Two very, very full days – can we shorten first morning to allow more work time? Both days ran a bit after 4 and I think we have rushed a bit.

· 

o Sally (she is fabulous) did an excellent job of facilitating. It may be helpful to establish group norms/rules before starting discussion. Some members of the panel seemed to dominate the conversation while others rarely, if at all, spoke.

o Excellent food and snacks! Thank you!

4