Notes From: MCAS 2.0 Accessibility Work Group
January 28, 2016 – 2:00-5:00 p.m.
Dan Wiener, Administrator of Inclusive Assessment
Robert Pelychaty, Accommodations and Portfolio Appeals Coordinator
Participants (in person)
Suzanne Recane / Carrie Ingrassia / Sean McAdamLatoya Gayle / Amy Foley / Kathleen Buchanan
Kate Gearon / Leslie Trotta / Kelly-Ann Cooney
Marcia Mittnacht / Mardi Loeterman / Lauren Scorpio
Deb Hand / Leslie Sullivan
Participants (via phone conference)
David Thompson / Scott Holcomb / Kathy ElichSarah Kaye / Marsha Olsen / Charitra McCarty Byrd
Amy Winston
Introductions, roles, and responsibilities
The facilitator (Dan W.) described the goals and tasks the group needs to accomplish. He focused the group on the agenda and informed the group of future meeting dates and times.
Our goal is to obtain information from participants based on their experience with implementing accommodations for MCAS and PARCC tests in order to inform the Request for Responses/Proposals being developed now for the next five-year MCAS contract; and also to learn from the lessons of PARCC and MCAS in order to develop the best possible accommodations and accessibility policies for Massachusetts students. This group will not make policy, but will make recommendations in order for the Department to develop policies.
Current accommodations policy
Testing policies have evolved in recent years, due to new online tests based on the Common Core, and based on expanding ideas about accessibility and universal design for learning. MCAS policies have essentially remained as they were in 2001, when substantial revisions were last made. We’ve learned much from our experience with MCAS and our involvement with PARCC. Tests are increasingly available online which has led to development of more accessibility tools and supports available to all students. As a result of the increase in accessibility features and testing procedures available to all students on PARCC, the list of PARCC accommodations for specific student subgroups (SWD and ELLs) has grown smaller. Accessibility features on paper-based tests (PBT) are designed to mirror the tools and features generally available on computer-based tests (CBT), although differences will continue exist between the formats (e.g., writing tools and screen enlargement do not exist on the paper tests).
The group reviewed the list of PARCC accessibility features for all students, and test administration considerations determined by the principal; e.g., small group or on-to-one administration, separate setting, or specified area of a room.
The group reviewed the PARCC Accommodations and Accessibility Overview for Massachusetts Educators that compares PARCC and MCAS accommodations.
The process of developing the PARCC accessibility and accommodation policies with a consortium of states allowed for an overall broadening of eligibility for accommodations, and a sharing of ideas and perspectives among member states; but no state got exactly what it wanted in the process of reaching consensus. States agreed to adopt the policies adopted by the majority of PARCC member states.
MA educators’ experience administering the PARCC computer-based tests and providing accommodations
Comments:
o Students missed their reference sheets.
o Students did surprisingly well taking the PARCC tests on the computer.
o Students practiced with the CBT and their testing experience was better than expected.
o Students had to be familiar with keyboarding and be able to type sufficiently well to produce a written response.
o Students struggled with typing and being able to complete the session on time.
o Students need to learn CBT test-taking strategies.
o Using the note taking feature for the PARCC CBT was challenging.
o 3rd graders had a difficult time with the CBT. Young students fatigued quickly; strong eye-hand coordination is needed for the CBT.
o Students did not fatigue as much taking MCAS or PARCC PBT.
o As we think about CBT accommodations, we need to include accommodations or features that are readily available on all computers, such as automatic spell correct.
o A personalized approach to accommodations is very important.
o CBT and PBT were not comparable because there was no option to view video on PBT tests.
o Some test administrators did not understand the differences between accommodation and accessibility features, and were not certain how to provide accessibility features and to whom.
o Should all students have accessibility features available? Were some students shortchanged by not having certain features available? It could be a disadvantage for students who were not trained to use the accessibility features beforehand.
o Accessibility features should not be eliminated just because students were not prepared adequately to use them. Students should have received instruction on the use of accessibility features.
o Some students had prior experience with online assessments and seemed to be quite facile with using PARCC CBT. Routine classroom testing on computers was helpful for those students.
Summary
The work group discussed their PARCC and MCAS experiences and the differences between administering both tests, and specifically the unique experience of administering a CBT. In general, the work group found that PARCC and MCAS were accessible to students with disabilities, but suggested the need for more training and practice with CBT as well as the need to refine some of the current accommodation plan to develop a better set of policies.
Discussion Topics
The group reviewed the discussion questions prepared in advance and determined which questions are important to address at today’s session because recommendations were need immediately to inform development of the new MCAS RFP. The questions were read aloud to the group. (See attachment)
Comments
o Should the work group discuss how the accommodation policies align to MCAS 2.0 STE tests?
o The group should consider expanding consideration of the read-aloud for STE tests.
o It was a challenge to compare PARCC and MCAS accommodations as the numbering systems were different. The new test should correspond with old accommodations numbers for ease of identifying each accommodation or test feature. (Note: the Overview for MA Educators provided this crosswalk between MCAS and PARCC accommodations/accessibility features)
o The read aloud/ text-to-speech (TTS) voice should sound human, less mechanical.
o Students should be familiar with the voice of the CBT text-to-speech voice(s) and have ample time to practice using the system.
o Students should be able to control the signing speed of the signer for the embedded ASL video edition of the CBT Math tests.
o Schools need to have the capacity to support CBT testing. There was some concern about meeting the 2019 deadline for testing virtually all students online.
o Practice tests and sample items should be available for accommodated forms. Now they are very limited.
The work group reviewed the discussion questions and made ad hoc additions to the questions, specifically adding considerations for MCAS 2.0 STE. Department staff and work group together decided that the group should address: paper tests options, read- aloud, and side-by-side Spanish-English grade 10 Mathematics test at today’s meeting.
Discussion Topic:
v Should paper-based tests continue to be available as an accommodation for students with disabilities and/or ELLs if they are unable to use a computer?
Comments:
o The work group wanted to make PBT available for SWDs and ELLs, and to any student who is unfamiliar with technology.
o There could be a time limit on the amount of time a student can continue to take a paper based test (e.g., 1-2 years to learn technology if the student does not have a disability-related challenge). The time limit could motivate districts to teach all students how to use CBT.
o PBT should be available after 2019, but its use should be monitored.
o There should be a minimum student to computer ratio in schools before students are expected to participate through online tests.
o PBT must be offered after 2019 due to inequities of school district funding and economic conditions in the home between students.
Recommendations
· Continue to offer paper-based tests as an accommodation for SWD and ELLs, even after 2019.
· Students who are unable to use a computer, or are unfamiliar with a computer, should also be able to take a paper-based test. (Districts should develop a plan to move these students to the CBT as soon as possible).
Discussion Topic:
v Should read-aloud be available for Math and Science tests for all students, as is now available on PARCC; or only for ELLs and SWDs?
Comments:
o A read-aloud is necessary for many students and is vital to gain an accurate measure of certain students’ math skills.
o The challenges of the read-aloud of paper-based tests: consistent administration of the test (reading it correctly, pacing); and the burden on test administrators.
o Strong support that it should be available for ELLs and for SWDs.
o Could there be a cutoff based on a reading score in order to get the read-aloud/TTS accommodation for ELA? Should it be literacy sub-score of 4 or 1-3 or a Team decision (a group of people familiar with the student)?
o PARCC math test required a lot of reading; it seemed to be reading comprehension test.
o Since any student can have the PARCC math test read-aloud on PARCC tests, could any student have a word or phrase read aloud, as with the MCAS accommodation “16B?”
o The criteria should be the same for the read aloud or TTS as for other standard accommodation; i.e., that the student should use the accommodation/feature routinely for instruction.
o If a student uses the feature during routine instruction than it should be available on the test since it does not impact the construct being measured.
Recommendations:
· Continue to allow the read aloud accommodation for SWD and for ELLs for Math and STE tests.
· Some recommended that students without identified disabilities or ELL status be allowed to receive a read aloud, but not unanimous.
· Majority support to allow any student to have a word or phrase read aloud on math or STE tests
Discussion Topic:
v Should a Spanish language grade 10 mathematics test remain available?
Comments:
o It seems unfair to students who speak other languages. If the test is available in Spanish, than it should be available in other languages.
o Retain the Spanish language math test because it has helped many students achieve a passing score in at least one competency determination (CD) content area.
o Some districts have a significant number of Portuguese and Haitian Creole speaking students. Those students also need a test in their native language as well. The group was reminded about Massachusetts’s “English-only” status.
o Translations appear not to be produced at the highest level of accuracy in terms of language biases, different Spanish-language cultures, and regional references and dialect.
Recommendations
· Recommend continuing the grade 10 Math in Spanish (side-by-side if possible)
· Some recommended producing the grade 10 Math test in other languages, as well.
The meeting was adjourned and will reconvene the following week on Tuesday, February 2 at 2:00 p.m.
1