Request for Responses

Massachusetts Department of Elementary and Secondary Education

High School Equivalency Testing Program

Attachment A

14ACLAS1

Introduction

The Massachusetts Department of Elementary and Secondary Education (ESE) is seeking responses from bidders with the capacity to deliver a computer-based and/or paper-based assessment that will be aligned with the Common Core State Standards. The test battery will include assessments in reading, writing, mathematics, science, and social studies and will be used to measure the knowledge and skills of adults and out-of-school youth for the purpose of determining whether they qualify for the Massachusetts High School Equivalency Certificate. The test battery must be available at over 30 test centers across the state to approximately 11,000 examinees annually. The Department will award a three-year contract to deliver high school equivalency tests from January 2014 through December 2016. Responses must show how all costs associated with this testing program will be covered by the fees paid by those taking the tests.

Responses must provide a narrative, test specification manual, and supporting documents that address the sections of this Scope of Services in the order in which they appear below.

Scope of Services:

I. Organizational Capacity

Responses must describe the vendor’s experience and expertise in developing and administering large-scale assessments. Eligible applicants must have a minimum of three years of experience in providing statewide or national high-stakes testing programs; they also must submit a minimum of three current professional references to substantiate their capacity and qualifications. Bidders must address the following topics.

·  Describe experience in administering computer-based tests and discuss their availability.

·  Describe experience in administering paper-based tests and discuss their availability.

·  Describe experience in developing and administering tests in the subject areas listed in the introduction above.

·  Describe experience in administering tests in languages other than English.

·  Describe experience in administering tests for examinees that require accommodations.

·  Describe experience in aligning assessments with the Common Core State Standards.

·  Describe experience in administering tests that contain constructed-response items.

·  Confirm the vendor’s capacity to have a test battery available in the quantities stated above by January 2, 2014.

II. Test Content and Forms

Responses must provide details regarding test content and design, including the item development process and describe how tests will be aligned with the Common Core State Standards.

·  Responses must describe how the test contains sections for English language arts (reading and writing), mathematics, science (not necessarily content specific), and social studies (not necessarily content specific). The test must also include at least one essay, with the remainder being machine score-able.

·  The ESE prefers test length to be comparable to the current Massachusetts high school equivalency test (75 minutes ELA/writing, 65 minutes for machine-score-able ELA/reading, 45 minutes for an ELA/writing essay, 70 minutes for social studies, 80 minutes for science, and 90 minutes for mathematics). Responses that propose a testing approach that measures equivalent constructs but makes more efficient use of test time are welcome.

·  Responses must address how the vendor will provide annually the numbers and types of forms listed below and additional alternate forms. Responses may propose to provide an existing test or develop a new test.

Three new test forms in English

A Spanish translation of each of the three English forms

An English Braille version of two English forms (paper-based test version only)

An English audio version of each of the three English forms (PBT only)

A Spanish audio version of each of the three translated Spanish forms (PBT only)

A large-print English version of each of the three English forms (PBT only)

A large-print Spanish version of each of the three translated Spanish forms (PBT only)

Responses may propose to use previously used items provided those items are not published on any website or in any textbook or review book.

III. Test Accommodations

Proposals must provide a plan for complying with the Americans with Disabilities Act (ADA) of 1990.

·  Describe in detail the available accommodations for examinees with disabilities, including but not limited to:

1.  Attention deficit/hyperactivity

2.  Psychological or psychiatric disabilities

3.  Learning or other cognitive disabilities

4.  Physical/chronic health disabilities

5.  Intellectual disabilities.

·  Responses should provide a plan that outlines a review and decision process administered by the vendor to determine eligibility for accommodations. The plan should address some or all of the following testing accommodations in each language/version:

1.  Extended time

2.  Special location/private room/small group

3.  Audiocassette/CD

4.  Large print with extended time

5.  Calculator/talking calculator

6.  Scribe

7.  Supervised breaks

8.  Signed interpreted instructions for the deaf/hard of hearing

·  Describe the information and application materials for individuals with disabilities including accommodation guidelines for advocates, diagnosticians, and examinees.

·  Describe the review and decision process for accommodation eligibility for both CBT and PBT, including the roles of the vendor, test center, and ESE in this process.

·  Describe any differences in accommodations procedures and available accommodations between CBT and PBT.

Responses must include copies of current and/or proposed test accommodation applications and approval materials.

IV. Test Delivery Systems

Computer-Based Test (CBT)

Massachusetts is moving toward CBT and plans to begin operational testing during this contract. With this future direction in mind, responses should address the topics in this section.

·  Describe the technology needed by test centers, including but not limited to computer requirements, the required operating system and software needs, ability to scan paper-based documentation for scoring, and required security systems.

·  Describe in detail the computer skills and knowledge needed by the examinees.

·  Describe how the vendor will gather all data from the test administration platform, score all items, sum the scores, and determine whether the test-taker has obtained a passing score.

·  Describe the process and timetable by which scores will be reported to examinees.

·  Describe how the vendor will transfer the demographic data and test results daily in an encrypted format to the Department of Elementary and Secondary Education.

Vendors must verify that local Massachusetts testing centers have the capability to test using the vendor’s test administration platform.

·  Prior to conducting CBTs, the vendor must provide a technical readiness tool, utility, or process for local testing centers to use in verifying the capacity of their technical infrastructure for conducting computer-based assessments that use the vendor’s test administration platform.

·  Responses must specify that the vendor can provide a test administration platform that can be used by examinees to take the test and by administrators to manage the testing process on a reasonable proportion of the computers in place at the state’s testing centers.

·  Responses must describe approaches to the CBT administration platform that meet the following requirements:

  1. Some testing locations in Massachusetts (correctional facilities, etc.) where the populations of examinees are prohibited from accessing the Internet will require the administration of a PBT.
  2. For testing locations that can access the Internet (e.g., community colleges, adult learning centers, libraries), ESE is open to a test administration platform that is either connected to the Internet in real time or that intermittently connects to the Internet through a software program installed on individual computers that connect to databases through the Internet for the purposes of pulling items, banking scores, or other information.

·  Responses should describe in detail the test system specifications for the vendor’s proposed test administration platform, which may include:

  1. Test access control
  2. Administrative access control
  3. Security or test content and examinee data
  4. Desktop security during testing
  5. Network availability
  6. Data interoperability

·  Responses must describe the user interface to the test administration platform that the vendor proposes to use at the state’s local testing centers, including: how the examinee navigates through the test; how items of different types are displayed on the screen and responded to; and how the user accesses and completes the test. Screen shots may be included. The description should also include details of any test-taking tools that are currently available or under development.

·  Responses must describe the administrative system for the proposed test administration platform. This system must be capable of managing the delivery of tests and supporting various aspects of the assessment program (e.g., data, test, security management).

·  Responses must propose a comprehensive solution for system monitoring and reporting that ensures the availability of the testing system to examinees and provides user statistics to the ESE. If such a monitoring system is not currently available, the vendor should propose a phase-in plan over the life of the contract.

·  Responses must address system availability, maintenance, and updates. The vendor’s test administration platform must be available to examinees, with the exception of approved scheduled downtime, to deliver and score tests and conduct related administrative functions. Testing in Massachusetts takes place throughout the year (including evenings and weekends). Responses must detail disaster prevention and recovery strategies and maintenance and update schedules that are currently in place.

Paper Based Test (PBT)

·  Describe and differentiate between the roles of the vendor, the test centers, and the Department with regard to the shipping, storage, and handling of secure test materials, including test booklets and answer sheets.

·  Describe in detail the method by which completed answer sheets will be scored, including the roles of the vendor, the test centers, and the Department in the scoring process.

·  Describe any materials and equipment needed by the test centers and/or the Department (i.e., scanners, scoring templates, secure mailers) to perform their roles in the scoring process.

·  Describe the process and timetable by which scores will be reported to the examinees.

·  Describe how the vendor will transfer the demographic data and test results daily in an encrypted format to the Department of Elementary and Secondary Education.

V. Scoring and Reporting

Responses must include a plan for vendor-provided scanning and scoring of PBTs and gathering the data and scoring the CBTs. The ESE must be able to defend the use of scores from the test proposed by the vendor to determine high school equivalency for adults and out-of-school youth. The plan must address the topics in this section.

·  Describe how raters will be hired and trained to score the constructed- response/writing component of the test and how inter-rater reliability will be ensured.

·  Describe how the vendor will verify comparability of the test under CBT and PBT administration conditions.

·  Describe how the test will be normed and whether out-of-school youth and adults are or could be included in the norming process.

·  Describe the process by which the cut score on the test indicating high school equivalency will be determined.

·  Describe the equating/linking approach the vendor will use to ensure that test forms are comparable from year to year (2014 to 2015 to 2016).

·  Describe how the results from the test are a reliable measure for determining whether examinees have reached the level of performance required to pass the current GED test, which is set at scores equal to or higher than those earned by the top 60% of graduating high school seniors.

·  Discuss how the vendor is moving toward incorporation of the Common Core State Standards, particularly the College and Career-ready Determination Policy as articulated by PARCC.

VI. Preparation Materials, Training, and Assistance

Responses must include information on preparation materials for examinees and training for staff in the testing centers on administering the test.

·  Describe the availability of preparation materials, including practice tests, for adult education programs and prospective examinees

·  Provide information on training opportunities for educators on the materials related to the assessment.

·  Describe training for testing center staff and ESE staff regarding the technical aspects of test administration and accommodations. Training may take place face-to-face or through live, interactive webinars.

·  Describe assistance and support that will be available to testing center staff and individuals before, during, and after testing regarding issues related to ordering materials, delivery, test administration (including CBT administration questions and issues), and scoring. Dedicated telephone, email, and fax numbers are acceptable methods for communication.

VII. Security and Test Center Procedures

The security of test content and student data is very important to the Massachusetts Department of Elementary and Secondary Education. The vendor must adhere to all relevant federal and state laws regarding the confidentiality of personal information and the state’s security protocols.

·  Describe how information and data will be kept secure at all times during the program.

·  Describe how the operational test (CBT, PBT) will be kept secure (encrypted files, shipment of materials by delivery service, FTP site, etc.).

Responses must describe all regulations and procedures that are followed to ensure the integrity of the tests and the testing process, including but not limited to the following:

·  Qualifications for test center personnel

·  Handling and storage of secure materials

·  Test room supervision

·  Verification of the eligibility and identity of examinees

·  Recordkeeping

If applicable, differentiate between the roles of the vendor, the test centers, and the Department with regard to developing and implementing these and other regulations and procedures.

Budget:

All costs associated with test development, registration, administration, training, materials, scoring, reporting, and any other program activities, including retests, must be covered by fees collected from examinees. The response must show how the fee schedule will support testing program costs.

Budget information must be submitted in task format with a budget narrative showing/explaining cost breakouts. The budget information will be used to help determine whether the proposed fees are reasonable and adequate for the program requirements.

The contractor must keep and make available records and accounts of project expenditures and income for the purpose of audit by the Commonwealth of Massachusetts and its agents, as authorized by law or regulation to conduct such audit.