Sample Gap Analysis Template 2
Template objective: To document the gap between the customers anticipated requirements and our current capabilities / products
Template owner: Program manager (developed in close collaboration with functional departments)
This is an example of a gap analysis. Please modify or develop something appropriate for your specific pursuit team, depending on the information available during the pursuit team stage
Overall Points to consider:
§ KY wants their own standard setting
§ KY wants a separate placement tool
§ KY wants a new form every third year
§ KY wants Interpretive Guides in the top four languages based on KY standards
Page of RFP / Requirement / Gap Analysis / Shelf or CustomPage 132 / Measurement of Annual Progress in English Language Acquisition / § Shelf provides measurement of overall proficiency in Listening, Speaking, Reading and Writing.
§ We offer an overall scale score and a Comprehension Scale score. / § Shelf
Measurement of Attainment of English Language Proficiency / § We have done an initial alignment to the KY standards but I do not know the extent of the gap. / § Shelf/Custom depending on extent of alignment
State AMAO, Title III, Part A / § CTB can track cohorts of students but this would require a custom report using their state ID number. / § Custom
Identification of ELLs / § Shortened version of test for purpose of identifying student as LEP, placing student into appropriate instructional program and designing instructional plans and services. / § Custom short form
o Depends on the definition of a short form. Shelf is developing two forms that are parallel that contain 100 items at all levels and take between 95-115 minutes to
Page 133 / Title I, Part A, Section 1111(b)(7) / § LAS Links meets the need for each state to provide an annual assessment measuring oral language, reading and writing skills in English / § Shelf as long as they want to use Form A or Form B.
Title III, Part A, Section 3121(d) / § LAS Links can track progress of children in attaining English proficiency including a child’s level of comprehension, speaking and listening, reading, and writing skills in English; Student attainment in challenging State student academic achievement standards.. / § Shelf – depending on the extent we match KY ELP standards. We offer a common, vertical scale that can track growth year over year across grade spans.
Page 134 / ELP standards are intended as guideposts for choosing and adopting a statewide assessment / § LAS Links has an alignment to KY ELP standards. Unsure of the strength of the alignment. / § Custom based on gap analysis on KY standards to LAS Links framework.
Definition of “Comprehension” is the combination of listening and reading / § LAS Links uses a combination of Listening and Reading to define Comprehension. / § Shelf
a. be appropriate for students
b. be aligned to academic content standards
c. be valid and reliable
d. be consistent with technical standards
e. provide reasonable adaptations and accommodations for students with disabilities / § LAS Links covers grades K-12
§ LAS Links is aligned to KY standards although I’m not sure how well
§ LAS Links is valid and reliable
§ LAS Links is consistent with nationally recognized professional and technical standards
§ LAS Links plans to introduce Braille and large print versions of the test although a date for release has not been determined / § Customize items that don’t match with alignment
§ Customize Braille and large print versions if timing doesn’t align with KY needs.
Page 135 / Test Structure – Aligned to KY standards for students K-12 in domains of Speaking, Listening, Reading, and Writing in at least 4 grade spans. / § LAS Links covers students K-12
§ LAS Links covers all four NCLB domains
§ LAS links has five grade spans (K-1, 2-3, 4-5, 6-8, 9-12) / § Shelf minus the alignment to KY standards.
Page 136 / Administration structure / § Administration is same for all grade spans K-12. Please see slide #11 from LAS Links PPT presentation given in Nashville for overview.
§ Speaking – individually administered
§ Listening – individual or group
§ Reading – individual or group
§ Writing – individual or group / § Shelf
Contractor shall work with the individual local school districts to ensure that all LEP students are assessed and shall facilitate the integration of LEP student assessment information with state’s academic and information systems. / § LAS Links can ensure all students are assessed and use central scoring and reporting for Listening, Reading and Writing and provide local scoring for Speaking via the online placement tool. / § Shelf – From Table 21 I can see us pursuing Option II, B
Page 137 / Test Specs:
§ Align to KY standards
§ Define Content
§ Psychometric properties of items
§ Language proficiency level of items
§ Amount of time required to take test
§ Procedures for scoring/test administration / § LAS Links – need to understand alignment
§ We can provide psychometric properties of each item, proficiency levels, procedures for scoring and test administration / § Shelf – depending on alignment to standards.
Alignment to KY ELP standards / § We need to look at the alignment we have / § Shelf/custom
Page 138 / § Longitudinal Assessment
§ Vertical Scaling
§ Forms & Equating
§ Validity and Reliability
§ Developmental Appropriateness / § LAS Links forms A and B provide longitudinal capabilities through one common, vertically aligned scale
§ We have two parallel forms
§ A technical bulletin is due out in December 2005
§ We can provide proof of developmental appropriateness / § Shelf
Page 139 / Test Construction
§ Test Forms
§ Test Item Development and Item Banks
§ ELL Identification Test
§ Computer-based test design
§ Language Accessibility
§ Bias and Sensitivity Review
§ Inclusion/Fairness
§ Security Procedures / § LAS Links does have two equivalent test forms
§ LAS Links does provide multiple choice and constructed response item types
§ LAS Links will not provide a new form every third year
§ LAS Links will not provide items in an item bank
§ LAS Links will not provide a separate identification test or “short form”
§ LAS Links will not be delivered online
§ LAS Links is “language accessible”
§ LAS Links has been through a bias and sensitivity review
§ LAS Links plans to offer Braille and large print versions. Date TBD. We do offer accommodations for administration.
§ LAS Links was developed under rigorous security procedures. / § Shelf for two equivalent forms
§ Customize Item Banks
§ Customize New Form every third year – current form has 100 items per grade span.
§ Customize Short Form used for placement and I.D – need definition of this
§ No part of the test will be online
§ LAS Links in Braille and large print may not be delivered by due date.
Page 141 - 142 / Test Administration / § LAS Links will work as annual administration tool.
§ LAS Links does not have a separate placement tool – Forms A & B can be used for initial identification or for annual assessment / § Customize placement tool
Administration Materials / § LAS Links will not be online – I know it’s a cost option but KY really likes this feature. / § Customize online solution if customer opts for it
Test Administration, Training and Support Materials
§ Testing Coordinator’s Guide
§ Test Administration Manual
§ Student Response Documents
§ Training Procedures
§ Test Accommodations / § LAS Links will provide a test administration manual (Examiner’s Guide)
§ LAS Links will not provide a District or School testing coordinator’s guide
§ LAS Links does not yet have administration manuals for large print or Braille
§ LAS Links will offer precoded labels for use on documents
§ LAS Links will offer training on administration and scoring of test
§ LAS Links has testing procedures for students with special needs or disabilities / § Shelf Examiner’s Guide
§ Customize District/School coordinator’s guide
§ Customize large print and Braille admin manuals depending on timing of shelf product.
§
Page 143 / Packaging of Test and Test Support Materials / § LAS Links can package, ship and provide packing lists. We will not include overage. / § Customize any account management requests to provide overage, scheduling extra personnel, etc.
Distribution of Test and Test Support Materials
§ Contact website
§ CTB sponsored website where KDE track materials shipments
§ Toll-free help line / § LAS Links will not provide a “secure contact list website” for KDE to update contacts, track materials website.
§ Toll-free hotline will only be what is currently available through the catalog. / § Customize secure websites and hotlines
Page 144 / Collection and Receipt Procedures
§ Collection and return of secure materials
§ Pre-paid shipment of materials back to CTB / § Shelf does not manage customer return of documents
§ Shelf does not pay for document return / § Account management to ensure test materials are returned on time for scoring
§ Custom to pay for returned documents
Security Procedures / § Shelf offers additional security at additional cost. / § Shelf
Storage
§ One year storage
§ Electronic imaging & duplication technologies to archive responses for five years / § Shelf offers storage for up to 90 days and it is not electronic storage
§ Shelf can return documents for a fee / § Custom storage
Page 145 / General Scoring Plan
§ General Scoring
§ Scanning & Scoring
§ Constructed Response Scoring
§ Constructed-Response scoring protocols
§ Constructed-Response scoring materials
§ Scoring procedures for CR
§ Scoring Training
§ Local Scoring
§ Scoring/Training for Placement test
§ Quality Assurance
§ Security Procedures / § LAS Links will provide scoring of all parts of the test. Customers must score the Speaking portion of the test.
§ CTB will scan and score all student tests.
§ CTB will provide staff to score constructed - response items. It is unclear if these procedures will meet with the KDE requirements
§ Shelf will establish scoring procedures and protocols for scoring locally and scoring Speaking for all grades.
§ LAS Links does not have a “placement” tool so therefore will not have a training/scoring plan / § What are the KDE requirements for scoring constructed-response items? If they need to approve this it may need customization
§ All scoring is shelf except for training/scoring plan for placement test
Page 148 / Scoring Data Mgmt & Analysis
§ Electronic Data Mgmt
§ Data dictionary / § LAS Links will have CSV files
§ Unclear if descriptions of data elements will be in line with KDE expectations / § Possible customization of data management – data dictionaries
Data warehousing / § A plan for longitudinal tracking has not been defined. Shelf is revisiting this in Dec. 2005 as it is unclear of the demand for this feature. Even if we provided it, I’m sure we are using proprietary systems (top of 149). / § Custom longitudinal tracking
Calibrating/Scaling and Equating / § LAS links has calibrated and scaled all items
§ LAS Links does not currently use IRT theory but it can be made available / § Shelf
Alignment Study / § CTB has already provided an alignment. As mentioned before, it is unclear how well LAS Links aligns to KY ELP standards. We may have to conduct a new alignment if the standards have been updated since March 2005. / § Shelf
Linking Study / § LAS Links will provide linking tables back to LAS, Spanish LAS and PreLAS / § Shelf
Security Procedures / § As mentioned before, shelf can ensure security with all documents / § Shelf
§ / §
Page 150 - 151 / Setting Proficiency Standards / § LAS Links has already held a standard setting. A new standard setting would be custom / § Custom Standard Setting
Page 151 / Reporting Plan
§ Student Level Reports
§ School, District, State Summary Reports
§ School Reports
§ District Reports
§ State Reports
§ Placement test reports / § LAS Links will provide student/district level reports with all specified information minus the state I.D. number
§ State level reporting will be revisited in Dec. 2005
§ Reports will not be offered electronically
§ LAS Links does not offer a placement test so combined reports with annual and placement tests are not an option / § Customize state reports
§ Custom deliver electronic reports
§ Custom placement test reports
Page 152 / District/State AMAO Title III Reports / § LAS Links does not currently offer this / § Custom Title III AMAO Reports
Page 153 / Electronic Student Data Files
§ Monthly reporting
§ Secure website / § LAS Links will not deliver on a monthly basis student information in any form
§ Shelf does not have a secure website to post any test results / § Custom delivery of monthly student data files
§ Secure website to post placement test results
§ Electronic Student Response Files
§ AMAO Reports
§ Research Files / § No data will be available online / § Custom Student files
§ Custom AMAO Reports
§ Custom research files
Page 154 / Interpretive Guide / § This will need to be based on a custom standard setting
§ Guides need to be translated into 4 languages / § Custom Interpretive Guides
Technical Reports / § Annual reports will not be produced
§ Updating item bank – LAS Links will not have item banks / § Custom Technical Reports