RFP #071I0200166

Interoffice Memo

To: / Greg Faremouth, Director
Purchasing Operations
From: / Pamela Platte, Buyer
Purchasing Operations
Date: / September 20, 2010
Subject: / Award of RFP - #071I02000166 – Statewide Longitudinal Data System

GENERAL:

The State of Michigan (State), through the Michigan Department of Technology Management & Budget (DTMB) and the Center for Educational Performance and Information (CEPI), have issued this Request for Proposals (RFP) for the purpose of obtaining a state-of-the art Michigan Statewide Longitudinal Data System (MSLDS) and related services to enable the automated access to data and data management services as part of the State’s Education Decision Support System (DSS) initiative. The term of the contract is for an initial 4 year period with five one year options, by written mutual consent of the primary contractor and the State.

JOINT EVALUATION COMMITTEE:

The Joint Evaluation Committee (JEC) for this RFP consisted of the following individuals:

Pam Platte
Purchasing Operations
DTMB / Karen Buckwalter
DTMB
David Judd
MDE / Paul Bielawski
CEPI
Michael McGroarty
CEPI / Lauren Rosier
DTMB

BIDDERS:

RFP was posted on the Bid4Michigan Website on June 18, 2010. Six (6) organizations submitted responses to this RFP by the published due date of August 6, 2010, 3:00 PM:

  1. Compuware
  2. Dewpoint
  3. IBM
  4. McCann Associates
  5. Pearson
  6. Systems Technology Group

3.020Award Process

3.021Method of Evaluation

Proposals will be evaluated by a Joint Evaluation Committee chaired by MDTMB Purchasing Operations.

3.022Evaluation Criteria

The following chart represents the scoring of the particular factors as located in the RFP.

Criteria / Weight
1. / Requirements & Technical Solution and Fit (scope of work) (Section 1.104 Work and Deliverables) / 35%
2. / Project Understanding, Approach and Management (Section 1.104 Work and Deliverables) / 20%
3. / Qualifications and Experience of the Company (Article 4) / 15%
4. / Acceptance of the State’s Terms and Conditions (Article 2) / 10%
5. / Skill Set of People Assigned (Appendix G) / 10%
6. / Turnover Approach (Section 1.104) / 5%
7. / Level of adherence to the State’s request to format responses as outlined in the RFP template. / 5%
Total / Total / 100

Oral Presentation: A Bidder may be requested to make an oral presentation. Oral presentations provide an opportunity for the Bidder to clarify their proposal through mutual understanding. If required, Purchasing Operations will schedule the oral presentation.

Site Visit: The State may conduct a site visit to tour and inspect the Bidder’s facilities. Purchasing Operations will schedule these visits, if required.

3.023Price Evaluation

Only those proposals receiving a score of (80) points or more of the total maximum possible score will be considered for award.

Price proposals will only be requested from those Contractors meeting the minimum point threshold following the proposal evaluation and oral presentation. DO NOT SUBMIT ANY PRICING INFORMATION WITH THE PROPOSAL. PRICING INFORMATION WILL BE REQUESTED OF THOSE CONTRACTORS WHO SCORE 80 OR HIGHER AS A RESULT OF THE JEC/ORAL PRESENTATION. If a price proposal is requested, the State expects that it will be delivered within two business days of the request.

3.024Award Recommendation

The award recommendation will be made to the responsive and responsible Bidder who offers the best value to the State of Michigan. Best value will be determined by the Bidder meeting the minimum point threshold and offering the best combination of the factors stated in Section 3.022, and price, as demonstrated by its proposal.

3.025Reservations

The State reserves the right to:

  • Consider total cost of ownership factors in the award recommendation (transition costs, training costs, etc.).
  • Award by item, part or portion of an item, group of items or total proposal, to reject any and all proposals in whole or in part, if, in the Director of Purchasing Operations’ judgment, the best interest of the State would be served.
  • Award multiple, optional use contracts. In addition to the other factors listed, offers will be evaluated on the basis of advantages and disadvantages to the State that may result from making more than one award.
3.026Award Decision

Award recommendation will be made to the Director of Purchasing Operations for review and approval.

3.027Protests

A Bidder wishing to protest the award recommendation must submit a protest, in writing, by 5:00 p.m. EST on the date stated on the notice of award recommendation. Bidder must include the RFP number and clearly state the facts believed to constitute error in the award recommendation along with the desired remedy. More information is available at click on the “Vendor Information” link.

3.028State Administrative Board

The State Administrative Board (SAB) must approve all contracts and purchase orders in excess of $25,000. The decision of the SAB regarding the award recommendation is final. However, SAB approval does not constitute a contract. The award process is not complete until the Bidder receives a properly executed Contract or Purchase Order.

EVALUATION SUMMARY

A. CompuwarePass:  Fail: Points: 84

Criteria / Score / Weight
1. / Requirements & Technical Solution and Fit (scope of work) (Section 1.104 Work and Deliverables) / 29 / 35%
2. / Project Understanding, Approach and Management (Section 1.104 Work and Deliverables) / 15 / 20%
3. / Qualifications and Experience of the Company (Article 4) / 15 / 15%
4. / Acceptance of the State’s Terms and Conditions (Article 2) / 10 / 10%
5. / Skill Set of People Assigned (Appendix G) / 6 / 10%
6. / Turnover Approach (Section 1.104) / 5 / 5%
7. / Level of adherence to the State’s request to format responses as outlined in the RFP template. / 4 / 5%
Total / Total / 84 / 100

Summary:Compuware provided experiences similar to the size and scope of work the State of Michigan (SoM) required in regard to this RFP. The vendor’s proposed technical solution met the SoM requirements for the purpose of obtaining a state-of-the art MSLDS and related services to enable the automated access to data and data management services as part of the State’s DSS initiative.

1.Requirements & Technical Solution and Fit (scope of work) (Section 1.104 Work and Deliverables)

  • Strength: Vendor will provide a prototype solution based on the RFP requirements prior to the start of the project. (Page 18)
  • Strength: Vendor will provide up to 40 hours of consulting support as necessary to assist the State of Michigan Vantage resources in preparing and conducting the load testing User Acceptance testing. (Page 18).
  • Strength: Vendor has proposed a strong and detailed Testing Approach. (Pages 67-75)
  • Weakness: Vendor proposed SQL Server Integration Services (SSIS) rather than the RFP requirement of SAS Data Integration Studio to perform the ETL process from the source systems to the SLDS. In RFP Section 1.103 Environment the State outlined the software tools and indicated the following: “SAS Data Integration Studio – for extraction, transformation, and load process creation”
  • Weakness: Vendor mirrored the RFP requirement language and did not provide a detailed strategy for capacity planning. RFP Section 1.104 Work and Deliverables C. Technical Architecture/Design, Capacity Plan states the “Bidder must provide a strategy for capacity planning and recommendations for additional items for a successful system implementation and ongoing maintenance.”

2. Project Understanding, Approach and Management (Section 1.104 Work and Deliverables)

  • Strength: Vendor project plan exceeds the RFP deadline requirements (Vendor’s proposed implementation date is 9/9/2011 which exceeds the State’s precedes the States planned implementation date of 9/30/2011) and contains very detailed milestones. (Attachment A)
  • Weakness: Vendor did not demonstrate the required detail for the RFP requirement in their response to Sections 1.104 Work and Deliverables C. Technical Architecture/Design, Metadata Plan, Configuration Plan, and Installation Plan. The Metadata Plan strategy does not address the process for maintaining metatdata. The Configuration Plan and Installation Plan strategies address what these plans are but do not detail strategies for completing the plans. (Pages 52, 57-59)
  • Weakness: On page 17 of their proposal the vendor identified eight functional areas and has assumed equal effort will be required for JAD sessions. On page 18 Compuware states, “Compuware has divided the business requirements provided in Appendix B and Addendum 6 into the eight functional areas. Three JAD sessions will be scheduled for each of these functional areas, for a total of twenty-four (24) 3-hour sessions. These 24 sessions will be spread over a period of 8 weeks, providing time for feedback, updates and approvals.” The complexity of the functional areas identified by Compuware is vastly different and to assume equal amounts of time will be required for each area demonstrates the vendor’s lack of understanding of the complexity of each area. The complexity can be ascertained by comprehending the information included in the RFP, Appendix B. For example, the business rules and calculations (as outlined in Appendix B) required to create the Discipline and AYP/Assessment extracts far outweighs the complexity to create the Graduation/Cohort and Alternative Education extracts, yet an equal amount of JAD time has been allocated to both of these groups.

3. Qualifications and Experience of the Company (Article 4))

  • Strength: Vendor demonstrated in its corporate references an advanced understanding of MSDS and EEM (two of the three Phase 1 source systems) and has an intimate knowledge of both applications and databases because Compuware developed these applications for the State. (Article 4)
  • Strength: Vendor’s corporate references demonstrated experience in business intelligence. (Article 4)

4. Acceptance of the State’s Terms and Conditions (Article 2)

  • Vendor has met the requirement by agreeing with no exceptions to the State’s Terms and Conditions. (Article 2)

5. Skill Set of People Assigned (Appendix G)

  • Strength: Vendor’s proposed Project Manager demonstrated extensive (4+years) experience within the past 5 years managing projects for the State of Michigan which exceeds the “minimum 2 years of experience (in the past 5 years) in managing projects for the public sector (State of Michigan experience strongly preferred but is not required) noted in the provided resume template in Appendix G.
  • Weakness: Vendor’s proposed System Architect key resource did not meet the requirements of this RFP. Vendor selected “No” to the question, “Does resource have the required skill” for the following RFP requirements in Appendix G under the required skills section of the provided resume template: “Minimum 7 years of experience (within the past 10 years) in a lead database administrator role for large scale data warehousing solutions of similar size and scope of this RFP” and “Minimum 5 years of experience (within the past 7 years) architecting large scale data warehousing/business intelligence solutions of a similar or larger size and scope of this RFP”.
  • Weakness: Vendor’s proposed Project Manager does not have a Project Management Professional (PMP) certification. The RFP requirement in Appendix G under the required skills section of the provided Project Manager resume template is “Certification: Project Management Professional (PMP) certification”.

6. Turnover Approach (Section 1.104)

  • Vendor met the requirement

7. Level of adherence to the State’s request to format responses as outlined in the RFP template

  • Vendor generally adhered to the State’s request to format responses as outlined in the RFP template.
  • Weakness: Vendor did not use the non key resource template provided by the State of Michigan, for their non-key, but named project resources, as required in the RFP.

Pricing:

  • $1,882,687.00

  1. DewpointPass:  Fail: Points: 98

Criteria / Score / Weight
1. / Requirements & Technical Solution and Fit (scope of work) (Section 1.104 Work and Deliverables) / 35 / 35%
2. / Project Understanding, Approach and Management (Section 1.104 Work and Deliverables) / 20 / 20%
3. / Qualifications and Experience of the Company (Article 4) / 15 / 15%
4. / Acceptance of the State’s Terms and Conditions (Article 2) / 10 / 10%
5. / Skill Set of People Assigned (Appendix G) / 8 / 10%
6. / Turnover Approach (Section 1.104) / 5 / 5%
7. / Level of adherence to the State’s request to format responses as outlined in the RFP template. / 5 / 5%
Total / Total / 98 / 100

Summary:Dewpoint provided experiences similar to the size and scope of work the State of Michigan (SoM) required in regard to this RFP. The vendor’s proposed technical solution met the SoM requirements for the purpose of obtaining a state-of-the art MSLDS and related services to enable the automated access to data and data management services as part of the State’s DSS initiative.

1.Requirements & Technical Solution and Fit (scope of work) (Section 1.104 Work and Deliverables)

  • Strength: Vendor’s proposed design exceeds the RFP requirement. Vendor provided a great deal of detail and demonstrated specific knowledge of the State’s needs and goals for this project. Vendor’s descriptions of the ETL process flow reflect this specific knowledge. (Throughout vendor’s response to 1.104)
  • Strength: Vendor demonstrated a strong knowledge of the CEPI and OEAA data and as a result provided an informed approach in reference to Article 1, 1.104 Work and Deliverables C. Technical Architecture/Design.
  • Strength: Vendor demonstrated that they recognize the inconsistent nature of the State’s data and proposed the construction of a set of data quality reports. (Page 33)

2. Project Understanding, Approach and Management (Section 1.104 Work and Deliverables)

  • Strength: Vendor provided a detailed test plan, capacity plan and configuration plan.
  • Strength: Vendor’s capacity plan demonstrated the use of SAS functions for reporting processor usage for measuring performance.
  • Strength: Vendor identified Vantage as a proactive performance validation tool and referenced performance tuning being a continuous activity, “rather than a reaction to an acute performance problem”. (Page 37)
  • Strength: Vendor demonstrated advanced knowledge of the State’s SUITE methodology and provided specific references to SUITE templates throughout their proposal. (Several examples on page 25)
  • Strength: Vendor demonstrated advanced understanding of the State’s enterprise architecture within their proposed capacity plan. (Page 59)
  • Strength: Vendor demonstrated advanced capability to communicate in detail and in their own words, technical concepts associated with this RFP in a way that was understandable to audiences with varying technical capabilities. This demonstrates the vendor understands that their clients represent both business and technical interests.
  • Strength: Vendor’s proposed implementation date is 9/30/2011 which meets the State’s preferred implementation date of 9/30/2011.

3. Qualifications and Experience of the Company (Article 4))

  • Strength: Vendor’s corporate reference demonstrated specific experience with all Phase 1 source systems and some future phase source systems including system structures, cleansing, storing, analyzing, and reporting. (Corporate Reference #2, Article 4 Section 4.014 Prior Experience/Past Performance)
  • Strength: Vendor’s corporate reference demonstrated experience with Data 4 Student Success (D4SS), which is a core component of the data portal referenced in Section 1.002 Background of the RFP. (Corporate Reference #1 in Article 4 Section 4.014 Prior Experience/Past Performance)

4. Acceptance of the State’s Terms and Conditions (Article 2)

  • Vendor has met the requirement by agreeing with no exceptions to the State’s Terms and Conditions. (Article 2)

5. Skill Set of People Assigned (Appendix G)

  • Strength: Vendor’s proposed SAS institute System Architect/DBA (SAS) expert demonstrated advanced, relevant experience with school districts, BI reporting and developing ETL jobs (Appendix G). His proposed role on the MSLDS project was Strength: thoroughly described in the vendor’s RASIC chart provided at the end of the vendor’s Article 1 Statement of Work section.
  • Strength: Vendor’s proposed system architect demonstrates a strong knowledge of State of Michigan infrastructure and enterprise architecture. (Appendix G, Page 5 System Architect/DBA Resume Template)
  • Weakness: Vendor’s proposed key resource for the System Architect/DBA did not meet the requirements of this RFP. Vendor selected “No” to the question, “Does resource have the required skill” for the following RFP requirement in Appendix G under the required skills section of the provided resume template: “Minimum 3 years of experience (within the past 7 years) using Microsoft SQL Server and SAS. (Appendix G)

6. Turnover Approach (Section 1.104)

  • Strength: Vendor identified that they expect there will be a “gap between existing skill sets amongst the State staff and the anticipated skills required to operate and maintain the proposed system” (in addition to the gap and training needs identified in Work and Deliverables 1.104 F. Training section of the RFP. As a result, the bidder proposed to work with the State to develop a training plan as an additional service with their turnover plan. (Page 71)

7. Level of adherence to the State’s request to format responses as outlined in the RFP template

  • Vendor adhered to the State’s request to format responses as outlined in the RFP template.

Pricing:

  • $1,533,226.83

  1. IBMPass: Fail:  Points: 74

Criteria / Score / Weight
1. / Requirements & Technical Solution and Fit (scope of work) (Section 1.104 Work and Deliverables) / 27 / 35%
2. / Project Understanding, Approach and Management (Section 1.104 Work and Deliverables) / 18 / 20%
3. / Qualifications and Experience of the Company (Article 4) / 15 / 15%
4. / Acceptance of the State’s Terms and Conditions (Article 2) / 0 / 10%
5. / Skill Set of People Assigned (Appendix G) / 4 / 10%
6. / Turnover Approach (Section 1.104) / 5 / 5%
7. / Level of adherence to the State’s request to format responses as outlined in the RFP template. / 5 / 5%
Total / Total / 74 / 100

Summary:IBM provided experiences similar to the size and scope of work the State of Michigan (SoM) required in regard to this RFP. The vendor’s proposed technical solution did not meet the SoM requirements for the purpose of obtaining a state-of-the art MSLDS and related services to enable the automated access to data and data management services as part of the State’s DSS initiative.

1.Requirements & Technical Solution and Fit (scope of work) (Section 1.104 Work and Deliverables)

  • Strength: Vendor proposed to provide value-added assets that “support the acceleration of certain design and development activities” to the State, “free of any license charges to the State.” These include their Pre-K through 20 Experience Modeler, Insight at Schools Data Warehouse Model, and Insight at School Integration Assets. (Page 2)
  • Strength: Vendor proposed to “leverage our Capacity Planning methodology and our experience assisting other states in sizing similar Longitudinal Data systems to develop and maintain a MSLDS Capacity Plan.” This provides added value to the State. (Page 60)
  • Weakness: On pages 53 – 56 of the vendor’s proposal, the vendor listed the source systems without providing a detailed solution for Extract, Transform, Load (ETL) and report design as required in the bidder box for Section 1.104 Work and Deliverables, C. Technical Architecture/Design. The bidder box states, “Bidder must provide a detailed solution (including proposed methodologies) and recommendations for additional items for a successful implementation system”.
  • Weakness: Vendor’s proposed test metrics on page 80 are not consistent with the State’s RFP requirements as identified in Section 1.104 Work and Deliverables, D. Development and Testing, Testing and Test Results which demonstrates a lack of specific Michigan Statewide Longitudinal Data System (MSLDS) focus for the vendor’s testing strategy.

2. Project Understanding, Approach and Management (Section 1.104 Work and Deliverables)