Interoffice MemoRFP #071I2200091

To: / Greg Faremouth, IT Division Director
DTMB Procurement
From: / Steve Motz, IT Division Buyer
DTMB Procurement
Date: / November 7, 2012
Subject: / Award of RFP – 071I2200091 – Enterprise Fraud Detection

GENERAL:

The State of Michigan (SOM) issued Request for Proposal (RFP) #071I2200091 to solicit proposals from qualified bidders to install, configure and implement a commercial off-the-shelf (COTS) software data analytics product used for the purpose of fraud detection.

JOINT EVALUATION COMMITTEE:

The Joint Evaluation Committee (JEC) for this RFP consisted of the following individuals from the Michigan Department of Community Health (DCH), the Michigan Department of Human Services, DTMB, UIA, the Independent Validation and Verification (IV&V) team, and the Project Control Office (PCO):

Voting Members / Technical Advisors
  • Michele Warstler - DCH
  • Robin Thomas - DHS
  • Kirt Berwald - DTMB
  • Steve Motz - DTMB
  • Clay Tierney - UIA
/
  • Cynthia Green Edwards - DCH
  • Karen Parker – DCH
  • Pam Huffman - DCH
  • Kevin Coveart – DTMB
  • Lynne Draschil – DTMB
  • Jim Hogan - DTMB
  • Brad Pagratis - DTMB
  • Debra Patterson – DTMB
  • Linda Pung – DTMB
  • Scot Ellsworth
  • Andrei Verevko - DTMB
  • Larry Freimark - IV&V PM
  • John Henige – UIA
  • Jim Mahony - PCO
  • Tim Saar - PCO
  • Bob Wicks –PCO
  • Greg Beachnau – PCO

BIDDERS:

The RFP was posted on the DTMB Website on February 17, 2012. Four (4) organizations submitted RFP responses by the published due date of March 23, 2012:

  1. IBM Corporation
  2. Information Builders
  3. On Point Technology, Inc.
  4. SAS Institute Inc

AWARD PROCESS:

Method of Evaluation

Proposals were evaluated by the JEC chaired by DTMB Procurement.

Step 1: Minimum Mandatory Requirements
As outlined in the RFP, to be eligible for consideration, every bidder was required to meet the minimum qualifications and conditions listed below. The State determined that all four (4) bidders passed the following minimum mandatory requirements:
  1. The COTS data analytics product being proposed must currently be in production in at least one (1) customer environment for the purpose of fraud detection.
  2. The Prime Bidder must have previous experience with the COTS data analytics product being proposed as a fraud detection solution. This experience must include at least one of the following:
  1. configuring the COTS data analytics product being proposed as a fraud detection solution
  2. implementing the COTS data analytics product being proposed as a fraud detection solution
  3. project management of the COTS data analytics product being proposed as a fraud detection solution
Evaluation Criteria

Following the determination of the pass/fail review of the mandatory minimum requirements, all bidders were evaluated against the evaluation criteria outlined in Step II shown in the table below. Bidders who received the opportunity to provide a Best and Final Offer were evaluated against this same evaluation criterion.

Evaluation Criteria / Points
Step II Technical Proposal Evaluation
- Approach / 80
  • Section 1.104 – Solution Overview
/ 20
  • Section 1.104.I – Services and Deliverables to be Provided
  • Appendix B: Breakdown of Hardware and Related Software
  • Appendix D: Preliminary Project Plan
/ 40
  • Section 1.104.IIA – Technical/General System Requirements
/ 5
  • Section 1.104.IIB – Functional Requirements
/ 15
- Corporate Experience / 12
  • Section 4 – Required Bidder Information
/ 12
- Staffing / 8
Section 1.201 - Bidder Staff Roles and Responsibilities
  • Appendix A: Project Manager Resume Summary Template
  • Appendix E: Organization Chart and Staffing Resource Table
/ 8
Total / 100

Oral Presentation: The State requested that all vendors within the competitive range conduct an oral presentation of their proposal.

Price Evaluation

From the Step II evaluation criteria scoring, only those proposals receiving a score of eighty (80) points or more of the total maximum possible score were considered for award.

All price proposals were opened. However, prices were only evaluated from those Bidders meeting the minimum point threshold of 80 points. Evaluation of price proposals included consideration for a Qualified Disabled Veteran Preference. Public Act 431 of 1984, as amended, establishes a preference of up to 10% for businesses owned by qualified disabled veterans meeting the minimum point threshold for passing.

Pricing Negotiations

The State entered into negotiations with bidders based on their submitted price proposed. No modification to the technical proposal was allowed. The State requested that all vendors that were determined by the JEC to be in the competitive range at the time of request, submit updated pricing to reflect any pricing reductions bidders were able to provide the State.

Best and Final Offer (BAFO)

After thorough review of all proposals submitted and giving due consideration to the technical and pricing aspects, it was in the best interest of the State to issue a Best and Final Offer (BAFO) request per section 3.046 of the Request For Proposal (RFP).

The selection process described in the RFP did not lead to a viable award recommendation due to additional functional requirements that were defined for the Michigan Department of Community Health (DCH) Medicaid program that were not included in the initial RFP. The updated requirements for the Department of Community Health Medicaid functionality were provided in the BAFO. The State evaluated BAFO responses based on the evaluation criteria included in the original RFP.

The JEC conducted a second round of oral presentations for those bidders who were required to respond to the BAFO request.

Award Recommendation

The award recommendation was made by the JEC to the responsive and responsible bidder who offered the best value to the State of Michigan. Best value was determined by the bidder that met the minimum point threshold and offered the best combination of the factors stated in RFP section 3.022, and price, as demonstrated by its proposal.

Reference Calls
The State contacted references for those vendors within the competitive range.
Reservations
As stated in the RFP, in awarding a contract, the State reserves the right to:
(a) consider total cost of ownership factors in the final award recommendation (i.e. transition costs, training costs, etc.).
(b) award by item, part or portion of an item, group of items or total proposal, to reject any and all proposals in whole or in part, if, in the Director of Procurement’ judgment, the best interest of the State will be so served.
(c) award multiple, optional use contracts. In addition to the other factors listed, offers will be evaluated on the basis of advantages and disadvantages to the State that may result from making more than one award.
(d) consider overall economic impact to the State in the final award recommendation. This includes considering principal place of performance, number of Michigan citizens employed or potentially employed, dollars paid to Michigan residents, Michigan capital investments, economically disadvantaged businesses, etc.
(e) award to another ‘best value’ contractor in case the original Awardee does not accept the award. This reservation applies for all of our solicitations whether they are quotes, bids, proposals, pre-qualified or pre-registered programs.
(f) give a preference for products manufactured or services offered by Michigan firms if all other things are equal and if not inconsistent with federal statute. (See MCL 18.1261)

(g) disqualify any bid based on Sections 1.1 through 4.1 in the general Certifications and Representations (completed at vendor registration).Award Decision

Protests

A bidder wishing to protest the award recommendation must submit a protest, in writing, by 5:00 p.m. EST on the date stated on the notice of award recommendation. Bidder must include the RFP number and clearly state the facts believed to constitute an error in the award recommendation along with the desired remedy. More information is available at click on the “Vendor Information” link.

State Administrative Board

The State Administrative Board (SAB) must approve all contracts and purchase orders in excess of $250,000. The decision of the SAB regarding the award recommendation is final. However, SAB approval does not constitute a contract. The award process is not complete until the bidder receives a properly executed contract or purchase order.

1

IBM Corporation

EVALUATION RESULTS:

A. IBM Corporation

Weight / Score
Approach / A. Section 1.104 – Solution Overview / 20 / 13
B. Section 1.104.I – Services and Deliverables to be Provided
B1. Appendix B: Breakdown of Hardware and Related Software
B2. Appendix D: Preliminary Project Plan / 40 / 28
C. Section 1.104.IIA – Technical/General System Requirements / 5 / 5
D. Section 1.104.IIB – Functional Requirements / 15 / 12
Corporate
Experience / E. Section 4 – Required Bidder Information / 12 / 8
Staffing / F1. Section 1.201 - Bidder Staff Roles and Responsibilities
F2. Appendix A: Project Manager Resume Summary Template
F3. Appendix E: Organization Chart and Staffing Resource Table / 8 / 5
TOTAL / 100 / 71

A. Section 1.104 – Solution Overview

(13/20 Points Possible)

Strengths:

  • Proposed solution architecture can utilize SOM VMWare based infrastructure (Art1 -(Statement of work) 1.103, Page 1)
  • Proposed solution and environment overview outlined functional design, including interfaces with required SOM data sources and systems. (Page 2, Solution Overview)
  • Proposed solution could leverage many of the State’s existing tools.
  • Proposed solution offers a combination of rules based, anomaly, predictive, and social network analysis, which was further demonstrated during their oral presentation. (Page 10, Solution Overview B)
  • The user interface demonstrated was user friendly. (Oral Presentation)

Weaknesses:

  • Phases 1 and 2 of Bidder proposal both refer to UIA. The proposal did not demonstrate an understanding of the Phase 2 requirements associated with DHS. (Page 7, Solution Overview A)
  • ANBE and SPSS are discussed separately, but the proposal did not identify where the two proposed solutions have been implemented together. (Page 14, Solution Overview H)
  • During the oral presentation the bidder did not demonstrate an in depth overview and understanding of the entire solution they are proposing to the State. Both solutions were demonstrated independently with minimal discussion surrounding the integration of the two proposed software products. (Including both SPSS and I2)

B1. Section 1.104.I – Services and Deliverables to be Provided

B2. Appendix B: Breakdown of Hardware and Related Software

B3. Appendix D: Preliminary Project Plan

(28/40 Points Possible)

Strengths:

  • Responses indicated compliance with the requirements of section 1.104.I, by not identifying any exceptions (Various Pages)

Weaknesses:

  • Responses throughout Article 1 lacked detail and did not demonstrate that the Bidder understood the State requirements or had an approach to meet all of the requirements. (Various Pages)
  • Bidder included an assumption that there will be no change required for data mapping for MiDAS. This assumption increases risk to timeline and cost as there will likely be changes. (Page 8, Solution Overview A)
  • The project plan assumed a Contract Start date of 4/18/2012, however the RFP clearly indicated 6/1/2012. (Appendix D)
  • Bidder’s approach for Phase 3 failed to incorporate the additional model in Task 9.4.
  • During the question and answer period a vendor asked, “Do you envision any employees participating in this project, if so how many?” The State responded with the following, “The State will have limited resources available to assist in the configuration of the solution to SoMI requirements. However, do not make the completion of any activities dependent upon State resource participation.” The bidder only provided for 120 of their hours to perform the following Phase 3 tasks for UIA:

Phase 3 UIA: Requirement & Design

8.1 Requirements Definition

8.2 Functional Design

Phase 3 UIA: Construction & Testing

9.1 Construction and Testing Plan (SEM 02-Exp)

9.2 Data Preparation

9.3 Validate Existing UIA Fraud Detection Analysis Scenarios

9.4 Configure UIA Tax Fraud Detection Analysis Scenario

9.5 Integrations

9.6 Transition and Operations Planning

9.7 System Testing

9.8 User Acceptance Testing

Phase 3 UIA: Implementation

10.1 Production Cutover

10.2 Installation Testing

10.3 Post Implementation Evaluation Report (PIER)

C. Section 1.104.IIA – Technical/General System Requirements

(5/5 Points Possible)

Strengths:

  • All technical/general system requirements indicated in the RFP marked as “met” by the vendor with no source code modification or additional programming. (Part A, Pages 90-99)

Weaknesses:

  • Bidder failed to check indicator, and did not show if they met this requirement. (Error Handling Page 95, Requirement 8-2)

D. Section 1.104.IIB – Functional Requirements

(12/15 Points Possible)

All functional requirements indicated in the RFP were marked as “met” by the vendor with no source code modification or additional programming. In some instances only system reconfiguration is needed. (Part B, Pages 100-125).– See Weaknesses below:

Weaknesses:

  • Requirements 6-2 and 6-3 cannot be met by the proposed solution. Vendor indicated it could be met by marking an “A” signifying thatconfiguration was required to comply with the requirement. This should have been marked as “no” since the vendor does not have pre-configuration. (Proposal Page 118)
  • Many of the requirements that were indicated as being “met” out of the box in the technical proposal would require some level of configuration. This conflicted with the responses provided in the written proposal (Oral Presentation).

E. Section 4 – Required Bidder Information

(8/12 Points Possible)

  • IBM Provided 3 references summarized below:

1)Trust Solutions (Investigator for Federal Government Client in Medical Fraud)

  • This project was completed in 2007 and is for the SPSS product that was proposed.
  • This predictive modeling solution is used to uncover Medicare fraud.

2)US Department of HHS - Office of Inspector General (OIG)

  • This project is for the i2 investigative intelligence solution proposed.
  • This project has not been completed and is ongoing.
  • The solution is used to identify over billing, fake billing, fake fronts, identify theft, recipient benefit fraud, etc.
  • This solution i2 is a recent acquisition of IBM and the client has been working with IBM since September 2011.
  • IBM did not provide fraud detection experience of the COTS fraud detection product in in the response box, but discussed in the project description.

3)Canadian Revenue Agency.

  • This project is for the i2 investigative intelligence solution proposed.
  • This project has not been completed and is ongoing.
  • The solution is intended to identify fraud and misconduct.
  • IBM did not provide fraud detection experience of the COTS fraud detection product in the response box, but discussed in the project description.

Weaknesses:

  • Haven't done a complete enterprise fraud solution for multiple state agencies.
  • None of the references provided include the combination of all proposed COTS software implemented as a single solution. (Article 4)
  • IBM was notified of a termination for default on a contract with the State of Indiana (Article 4, Page 156)

F1. Section 1.201 - Bidder Staff Roles and Responsibilities

F2. Appendix A: Project Manager Resume Summary Template

F3. Appendix E: Organization Chart and Staffing Resource Table

(5/8 Points Possible)

Weaknesses:

  • IBM Project Manager is only allocated to 10%on-site. Vendor provided conflicting percentages in Appendix A and Appendix E regarding the level of commitment and subsequently clarified that the commitment will vary from 10% to 30%. (Page 127)
  • 2nd Requirement for "Documented experience in successfully implementing the Fraud Detection COTS product with data from both a legacy mainframe environment and relational database management systems (RDBMS)."

Bidder response did not indicate compliance with this requirement. (Page 127)

  • No client contact information provided for Project Manager. (Page 127)
  • Bidder did not indicate years of experience managing implementation of fraud detection COTS product as required. (Page 132, Project Manager resume)
  • Project Manager Resumedid not identify experience managing in government environment. (Page 133-134)
  • Did not provide an organizational chart and thestaffing resource plan does not demonstrate levels of participation. Vendor referenced the project plan file as having this information, but it was not provided, instead, they just listed resource/role names.

Price: The bidder did not meet the minimum point threshold, and pricing was not considered.

1

Information Builders

B. Information Builders

Weight / Score
Approach / A. Section 1.104 – Solution Overview / 20 / 16
B. Section 1.104.I – Services and Deliverables to be Provided
B1. Appendix B: Breakdown of Hardware and Related Software
B2. Appendix D: Preliminary Project Plan / 40 / 28
C. Section 1.104.IIA – Technical/General System Requirements / 5 / 5
D. Section 1.104.IIB – Functional Requirements / 15 / 9
Corporate
Experience / E. Section 4 – Required Bidder Information / 12 / 10
Staffing / F1. Section 1.201 - Bidder Staff Roles and Responsibilities
F2. Appendix A: Project Manager Resume Summary Template
F3. Appendix E: Organization Chart and Staffing Resource Table / 8 / 7
TOTAL / 100 / 75

A. Section 1.104 – Solution Overview

(16/20 Points Possible)

Strengths:

  • Bidder’s response indicated agreement and adherence with SEM/PMM. (Various Pages)
  • Bidder provided a good description on GUI based fraud reporting. (Page 17)
  • Maps demonstrated in the oral presentation were user friendly and users could easily change the weighting factors. (Oral Presentation)

Weaknesses:

  • Chart on page 11 does not demonstrate the cases moving to CAMS or FACTS. This is discussed later on in the proposal. (Page 11)
  • Bidder made an incorrect assumption that the State will make changes to the DTMB data warehouse, as necessary. This assumption increases risk to timeline and cost. (Assumptions, Page 225)
  • Bidder made an assumption that the DTMB data warehouse output for MiDAS is similar to FACTS and CAMS. This assumption increases risk to timeline and cost as this in unknown. (Assumptions, 7th Bullet, Page 225)

B1. Section 1.104.I – Services and Deliverables to be Provided

B2. Appendix B: Breakdown of Hardware and Related Software

B3. Appendix D: Preliminary Project Plan

(28/40 Points Possible)

Strengths:

  • Mapped IB methodology to SEM express. (Page 39)
  • Project plan and proposal demonstrated compliance with SUITE methodology.

Weaknesses:

  • Does not identify types of DTMB staff needed for technical review sessions. (Page 48)
  • Training plan is not very comprehensive. Did not provide all of the requested information in 1.104.3.(Page 68)
  • Section 1.104.6 does not address interface to FEE which is a unique requirement to Phase 2. (Page 72)
  • Section 1.104.9does not address interface to MiDAS which is a unique requirement to Phase 3. (Page 74)
  • The Bidder provided several versions of the Technical Infrastructure proposals which were considerably different from their original RFP response, which included further modifications following their BAFO response.

In particular, original Bidder response specified 2 CPU/cores configuration required for IBI Fraud Viewer component and 4 cores for Fraud Integrator and Analyzer component. The recommended requirements gradually increased to 8 core configurations for each, Fraud Viewer and Fraud Integrator and Analyzer components. The Best and Final Offer process provided bidders with the opportunity to revise their hardware recommendation based on the additional Medicaid requirements. In response to the BAFO, IBI increased the number of cores for the FDPS viewer from 2 to 4, however they did not change the recommended cores for FDPS Integrator & Analyzer and continued to suggest 4 cores for both components (see below).