Guidelines for Evaluating EHR Vendors
I. Before Looking at Vendors[1]
§ Know your priorities: identify the problems and issues within your practice, which the system needs to solve.
o Assess the needs of providers and staff. (See HealthInsight’s Personal Goals List, which may help providers and staff identify needs in their area) Summarize/prioritize the practice needs. (See HealthInsight’s Practice Goals List)
o Map your workflow and identify inefficiencies that can be addressed through and EHR. (HealthInsight can do this for you, see Process Mapping brochure)
§ Establish a team: which will be responsible for evaluating systems.
§ Include staff in evaluation process: the system has to meet their needs as well.
o Ensure the system selection team includes representatives from all departments/areas.
o Make sure the needs assessment addresses the needs of the entire practice.
o Allow staff opportunities to participate in product demos, especially for products that are on the practice’s short list.
o Give staff an opportunity to weigh in on the final decision.
II. Moving from over 100 Potential Products To About 5
· Examine “review” websites and articles: relating the results to your identified needs/ priorities. Be aware of limitations of each source. HealthInsight can help you make sense of this information. Sources of information include:
o CCHIT certification. Choose from vendors that are certified by the Certification Commission for Healthcare Information Technology. These vendors must have a minimum level of functionality as determined by expert workgroups. Medicare and Medicaid incentives starting in 2011 will require your vendor is CCHIT certified. http://www.cchit.org/choose/ambulatory/08/
o The AC Group Rankings. Twice per year, AC Group publishes a detailed report on vendor PMS/EHR functional, usability, and company viability. This is the most comprehensive evaluation of functionality that we know of. Possible problems with the rankings are inaccuracies in vendor self-reporting, and that vendors have enhanced their functionality since the last report. A free report is available at: See http://www.acgroup.org/2009pmsehrreports.html. HealthInsight is able to provide more detailed information and analyses. Contact your liaison.
o KLAS research rankings. This report is based on user satisfaction results. You can get a free report by rating your current practice management system. http://www.klasresearch.com/Klas/Site/Products/OnlineRatings.aspx
o Ongoing Physician Product Reviews at AAFP (for AAFP members only). The strengths of this report is that it is continually updated, and possible problems are that physicians with strong feelings may be more likely to submit reviews, and these reviews are unstructured and only provide a summary ranking. http://www.centerforhit.org/x290.xml.
III. Conducting a Product Demo
Establish a process: this will allow you to evaluate systems in a more consistent manner, to more effectively make an “apples-to-apples” comparison, and to not be as easily distracted by sales pitches.
§ Develop a standard set of attributes to evaluate and questions to ask. See HealthInsight’s Vendor Evaluation Matrix.
§ Utilize a scorecard or similar mechanism to consistently document your evaluation of vendors and systems based on priorities and system features/functionality. See the bottom section of HealthInsight’s Vendor Evaluation Matrix.
§ Use a patient visit-based scenario: ask vendor to stick to the one test patient for this portion of the demo:
o Multiple chronic conditions, e.g., diabetes and hypertension
o Acute and chronic conditions addressed at the same visit
o Different diagnostic and/or treatment needs
o Utilize an actual patient you have had or use HealthInsight’s example Sample Scenario.
o Steer the demo to tricky situations you may have recently encountered
§ Look at all of your processes: ask vendor to show you processes/tasks that occur outside of the patient visit:
o Prescription refills/renewals
o Lab results follow-up
o Notification letters to patients
o Scheduling, registration and billing functions (if the software includes the PMS functionality)
§ Ask data questions:
o Identify all patients with diabetes who have not been seen in over a year
o Notify all patients on particular medication, e.g., Vioxx
o Find patients with a particular lab result, e.g., patients with LDL-C > 130 mg/dL
o Questions with multiple search parameters, e.g., patients with diabetes who within the past year have had a HbA1C > 9.0%
o What is the average HbA1C value is for a particular provider’s patients
§ Make the vendor show you: “It can do that” is not good enough. Have the vendor show you or point you to a client using a particular feature or function.
IV. Contacting References and Conducting Site Visits
§ Find reference sites: ask your HealthInsight liaison for list of practices that use the top 1-3 products you have narrowed to, and ask the vendors for a list. Note that the references from the vendor will likely be the most satisfied customers.
§ Find those most like you: have your champion call the practices that are most similar to yours (size, specialties, problems and goals). If the clinic is cooperative, consider having other staff talk to their counterparts at the other practices (physician to physician, MA to MA, billing person to billing person). See Outlook Associates’ Vendor Reference Check Tool.
§ Conduct at least one site visit: before conducting a site visit, it will be desirable to narrow to two vendors. If you have a top one vendor that you think is a much better match than your second, conduct a site visit for this vendor first. If the site visit goes well, you may choose not to conduct a site visit for your second choice. Chose practices that are most similar to yours (size, specialties, problems and goals). If you want to stay local, this may be difficult – it is worth traveling to do this!
o Bring practice staff along
o Understand motives of the reference site for hosting visits
§ Conduct a structured site visit:
o Establish overall goals
o Develop a set of questions to ask (Outlook Associates’ Site Visit Evaluation Tool)
o Identify processes and functions to observe
o Spend as much time as possible watching the system in use
HealthInsight prepared this material under a contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services (DHHS). The contents presented do not necessarily reflect CMS policy. PUB# 8SOW-OM-TN-05
Page 2 of 2
[1] Outlook Associates produces a useful guide (Evaluating and Selecting an EMR System) that HealthInsight provides to DOQ-IT participants free of charge. If you want to purchase these directly from Outlook please see the store section at www.outlook-associates.com.