BUSINESS CASE STUDY V 11
JUNE 2009
Shane Reti
Henry Feldman
Eli Kaldany
Charles Safran
Introduction
On Friday April 3rd 2009, a confluence of unanticipated events in the Middle East was triggering a crisis in Boston. Several thousand Middle Eastern doctors were approaching a deadline for online medical licensing, and the previously reliable licensing software program was inexplicably failing. Key stakeholder patience had worn out and with the Muslim weekend commencing that day, it was likely that even more users would flood the system. The Chief of the Division in Boston got off the phone having just promised the key stakeholder that the problem would be resolved within twenty-four hours. As he walked into the team meeting he reflected, how had all the simulations been wrong, were all the key stakeholders informed and still supportive, and whether his promise could be fulfilled.
Background
The Division of Clinical Informatics (DCI) was formed in the 1970’s as an academic research division inside a large hospital. Primarily composed of medical doctors with advanced IT skills pursuing a research agenda, in 2007 a new Chief of Division was appointed heralding a new strategic direction. The new Chief brought not only vast organizational and political experience, but also industry experience as a previous owner of a health information technology (HIT) company employing 25 people. Under his direction the division began to build an experienced team of medical doctors with backgrounds from Microsoft, The Boston Consulting Group, hospital administration, project management and the FDA. In April 2008, the division laid a research platform by successfully concluding their first contract negotiations for two substantive HIT contracts in the Middle East. A small software development team was quickly built up under the direction of a physician leader (Chief Information Architect) with substantive HIT experience. With the contracts signed, strategically the division faced future uncertainty over whether it was fundamentally an academic research division, or a commercial entity.
The Middle East Project
The Middle East project involved two substantive contracts, licensing and chronic disease management, both of which were signed on April 7 2008.
Licensing
The Signed Contract
The licensing contract was with a newly formed government regulatory authority responsible for all health professionals within the country. The contract involved developing a software application to license health professionals including doctors, nurses, dentists, pharmacists, and other ancillary health providers. Licensing ascertains that a health professional has verifiable qualifications from accredited institutions, and that they are up to date and appropriately qualified to practice.
The key stakeholder in the Middle East was the government regulatory authority. Decision-making was vested in a person best visualized as the equivalent of a CEO. This man was an experienced clinician and a well-respected official who understood the workings of government agencies, had direct access to government ministers, and long established dealings with the medical fraternity. The CEO had limited experience in software development, and so he relied on his direct report, the implementation director, who had a software and project management background, and who lead the regulatory team ultimately responsible for licensing. When the contract was signed, there was much anticipation that the new licensing environment would raise the quality of medical care to be world class, that expert medical practitioners would be drawn to the developing area, and that citizens would be less inclined to access and spend health care dollars overseas.
Soon after the contract was signed it became apparent that it was non-specific on a number of points. The very first deliverable was in fact a needs analysis that would inform milestones for the rest of the contract. The key stakeholder also decided that they immediately required a fully functional prototype within 6 weeks, developed to current licensing policy understandings (more will be said on the licensing policy environment later). The request for a prototype was partly to allay stakeholder concerns over the ability to actually deliver the system, as well as a desire to demonstrate short term gains to garner further support with officials within the ministry. Significant resource was applied by the division to meet this short notice request which was delivered, on time, and fully functional.
Currently employed medical doctors were the first group to be licensed but the exact number of doctors was largely unknown although thought to be in the thousands. At least two-thirds of the existing legacy register was subsequently found to be invalid, with entries for unidentifiable registrants, buildings, and duplicated entries. Cleaning the baseline legacy data required mostly manual effort by the division, and was very resource intensive. Not only were the absolute numbers unknown, but most of the doctors were foreign graduates with qualifications from overseas, many where English was not the first language, and sometimes where medical schools no longer existed or current postal addresses could not be reliably determined. These qualifications would need to be verified and it was uncertain whether this large resource intensive effort was part of the contract or not.
Value based costing by the division had facilitated a firm fixed fee contract, paid quarterly, and renewable annually. Initially the two contracts (the second to be described shortly) were all part of one contract allowing the division to align funds and resources to which ever contract had the greatest need at any given time. Late in negotiations the contract funder requested the contracts be separated which was reluctantly done, making it more difficult although still possible to cost shift funds and resources. No allowance was made for costing change orders, as the division promoted part of its’ value as an ability to be flexible to stakeholder needs. Travel permission was required and paid for by the client creating some difficulties when the Boston team considered they needed to be onsite in the Middle East and the client disagreed.
Contract Kick Off
Soon after signing it became apparent that planning and scheduling would be difficult because the actual licensing workflow and polices were still being developed by the regulatory authority. Exactly what path a licensing applicant would follow, what requirements and who would verify each step was still to be decided. In pre-contract negotiations the division had differentiated itself from standard IT developers on three main factors, institutional brand name and reputation, agile iterative software development, and input from clinical informaticians. All these factors were now expected to contribute resources in some initially undefined way towards developing policy.
Understanding the environment and the exact needs of the client was a high priority at the start of the project. Issues such as quality assurance and quality control would follow as the project developed, and being a very small team of initially only two or three developers, it was thought that an open-door, intimate development environment would at least see the team through the define, design and early development phases of software production. Everyone acknowledged the absolute need for quality processes, but the greater task initially appeared to be understanding the environment and writing code.
Quarterly reporting against deliverables could also be difficult as they were often framed in relatively non-descriptive and vague terms such as “Detail performance criteria Q1”. Exactly whose performance, in what detail, and how this information would be presented was unclear to all parties, including eventually the author of the terms. It should be noted however that negotiations around these contracts had been noncommittally explored over a number of years without success, and the head of division was of a view that pedantic attention to such detail in the absence of good faith and trust was partly responsible. Accordingly there was some acknowledged acceptance from both parties of just “doing what was necessary” to get the contract signed and then moving on as soon as possible.
Chronic Disease Management
The second contract was a single disease-based, chronic disease management contract worth three times more in revenue than Licensing. This contract committed the division to provide health information technology support for installation of a prestigious US disease-based program.
This project involved a ground up build of a completely new clinic with new equipment and workflows. The division made active contributions to decision making well beyond just informatics support, including recruitment, design layout, and general strategy as well as providing HIT support. The clinic location was moved three times to its final position requiring design and engineering changes each time. Tensions arose with existing disease-based programs run by local clinicians. The chronic disease management contract was drafted similar to the Licensing contract, with non-specific deliverables reflecting uncertainty of stakeholder needs and time frames.
Contract Year 1 April 2008 – Mar 2009
As the two contracts continued to roll out in 2008, the smaller licensing contract slowly started to consume more and more division resources. The main reason for this was the lack of licensing workflow policies at the government agency. Policy development was also hampered by the fact that the regulatory agency itself was restructuring away from the main health ministry, and was urgently seeking skilled staff in a resource-constrained environment. Consistent with the aim of world-class standards, and pressing for time, the division ended up directly incorporated many US based licensing policies. The net effect was that licensing policy was being written and signed off by the Middle East team, at the same time as it was being incorporated into software development. This design-and-build methodology resulted in numerous software changes often with deep and complex linkages. Opportunities for quality assurance and testing were reduced by constant workflow changes.
Midway through 2008, the regulatory agency advanced the licensing time frame by 6 months from that which had been originally agreed. The licensing of medical doctors was now to be complete by April 1 2009. The division conceded to this request because initial marketing had highlighted a rapid application and development environment, the deliverables were non-specific and open to interpretation, and the division felt like it had to prove itself in year one of the contract.
Advancing the development time frame by 6 months disrupted division resourcing, and incomplete workflow policies and resource constraints at the regulatory agency continued to be a significant rate-limiting hurdle. To facilitate progress, the division went to extraordinary lengths to make sure milestones were reached. For example, one of the first steps to online licensing involved sending a printed copy of a username and password to the last documented postal address of all medical doctors. The division generated the mail merge file and transmitted it to the agency, however the agency neither had the equipment or the workforce to physically action the mail out. Consequently the division ended up printing roughly 5,000 letters, and the Chief Information Architect, Lead Technical Specialist, Chief of Division and various family members folded every letter and inserted and sealed them into envelopes for overseas shipping to the agency.
The initial response to the increasingly resource intensive licensing contract was to transfer resources from the chronic disease management contract to licensing. This was only partially successful however because while the disease management program was significantly delayed in clinic construction, thus freeing up resources, the division undertook a self-initiated, major strategic change in direction for the program. The initial foundation to the chronic disease management contract was envisaged as a simple “lite” versioned medical record for the new clinic. In several years time this would eventually migrate to a full commercial package with appropriate future proofing in place. Midway through 2008 it was apparent the “lite” medical record was developing more and more complex features, and it was decided to change its nature to a fully featured electronic medical record that would take a competitive position in the marketplace. The justification for this was that it had changed so much from initial design and had taken so much resource by this point, that it might as well become fully featured, and in so doing provide better clinical support to the new clinic when it opened, and enhanced data capture for external reporting and internal research purposes. Much of the resources potentially freed by delays in constructing the new clinic were then absorbed by this new strategy.
While the licensing contract was being squeezed, another and unexpected group of stakeholders started to influence the project. There was a growing groundswell of discontent amongst the doctors being licensed as they reluctantly approached the April 1 2009 deadline. Every country has political stakeholders and the Middle East was no different. The Middle East environment was one where many of the doctors had direct family and personal connections to significant political figures, and this sometimes influenced licensing workflow and requirements, which in turn affected the software development timetable. Religious holidays, vacations, time zones, translation and Muslim weekends over Fridays and Saturdays reduced communication opportunities and also altered productivity schedules.
Leading up to April 2009, the end of contract year one, the licensing schedule had been advanced, the licensing application rapidly deployed, and applicants could start registering before an April 1 application deadline. April was an important month for other reasons as pending satisfactory progress, both contracts would then be renewed for a further two years.
International backdrop to the April 1st 2009 deadline
From late 2008 through early 2009, the international economic crisis was in full flight. Middle Eastern countries were increasingly price sensitive in response to a 75% drop in crude oil. Many health contracts in the Middle East were terminated on the basis of poor economic outlook with mutual dissatisfaction amongst all parties. Tensions were high in the Gaza strip with interest in US positioning on these issues. In December 2008, President Bush was involved in a “shoe throwing” incident in Iraq, and on Jan 20 2009 a new US president was inaugurated. Very clearly the division was US-based and this created a potential flash point and focus around an activity such as licensing that an influential group such as medical doctors did not want to undertake anyway.
Licensing Fails on April 1st2009
The deadline to license all medical doctors was April 1st 2009. In the week leading up to the licensing deadline, increasing numbers of doctors had been accessing the online software. Where previously 30-40 doctors were online at any one time, this had increased to 150. The division had previously run limited simulations using industry standard simulation tools. These simulations showed load capacity of up to 400 doctors, however from Sunday March 29th onwards, the software inexplicably started crashing up to 15 times per day at variable loads above 40 users. The development team could temporarily correct the problem by manually resetting the software, only until the load increased again an hour or two later. With each crash, unsaved work was lost. The Middle East implementation director was being deluged with disgruntled doctors now able to focus their anger against the licensing process via the failing software. With contract renewals up for discussion that week, the timing could not of been more unfortunate.
On Monday March 30th the development team started a crises management plan that involved rechecking all code, increasing client communication and progress reports, construction of software patches and work arounds, and reworking monitoring and simulations. The Middle East implementation director elected to extend the licensing deadline by one week to April 8. From the Monday to the Thursday the development team worked twenty four hours a day to find a solution, however only minimal progress was made, the root problem remained and crashes continued at a rate of up to 8 times per day.
By Friday April 3rd the Middle East implementation director was indicating that political reserve and patience had nearly been exhausted. The Boston development team were taking turns to be up several times per night to reset the software and were now tiring. The development team physician leader (Chief Information Architect) was in constant email and conference call contact from London where he was presenting a paper that day, the next two years of contract were renewable in 4 days time, and a resolution had been promised within 24 hours.