AN ANALYSIS OF THE
“NRF ONLINE”
PROJECT
AUGUST 2006
Rob Drennan
Contents
1 Purpose 2
2 Acknowledgments 2
3 Introduction 2
4 Project Kick-off and Onset of Trouble 2
5 Structure of this Analysis 2
6 General Problems 2
6.1 Immaturity of ICT Industry 2
6.2 Business Analysis 2
6.3 Investment of Own Labour into the Project 2
7 Specific Problems 2
7.1 Lack of Ownership 2
7.2 Lack of Business Capacity 2
7.3 Lack of User Consultation 2
7.4 DCX’s Shortfalls 2
7.5 Selection of Service Provider 2
7.6 The “One-NRF” 2
7.7 Complexity and its Underestimation 2
8 Recommendations 2
8.1 Selection of the Service Provider 2
8.1.1 Realistic Price 2
8.1.2 Knowledge of NRF Business 2
8.2 Pre-project Update of Scope, Plan and Cost 2
8.3 Scope 2
8.4 Rollout and Piloting 2
8.5 Flexibility and Over Design 2
8.6 Consultation 2
8.7 Project Team 2
8.8 Important Points 2
Appendix 1: The Timeline of Actual Events 2
Appendix 2: Check list of items to be considered when embarking upon a customised ICT project 2
List of Figures
Figure 1: Design Philosophy and Progress as at December 2005 2
1 Purpose
This analysis was commissioned to identify the stumbling blocks that impeded the role out of the “NRF Online” project. Therefore, its purpose is to record the learning that arose from the experience for the future benefit of the organisation.
2 Acknowledgments
The author wishes to acknowledge the inputs of the many people, far too many to name, who contributed positively to the project, especially those who contributed to its rescue, and to those who contributed to this analysis.
3 Introduction
On 28 January 2004 the NRF entered into a contractual agreement with an information and communication technology (ICT) service provider, called DataCentrix (DCX), that was selected through a vigorous and competent, but rather rushed tender process to design and build a web-based award (application and evaluation) and grant payment system. This system, that came to be known as the NRF Online system, was meant to, as a minimum:
· Receive electronic applications,
· Manage and facilitate the assessment of these applications using peer review (postal and panel meeting),
· Provide a communication channel to applicants informing them of the outcomes of the assessment process,
· Facilitate the payment of grants, (that is making awards, releases, payments, adjustments of grants and reporting on grant expenses). Consequently, the system would need to be integrated with the existing NRF financial system, called Great Plains.
The design philosophy deployed to achieve these requirements, illustrated in Figure 1, consists of the following elements:
· Two databases: one for award (grant and evaluation) data and the other for document (proposals) storage;
· A web portal allowing system access by internal and external users;
· A work flow engine for directing activities (approvals, etc.) to internal and external users;
· A data viewer for reporting of management sensitive data.
Figure 1: Design Philosophy and Progress as at December 2005
The overarching mandate given to DCX was to design a single portal allowing rapid and simple access to the full range of NRF offerings, including individual evaluation and a number of the NRF granting opportunities. This mandate arose from the strategic intent of lowering the bureaucratic entry barrier to these offerings by consolidating the requirements: The so called one-NRF.
This ICT project, referred to hereafter as the Project, was required to be completed within a year at a cost of R2.181 million. The actual project timeline is presented in Appendix 1.
4 Project Kick-off and Onset of Trouble
The Project was initiated with much excitement emanating mainly from the anticipated benefits offered by a web-based system operating within the Windows environment over the legacy system which was a DOS based database (called Alpha).
Unfortunately, progress slowed soon after project onset. The Project plan had to be changed as a result of the slippage and those elements of the system that were completed were less then ideal: Coding and testing were slow and to exasperate the situation corrections stemming from the testing cycles were deployed in such a manner that previous corrections were re-introduced. Furthermore, improvements identified during testing could not be coded into the system without incurring extra costs as this constituted scope change. However, these improvements should have been part of the initial scope as they were inherent to the normal business practice, but failed to be identified by the business analysis cycle.
Within a year, which was coincidently the expected Project completion date (see Appendix 1), only the application section had been completed for the major granting programmes. In other words, less than a quarter of the Project deliverables had been accomplished. The implication of this delay was that business had to proceed using a hybrid of ICT systems: The NRF Online would be used for applications and to undertake some of the peer review steps, while the legacy system supported the financial part of grant management. This meant that staff were required to make manual data transfers and had to spend more and more time assisting external users to complete their applications. External users were becoming increasingly frustrated with what aught to have been a straightforward process of submitting a grant application. Reviewers were being frustrated by automated workflow messages with attached proposals. Grant and scholarship payments were delayed. For all concerned, the Project had taken the NRF a step backwards in its desired to reduce bureaucracy. The impact was damaging not only to the NRF, but also to the entire National System of Innovation (NSI).
5 Structure of this Analysis
A chronology of the problems that beset the Project is not, however, the subject of this discussion. The focus is placed rather on:
· The distillation of the problems from the hurly burly of normal operations: The key questions to be answered being, “Why and how did the Project go wrong?”, and
· An analysis of the problems: The emphasis here is placed on organizational learning, which will, should the paper succeed in it purpose, assist in avoiding the pitfalls of similar future ICT projects.
The author claims no special knowledge in these matters, but has had a vested interest in the matter ever since the reorganisation of RISA created GMSA and made the NRF Online system integral to the operation of GMSA. Henceforth, the author has spent many hours asking questions of various stakeholders about why they thought the Project was less than successful and what should have be done to avoid these problems. The answers were carefully assimilating into learning points.
The information distilled from these conversations is presented below, firstly as a set of general problems that may affect any or all ICT projects and, secondly a set of specific problems that plagued the Project. Finally, a set of recommendations is presented. These recommendations can be used as a manual by potential project sponsors or project managers during the management of future ICT projects if they wish to avoid the mistakes made on the Project.
To facilitate efficient use of this document the major learning points are highlighted in the margin thus and written in italics.
6 General Problems
By way of introduction to the analysis of the problems that plagued the Project, it is worth noting some generalisations about the ICT industry in South Africa[1].
6.1 Immaturity of ICT Industry
Information and communication technologies are not currently mature, despite the claims of their practitioners. By way of comparison, Civil Engineering in the form of road construction uses mature technology having found it roots in the history of mankind when the nomadic life style was exchanged to an agrarian way of life.
The implication of this assertion is that the management of ICT projects is not an exact process, or at least not as precise as that for road construction, to continue the analogy. The estimation of task duration, choice of specific technologies, deployment of these technologies and end product suitability, during ICT projects, are enacted with significant inherent risk, despite the use of current good project management practice.
The immaturity of the technology and its application influences the quality of work produced. Especially in South Africa, the industry has moved through tremendous turmoil caused by rapid growth, speedy introduction of new technologies, which are typically developed elsewhere and merely sold locally, over and above the challenges placed on the industry by the dot com bubble and Y2K. Consider for a moment the trajectory of one isolated South African ICT company: Idion grew rapidly to become a listed company, was then bought out by to CS Holdings after the CFO was charged with fraud, CS Holdings was later bought by Bytes Technology, and all this in the space of three years.
Inadequate quality in turn leads to frustrated expectations. Here the ICT industry fails dismally because there exists within the industry a pervasive sales culture that says “anything is possible”. Sales people will promise the earth as the individual needs of the customer are spelt out. The consequence of this approach is that these needs may only possibly be met after extensive modification of the off-the-shelf solutions that void guarantees, cost more than expected and reduce the flexibility of the end product.
The learning point drawn from this analysis is that all ICT projects must be regarded as carrying higher than normal risk of running over time, over budget and not delivering as expected. To ameliorate these deficiencies extra protection must be arranged through a number of interventions including using additional skilled and experienced project management, extended timelines with added contingency buffers (of time and money) and a set of realistic, but conservative expectations.
6.2 Business Analysis
This leads conveniently into the consideration of business analysis: A major activity common to most ICT projects. It can be defined simply as the analysis of the existing business processes which gives rise to the construction of a coherent process description. This description is list of sub-process, instructions, rules, controls and approvals all of which govern the workflow through the organisation.
Business analysis is performed by investigating written process descriptions (if they exist) and by interviewing staff who implement the processes.
If skilfully done, business analysis can add value to a business by clarifying processes, identifying dead-ends, uncovering duplicate or unnecessary steps. On the other hand, it can lead to the introduction of unwarranted short cuts and cement into place inelegant processes. The negative outcomes of the business analysis arise when the analyst is unable or unwilling (possibly due to time constraints) to fully comprehend the complexity and subtlety of the existing business process.
Since business analysis is a major component of most ICT projects, the learning point must be to allow more than the anticipated time for this activity. (Remember also that this task does not only involve the time of the external consultant, but also of the internal practitioners. The impact of their involvement must be factored into their normal work loads. This point is discussed further, below.)
6.3 Investment of Own Labour into the Project
Building on the point above, ICT projects are not developed solely by external contractors or even by the business’ own ICT department. The tasks of business description and analysis (see above), which is particularly time consuming in the absence of written work procedures, testing of newly developed code and the support of external users during the release phase all involve internal staff. Those involved must therefore be regarded as a significant project resource. The failure to make allowance for this time will not only impact on the project, but also on the normal business.
The Project timeline shown in Appendix 1 indicates that the construction and deployment of the system for the Focus Area Programmes, Capacity Development Programmes, Evaluation Centre, International Bilateral Programmes, Innovation Fund, THRIP and the Equipment Programme was to be completed within a year (2004). Furthermore, it was to run concurrently with the normal granting and evaluation processes. Hindsight shows that this was almost impossible without causing disruption to the normal operating procedures.
This point is concluded by noting that ICT project managers should plan to cater for the business’ role in the project. Possible solutions are to dedicate people fully to the project and if necessary to hire additional staff on short term contract to cover the additional work load.
7 Specific Problems
Contrasted against this backdrop of general pitfalls that can bedevil the construction of any customised ICT solution are a number of specific problems that impacted on the Project. These specific problems are identified and presented in increasing order of impact. While reading theses specific problems it will become clear that many of them are not freestanding, but are interrelated and additive. This means that, although they are individually reasonably insignificant, together they can and did contribute to the disruption of the Project.
7.1 Lack of Ownership
The ownership referred to here is the hands-on participation of the users of the system in shaping the system to meet their own needs. The users need to influence the design, construction and testing of the solution. This lesson was reinforced by the THRIP experience in building its own ICT platform prior to 2001.
The attitude of many of the managers of the granting programmes was that the responsibility for development belonged entirely to the Project team. By late 2004 it was extremely difficult to get some managers to attend the planning and progress meetings. It must be stated clearly that there were some notable exceptions to this generalisation.
Lack of ownership impacted the Project in a number of ways: Decisions were delayed or rushed; compromises were used in place of bold leadership and the workload of those who did take responsibility rose to unprecedented levels.