How to deliver successful IT Projects
Risk Identification in the Implementation and Roll out of Large Systems
1. Introduction
A few years ago, the E-Strategy Board had oversight of a number of IT projects involving large, complex systems with large and complex user requirements and data sets. There have been continuing reports in the national press of significant problems with systems such as these in commercial organisations and in other Universities. It is estimated that 50% - some say 80% - of large IT projects fail.
http://www.computerworld.com/managementtopics/management/project/story/0,10801,99488,00.html
“Billions of pounds are wasted every year on new IT systems, according to a report published by the Royal Academy of Engineering and BCS”.
http://www.bcs.org/server.php?show=conWebDoc.1761
"The UK public sector alone has spent an estimated £12.4 billion on software in the last year and the overall UK spend on IT is projected to be a monumental £22.6 billion," says Basil Butler, Chairman of the working group that produced the report. "We looked at a range of studies showing that only around 16 per cent of IT projects can be considered truly successful."
The Press Release for this report goes on to explain: “One of the problems with cutting-edge software is that it is often hard to visualise what the system will do. I wouldn't ask an engineer to build a 1,000 metre long concrete beam suspended at one end because I know it can't be done - I have a physical perspective on it," one respondent told the working group. With software it's never like that. We don't have any underlying feel for whether something is even feasible. It is a cardinal mistake to select suppliers for a complex IT project on the basis of price alone, since it is very difficult for suppliers to accurately predict costs at the outset. If a customer is asking for something unrealistic or ultra-high risk, the supplier should tell the customer. ‘Projects are often poorly defined, codes of practice are frequently ignored and there is a woeful inability to learn from past experience,’ says Professor McDermid (University of York). ‘The role of systems architects is critical - their job is to translate a business vision into a technical blueprint. They often hold the keys to success in complex IT projects but they are in very short supply. The UK could benefit enormously from exploring ways to identify and support people with these unique skills’."
http://www.bcs.org/server.php?show=conWebDoc.1761
This paper sets down some initial considerations on this matter and the steps that the University and the Project Groups (and their Boards) should take into account in the monitoring of their projects to ensure early alerts to potential issues that may arise during the course of the work, and which contribute to increased risk. Risk is present in all situations and in all projects – and it needs to be identified and managed.
2. Large Systems
2.1 Vision and Mission of the Organisation
One of the most important factors in project success is alignment with the vision and objectives of the organisation. If there is full alignment it is easier to achieve full buy-in from all levels of the organisation. Getting and maintaining support is a key success factor. Without buy-in, and sustained levels of buy-in during the lifetime of a project, the chances of success decrease dramatically.
2.2 Procurement and Implementation
Procuring and implementing large IT systems within budget, and within a planned timescale, is known to be difficult and challenging. In addition, the situation can be further complicated by incomplete or inadequately defined user requirements, or user requirements that subsequently change over time and the implementers of the system are faced with what is known as a “moving target”. Further aspects to consider are user interfaces, usability of the system, and the training required to enable existing users and new users to be able to successfully use the system to process their work. In addition, some of the existing work practices may need to be modified, or improved, in order to get the most out of the new system. Often the latter are not straightforward to progress as they involve social and cultural issues, and other factors that may be outside the direct control, or even influence, of those who are supplying the system, or even using it.
2.3 Communications
Communication about the new system to users and stakeholders is vitally important and is often underestimated. System providers need to ensure that user requirements and user views are fully taken note of and acted upon. When any new system, or upgrade of an existing system, is to be released, users need to be consulted well in advance to ensure that it is fit for purpose and there are no surprises to the users.
Internal communications within the project team are also an important matter. System implementers may be aware of potential delays in the pipeline well before the project manager, but may be reluctant to voice them in team meetings for fear of being held responsible for these, or of bringing bad news to meetings when the culture of the meeting is to hear good news so that staff leave the meeting motivated. Bad news is associated with demotivation. It is sometimes hoped that these problems will “go away”. Usually they do not - because they have not been addressed. Thus the culture of the working environment in which the implementation is being done can play a key role in the outcome. As many systems staff perceive these as ‘softer’ matters compared with technical ones, they can often be undervalued, or even ignored. Requests for additional resource to solve problems can be used to obscure poor performance of staff involved in the project. Conversely, staff who are able to deliver may come under pressure to take on more work to ensure the project keeps to deadlines. They may be reluctant to voice their feelings about the increased pressure this places them under for fear of being perceived by managers as ‘unable to cope’, leading to inability to progress in the organisation.
2.4 Ownership and Accountability
The project plan needs to clearly set out the lines of responsibility and accountability for the various areas of the project and the relationships between them. Even when this is clearly defined, problems can arise when issues are not identified, or are identified but not communicated (as noted in section 2.3). A further problem can arise when a team member identifies an issue, refers it up, but it is not accepted by the manager as an issue because they in turn wish to distance themselves from it, and so nothing is done about it. Team members may stop coming to meetings because they find other things to do, or they take sick leave. Replacements can be difficult to find at short notice and may not be possible because of budget constraints. Once a team is in having problems, delays often follow and it becomes more difficult, if not impossible, to get the project back on track.
It is a common problem in Universities and the public sector for staff at many levels in the organisation to accept responsibility for an area of work, but not to accept accountability for it, or for any issues that arise within the area. The objective is to ensure that when the music stops and an investigation starts, the monkey is on someone else’s shoulder6. This is a cultural issue which is difficult to change. It can contribute substantially to project failure.
2.5 Data Sets
Automation of existing manual processes, or migration of processing from a local system to a central system, often uncover so-called “hidden issues” that were not apparent at the time when the original user requirements were specified. Local systems could have been producing the correct results but using some non-standard short-cuts, or assumptions about the data, that are not as easy to replicate in the centralised system. When such data is transferred to the central system it can result in errors or system failures which are then ‘blamed’ on the new system. In reality the fault is in one or more of three areas – the data specification, the data context, and the data processing. This can be time-consuming to unravel.
2.6 Testing and Timescales
It is essential that any new facilities are implemented and tested at an early stage in order to allow adequate time for detailed review by the users, and for training schedules to be drawn up and defined. A common failure with large systems is to underestimate the time required to complete the implementation, resulting in having to rush into processing live data, which then causes errors and loss of user confidence in the system. In addition, there can be further problems with data such as that outlined in section 2.5 where there are some hidden issues which do not come to light until the data is processed in a live situation. Beta testing on trial data often does not uncover these issues. Thus when a new system is rolled out into user service it can produce errors for very different kinds of reasons. However, as far as the user is concerned they are all classed as “deficiencies of the new system”, so considerable efforts in training, support, PR, and mediation are required in order to accomplish a successful migration from the old way of processing to the new way of processing, and for the users to feel that the new system is successful. Often they prefer the old way of doing things because there were no problems with it, and the new system is perceived as generating extra work and introducing problems and errors that did not exist before. It is common for the user support and training required to be underestimated and for there to be low “buy-in” from users. There can be increasing tension and even polarisation between the providers of the new system and the users as it requires considerable effort on both sides, as well as possibly disturbing existing work practices. It is common for the roll-out of a new system, or an upgrade of an existing system, to fail to recognise the seriousness of many of these issues and to underestimate the time needed to address them by several orders of magnitude. When the difficulties become apparent over time a mutual blame situation can develop. Then support staff and users may seek to distance themselves from the system for fear of encountering further difficulties, and attracting further criticism.
2.7 Centralised Systems
A further issue arises with the use of centralised systems in a University where devolved resourcing is in operation. Schools need to feel in control of their budgets and their destiny, and therefore need to feel in control of their data and its processing. The University therefore needs to ensure that these requirements can be met in full – at least as efficiently as a School could implement them, and preferably more efficiently - and with added value information that is useful to the School.
A number of organisations are moving to ‘customer-centred design’ in order to achieve greater user satisfaction. Services like Flickr, Basecamp, Amazon and others ensure this is designed in, i.e., in order to attract new customers and retain current users it is necessary to provide features and functionality that customers want. http://davidcrow.ca/article/675/why-projects-fail
In addition, the value proposition of the new system needs to be clearly understood by management, stakeholders, and users. What is the return on investment, and is this understood by all parties?
3. How to ensure successful IT Projects
3.1 Project Success Factors
Effective User Input / 12.8%Complete Requirements / 12.3%
Not Changing Requirements / 11.8%
Executive Support / 7.5%
Technology competence / 7.0%
Sufficient Resources / 6.4%
Realistic Expectations / 5.9%
Clear Objectives / 5.3%
Realistic Time Frames / 4.3%
Use of known Technology / 3.7%
Other / 13.9%
Source: Standish Group
3.2 The Challenges of Complex IT projects
The Royal Academy of Engineering/BCS investigation into the challenges of complex IT projects produced a report4 which identified the following key success factors –
· Client-supplier relationship
· Contractual arrangements
· Evolutionary project management
· Requirements management
· Change management
· Measuring progress
· Risk management
· Technical issues
4. Metrics for Risk Identification
Metrics for assessing the risk associated with software projects can include a review of the following areas and giving each one a score –
i. User requirements
ii. User buy-in
iii. Schedule and time lines for the completion of project components
iv. Allocation of programming and system tasks to those with the right skills
v. Resignations, absences, and sick leave of staff
vi. Changes in staff allocations to the work to be done
vii. Beta testing
viii. Completion of system and user documentation
ix. Reports, quality, and identification of issues
x. Internal and external dependencies, e.g. waiting for an external supplier to supply a module for the system, or update to an existing module to provide the functionality required
The Bradford Project Management Method1 (based on PRINCE 2) outlines how to address risks and evaluate them (pp 31-33). It also summarises how to identify issues (p34). The Project should also have a Senior Customer (p 47) and a Senior Supplier (p 48). The Senior Customer is responsible for the specification of the needs of all those who will use the final product(s), for customer liaison with the project team, and for monitoring that the solution will meet these needs within the constraints of the Business Case in terms of quality, functionality, and ease of use. The role represents the interests of all those who will use the final product(s) of the project, those for whom the product will achieve an objective, or those who will use the product to deliver benefits.