Mixing Multiple Models: Improving Operations & Development

Mixing Multiple Models: Improving Operations & Development

By:

JillPritchet and FrancisKinsler

Software Measurement Services (SMS)

Introduction

Most organisations have to consider production issues alongside the development of new products. The efficiency and profitability of day-to-day operations are arguably more important than the prospect of some new product that will not be available for some months or years. Organisations need to survive in the short term as well. However, models such as the CMMI®are concerned primarily with improving the development of new products. They fail to capture the imagination of many business managers because they lack immediate applicability to the issues that concern business operations on a daily basis.

This can lead some managers to misunderstand the relevance of one model over another, and to see the issue from the perspective of the need to choose between the CMMI® and some other model, such as ITIL or CoBIT. There can also be an apparent contention between software development life cycle models, such as the Rational Unified Process (RUP) and CMMI®.

This paper discusses cases where multiple models have been combined to address a wider scope than that covered by any single model on its own. The aim is to show, based on pragmatic examples, how an organisation’s entire end-to-end value stream(s) can be improved in a systematic and efficient way without the exercise degenerating into a series of ‘method wars’.

All the models mentioned above are included in the discussion, which is presented in a business-oriented fashion that assumes that most managers are seeking improvements in efficiency, effectiveness, customer satisfaction, quality and profitability.

Organisations must consider production alongside development of new products. The profitability of day-to-day operations is just as important. Without survival, new development is worthless.

Our way of thinking

When starting out on the path of improvement our belief is that there are 3 key elements which can work together to equip an organisation to compete in this technological age. They are the use and application of Governance, Model(s) and Method(s). Together these 3 offer a framework that when applied correctly can ensure success.

However, although Management in an organisation may have heard about them,problems begin to surface largely due to a lack of understanding about how they apply to their specific situations. In addition, jargon can cloud the situation and this can lead to failure.

As the diagram shows there is overlap between them and although there may be similar requirements they are not always a direct match. So how does an Organisation know what approach they should be taking and how can they overcome the problems associated with implementing multiple approaches concurrently?

Gain a Common Understanding

The first step is one of level of understanding, in our experience many problems start purely due to a lack of common understanding of each term. The following table highlights the core details.

Term / Example / Description / Purpose
IT Governance / COBIT / Governance over information technology and its processes with the business goal of adding value, while balancing risk versus return. / To provide management and business process owners with an approach to IT Governance that helps in understanding and managing the risks associated with IT.
It ensures the integrity of both information and information systems.
Model / CMMI®, SW-CMM, EIA, SPICE, ISO; / A model is a simplified representation of the real world. They contain essential elements of effective processes for one or more bodies of knowledge. However, they are not processes or process descriptions themselves. / To provide guidance on what has to be done in order to satisfy the policies of the organisation, but they do not go to the extent of how the policies should be implemented.
Method / ITIL, RUP, SSADM, Prince, Value Stream Mapping / A method determines how things should be done. / To provide detailed information so that different people applying the same method do so in a consistent manner

Table 1 - Understanding the terms

But how do these become converted into practical application? The pyramid below illustrates the relationship between these cornerstones.

Figure 2 – Relationship between Policy, Process and Procedure

To Show How Multiple Models Can Be Combined To Address A Wider Scope

With the dawn of globalisation and a worldwide financial market, the Financial Services Authority (FSA) is recommending that governance be taken seriously within the financial institutions, such as the banks and clearing houses. Illustrated in the organisation chart below is the importance of choosing the best fit for a particular organisation. In the example, the IT part of the organisation has chosen to use CoBIT for the governance aspects, a mix ofCMMI®, RUP and ISO 9001:2000 within development and ITIL in the production area. This offers a greater coverage of the operation as a whole.

Figure 3 – Organisation chart showing how governance is applied in an organisation

And There Is Overlap Between The Models

Forexample, there is overlap between Configuration Management in CMMI® and ITIL. It does not mean that you have to repeat things or do them differently. It is about ensuring that what is implemented will satisfy all elements.

In the following part of this report we focus on one example to illustrate this selection of multiple models and how it was used with very encouraging results.

Case Study

The purpose of this case study is to help IT organizations understand how to integrate and use more than one SPI model e.g. ITIL, CoBIT, and CMMI® when improving the performance of their operations.

Using a case study from a client we have recently worked with we will explain why the decisions were taken to use more than one model and what can be learnt from the experience.

Which Frameworks and Models were involved?

Model / Acronym / Purpose / Clients area
Capability Maturity Model Integrated / CMMI® / Provides guidance to use when improving application development processes for software development / Development
IT Infrastructure Library / ITIL / Defines how Service Management is applied within specific organisations. / Production
Control Objectives for Information and related technology / CoBIT / A framework of good practice for control over information, IT and related risks. IT Governance and audit. / Development and Production
Rational Unified Process / RUP / A software development methodology based on UML, RUP organizes the development of software into four phases, each consisting of one or more executable iterations of the software at that stage of development. / Parts of Development
Value Stream Mapping / VSM / A tool that helps you to see and understand the flow of material and information as a product or service makes its way through the value stream. / Development

Table 2 – The Frameworks and models used in the Case Study

Description of the Case Study

The client is a large financial institution in a city in the UK with 350 staff in the IT department. They were implementing a number of improvement initiatives internally and we were called in to help them to achieve CMMI® maturity level 2 in their development area.

The Approach

Initially a CMMI® assessment was carried out in both the development and production areas to identify the key gaps against the CMMI® model. The resulting report was then used to prioritise the planned improvements.

Starting Scenario

The client was convinced that they were starting from a position where there were a few existing practices at Level 3 CMMI® and many more at Level 2 CMMI®. Reality was very different. During out initial CMMI®assessment it became clear that the existing practices, although defined in an existing quality system, were largely not being followed by staff, due to them being difficult to access in long, ‘wordy’ documents. A level of autonomy had developed over time, in different projects and the staff effectively did their own thing.

Quality Assurance had been present historically but in recent months had been passed to the projects and a percentage of project costs were being allocated to this activity. Although the budget was being taken for the QA activities they were often not being done. There was no independent audit carried out and as a result no action being taken where non-conformance with standards persisted.

The company knew there were problems but not how deep rooted they were. Our assessment showed that they were demonstrating many of the characteristics of a Maturity level 1 organisation (See table 1). There was a lot of work to be done.

CMMI® MATURITY LEVEL 1
KEY CHARACTERISTICS
Chaotic
Dependent on heroes
Changing requirements
Poor cost estimation and project planning
Lax change control
Lack of control over subcontractors

Table 3 – CMMI® Maturity level 1- Key characteristics

Key problems

• The organisation was too focused on flexibility with everyone doing their own thing;
• Lack of understanding at basic level e.g. 13 different change processes;
• Tried to please everyone and ended up pleasing no-one;
• Ended up with processes that were not being followed or maintained;
• Lack of management control;
• High turnover of staff in a ‘hire and fire’ culture;
• Ever more reliance on contract staff (approaching 50%), so no expertise was retained and much time was lost retraining new personnel;
• The planned deadline for CMMI® Level 2 compliance was ambitious.

Implementation of the improvement action

A number of parallel work packages were set up each with a mentor from our team working alongside each of the clients Technical Working Groups, which consisted of a Champion and TWG members who worked in the area being improved. There was a sponsor overseeing all the Work Packages who was a member of the senior management team.

Figure 4 - Organisation Chart showing the Technical Working Groups

At the clients request these work packages were initially focused on the Goals and Practices for each of the Process Areas of the CMMI® model. This was a significant problem for us as we knew that compartmentalising the organisation as if it could be organised around ‘arbitrary’ process areas rather than the ‘end to end network of business processes’ was not going to help the client to understand what their true value streams were. As a result the project was unlikely to provide the real benefits the project.

In the following example we will explain how our approach moved the client from their original direction by focusing on the work carried out by one of the TWG’s namely Change Management. This process area formed part of the Configuration Management process area and it encompassed all areas of the business.

The Change Management Process

The initial analysis (using a CMMI® assessment) of change management highlighted more than 13 different change processes being used in different parts of the operation e.g. projects, requirements, QMS, testing, environment, production etc. In some of these areas a process had been written and associated documentation e.g. documents, forms etc. had been produced.

Although some work had been carried out internally to move towards a standard single change process it had never been successfully implemented. In addition, they were unable to analyse the volume of changes across the operation as it was a totally manual process and very time consuming. This meant we were faced with not only process overload but the client was finding it hard to understand how ineffective this current approach was across the operation or the scale of the problem.

Using their existing draft process as our starting point our discussions began. It quickly became apparent that many of the stakeholders held the belief that a single change process was not suitable and that each areas needs were fundamentally different. This was a large part of the failure to move to a single change process.

During these discussions using their process model a number of issues became visible:

• everyone wanted to retain their own process and were finding it hard to adapt
• they were all focused on their own area and none of them considered any affect they might have on later stages of development as either important or their responsibility
• they did not understand how an effective process should work
• they had no consideration for the customer particularly the notion of an internal customer in the process
• there was a lack of visible management commitment to moving to a single process, it was being seen as an unnecessary overhead
• the organisation had not considered how the process was to be resourced. There was no provision made for WHO would do WHAT, WHEN or HOW they ONLY had a theoretical process description and there was no defined mechanism to make it effective operationally.

In spite of this there was willingness by some stakeholders to listen and it became clear that a few of them had become more aware of the issues associated with lack of control of change but did not have the training to know how to deal with the problem.

Looking at the problem in a different way

Change Management across the operation could be implemented using a number of models as illustrated below (ITIL, CMMI®, CoBIT) but the client had limited knowledge of them and we needed to persuade them to move from their existing approach?.

Figure 5 - Map illustrating where Change Management fits in the 3 approaches

There was a need to come up with a fresh way of looking at their problem and this needed to be in a more visual way to aid understanding of the stakeholders involved. There was no point in reinventing the process as they had tried to do this and the initiative had been seen as unsuccessful. They were unlikely to buy in to the same approach again. Something visual that would build confidence was required.

Value Stream Mapping

Using their existing process we produced a value stream map of their ‘current state[1]’. This was the first time they had seen the 13 processes they had written in an end-to-end way and what it demonstrated was anunderlying process that had a cycle time[2] in the region of 40 hours per change with an elapsed time in some cases of several weeks. There was potential for large scale improvement.

The creation of a ‘future state map’ using Value Stream Mapping forced the organisation to decide how the process was to be operated including allocation of responsibilities and resources for EXECUTING the process.

In addition a number of key activities which would normally form part of change management were missing including acceptance testing and audit trail. Until the map was produced they were unaware of these gaps.

Level of improvement

A second value stream map of the proposed ‘future state’ was produced and this is where the specific improvement areas were demonstrated. The exercise gave the client a different perspective on their problem and for the first time a realisation of the scale of their inefficiency when managing changes. The following areas for improvement were identified:

• The role of the Change Control Boards (CCB) could be enhanced to enable them to prioritise complex changes at their initial review rather than meeting on a second occasion saving meeting time.
• Introduction of a ‘supermarket[3]’ where changes are ‘stacked’ to be dealt with based on the priority set by the CCB rather than automatically being passed downstream and being actioned in no particular order.
• Impact analysis rather than being carried out on an ‘as required’ basis is done to a defined capacity of X changes per week making managing of scheduled changes more efficient.
• A single tool was proposed for all types of change requests to be logged the implementation of this would enable automatic updates on the Change Request history log which currently was a manual task.
• Efficiency of the CCB could be further improved if there was a single CCB for the operation rather than individual ones for each area
• Introduction of acceptance testing for changes would ensure that only effective changes were being implemented
• Introduction of a proper audit trail for all change requests would introduce clear ownership and responsibility for change management

The results

The ‘future state map’ demonstrated a proposed process that had a cycle time[4] in the same region as the current state map (40 hours per change) however, the elapsed time reduced from several weeks to approx 10 days per change. In addition, it was estimated that the cycle time for fast track changes (to a defined set of criteria) could be handled in a 6 hour cycle time and an elapsed time of approx. 1 day.