Need for Project Management Metrics (A)1

WEEKLY EXECUTIVE STAFF MEETING

Everybody knew that this would be a very unpleasant meeting. The selling price of the company’s stock was near a five-year low. The company was just downgraded by one of the rating agencies. Several Wall Street analysts wanted more information on what new projects the company was working on and the potential strength of the R&D projects in the company’s pipeline. And to make matters worse, the company was forced to slash the company’s dividend payment to conserve cash.

Unlike other companies that could produce new products quickly and deliver them to the marketplace with minimal cost, this company struggled. Historically, the company was more of a follower than a leader. The company had problems when it came to creating value through innovation processes. In the past, when- ever new products were created, the company was good at adding value to exist- ing products using process reengineering, product modifications and enhancements, quality initiatives, and business process improvements. But this alone would not get the company through the current turbulent economic times. Innovation was a necessity and needed to happen quickly. The creation of customer-recognized value was needed and the company was struggling.

411

412THE NEED FOR PROJECT MANAGEMENT METRICS (A)

As Al Grey, President and Chief Executive Officer (CEO), entered the room, everyone could see that he was not happy. The company was in trouble and nobody really had a plan for how to rectify the situation. Finger-pointing and the laying of blame elsewhere had become the norm. The company had a multitude of talented people, but their achievements were less than par. This was particu- larly true of the R&D Group and the Engineering Departments responsible for the development of new products.

Al Grey stood up and addressed the senior staff:

I believe that I have identified the root cause of our innovation problems and we should be able to come up with a solution. I’m passing out to each of you two sheets of paper. In the first sheet (see Exhibit I), I have identified seven R&D projects that we considered to be total failures. You’ll notice that five of the seven projects that failed consumed some of our most talented people and yet there were no new products developed from these five projects or from the other two projects. You’ll also notice in which life cycle phase we made the decision to pull the plug on these projects. Several of these pro- jects had gone through to completion before we discovered, or should I say were willing to admit, that the projects would produce no fruitful results.

Over the past year, we worked on a total of twelve high-priority R&D pro- jects where we were convinced that success would be forthcoming. Only five of these projects generated any revenue stream, and none of the five were con- sidered as “home runs” producing the desired cash flow. In the second sheet of paper (see Exhibit II), I have shown how much money we have squandered on the seven projects that failed. We threw away millions of dollars.

Several of you have pointed the finger at the project management office (PMO) and tried to put the entire blame on their shoulders claiming that the process we use for our portfolio selection of projects was at fault. While it is true that perhaps our process, like any other process, could be improved, it was still all of us in this room that agreed with their selection recommen- dations. Not all projects that we recommend will be successful and any executive in this room that believes that they will all be a success is a fool.

Exhibit I.Failure identification per life-cycle phase

Strong

Resources

AverageWeak

#1 #2 #3

#4 #5

Life Cycle Phases

Weekly Executive Staff Meeting

413

Exhibit II.

Project

A B C D E F G

R&D termination costs and reasons for failure

Total $28,531,700

Original Budget

$2,200,000 $3,125,000 $2,200,000 $5,680,000 $4,900,000 $4,100,000 $6,326,700

Expenditure at Termination

$1,150,000 $3,000,000 $1,735,000 $5,600,000 $3,530,000 $3,200,000 $6,200,000

$24,415,000

Reason for Failure

Objective too optimistic Could not make breakthrough Structural integrity test failure Vendors could not perform Product safety test failures Specification limits unreachable Could not make breakthrough

Some projects will fail, but with a failure rate of seven projects out of twelve, the company’s growth could be limited.

Everybody in the room reviewed the two sheets. There was dead silence. Nobody wanted to speak. Using the PMO as the scapegoat would no longer work. There was a feeling that Al Grey was about to blame someone for the calamity but nobody knew who it might be.

Al Grey continued:

Given the reality that some projects will fail, why must we squander so much money by waiting until we get to the last two life cycle phases of our five phase methodology before we are willing to admit that the project might fail? Why can’t one of the project managers stand up in any of the earlier phases and state that the project should be cancelled?

Everybody then looked at Doug Wilson, Vice President for Engineering and R&D, for his response to the question. Even though a PMO existed, the majority of the project managers were engineers who reported directly to Doug Wilson. The project managers were “solid” to Engineering but “dotted” to the PMO just for project status reporting.

“I’m going to defend my people,” said Doug. “They work hard and have a history of producing results, profitable results at that! I know that some of our more recent projects have come in late, have been over budget, and the results have not been there. Our projects were challenging and sometimes this happens. It is uncalled for to blame my people for these seven failures.”

Ann Hawthorne, Vice President for Marketing, decided to intervene:

I have worked with engineers for decades. They are highly optimistic and believe that whatever plan they develop will work correctly the first time. They refuse to admit that projects have budgets and schedules. The goal of every engineer that I have ever worked with wants to exceed the specifications

414THE NEED FOR PROJECT MANAGEMENT METRICS (A)

rather than just meet them, and they want to do it on someone else’s budget. When you perform R&D on government contracts, you can always request more money and a schedule extension, and you’ll probably get it from the government. But our engineers are spending our money, not funds provided from an external source. It’s hard for me to believe that they couldn’t iden- tify early on that some of these projects should have been cancelled.

Everyone looked at each other wondering who would be next to be blamed for the problems. Al Grey then spoke out again:

Whenever I review one of our project status reports, all I see is information on budgets and schedules. The rest of the information is obscure or hidden and sometimes makes no sense to me. Sometimes, there are comments about risks. Why isn’t it possible to establish some metrics, other than time or cost metrics, which can provide us with meaningful information such that we can make informed decisions in some of the earlier life cycle phases? I see this as being the critical issue that must be resolved quickly.

After a brief discussion, everyone seemed to agree that better metrics could alleviate some of the problems. But getting agreement on the identification of the problem was a lot easier than finding a solution. New issues on how to perform metrics management would now be surfacing and most of the people in the room had limited experience with metrics management.

The company had a PMO that reported to Carol Daniels, Chief Information Officer. The PMO was created for several reasons, including the development of an enterprise project management methodology, support for the senior staff in the project portfolio selection process, and creation of executive-level dashboards that would provide information on the performance of the strategic plan. Carol Daniels then commented:

Our PMO has expertise with metrics, but business-based metrics rather than project-based metrics. Our dashboards contain information on financial met- rics such as profitability, market share, number of new customers, percent- age of our business that is repeat business, customer satisfaction, quality survey results and so forth. I will ask the PMO to take the lead in this, but I honestly have no clue how long it will take or the complexities with design- ing project-based metrics.

Everyone seemed relieved and pleased that Carol Daniels would take the lead role in establishing a project-based metric system. But there were still concerns and issues that needed to be addressed and it was certainly possible that this solu- tion could not be achieved even after significant time and effort would be expended on metric management.

Al Grey then stated that he wanted another meeting with the executive staff scheduled in a few days where the only item up for discussion would be the plan

Questions415

for developing the metrics. Everyone in the room was given the same action item in preparation for the next meeting: “Prepare a list of what metrics you feel are necessary for early-on informed project decision-making and what potential problems we must address in order to accomplish this.”

QUESTIONS

In answering these questions, do not look for the perfect answer. This problem is quite common today and plaguing executives, stakeholders, and other decision- makers involved in projects. There may be several answers to each question based upon your interpretation of the situation.

1. Can the failure of R&D actually be this devastating to a company? 2. When project work goes bad and failures occur, is it common practice for finger-pointing and the laying of blame to occur, even at the executive levels?

3. What information is found in Exhibit I? 4. What information is lacking in Exhibit I? 5. Is it possible that highly talented resources can overthink an R&D project to

the point where they look for the most complicated solution rather than the

simplest solution? 6. Is it good or bad to have five R&D projects out of twelve completed

successfully? 7. Can a PMO prevent failure? 8. Is it inherently dangerous to encourage a project manager to recommend

that his or her project be terminated during early life-cycle phases? 9. Who, if anyone, should be blamed for the failure of the projects in this

case study? 10. Is Ann Hawthorne’s description of engineers realistic? 11. Would you agree with Al Grey that the real cause of the failures appears to

be a lack of good metrics? If this is the cause, then how do you justify that

other projects were successful? 12. Should the PMO take the lead in the establishment of the metrics? 13. Are a few days enough time for the follow-on meeting and should

(Kerzner 411-415)

Kerzner, Harold. Project Management Case Studies, 4th Edition. John Wiley & Sons P&T, 08/2012. VitalBook file.

for Project Management Metrics (B)2

The weekly executive staff meeting concluded and everyone felt confident that the company was now heading in the right direction. Al Grey sent out a compa- nywide e-mail letting everyone know what was about to happen and that the com- pany needed everyone’s cooperation to make this metrics management initiative succeed. Al Grey stated:

As you all know, today’s business environment is changing rapidly. We can no longer rely solely upon our existing product lines for continuous growth. In the past, we have captured best practices and lesson learned, and this has improved the efficiency and effectiveness of our operations which then added to profits. Unfortunately, the best practices and lessons learned that we captured did not directly provide benefits to our innovation processes.

Because we are now in a dynamic rather than stable business environ- ment, we must rely heavily upon the creation of new products to achieve sustained growth. Our customers are demanding new products with higher quality and at a lower cost. Customers are now looking at how our products provided value to them and sometimes the importance of perceived value takes precedence over cost and quality considerations.

416

Meeting of the Minds417

We must now redefine our innovations processes to meet rapidly changing consumer demands. Our business development managers are being challenged to identify the value of business opportunities for new products that do not yet exist. Our R&D staff must develop these products and we must have an inno- vation process in place that allows us to achieve our strategic objectives.

Because of the turbulent business environment, time is no longer a luxury but a critical constraint in our innovation process. With limited resources to work with, we must be absolutely sure that we are working on the right mix of projects. We are in the process of developing a metrics management system to allow us to make better decisions with regard to the selection and development of new products with exceptional value. The metrics we create will help us ensure that we are creating products that have value. Metrics management is essential. We must know if we are heading in the right direction and if the light at the end of the tunnel is reachable. If the metrics indicate that we cannot achieve our goals on a particular project, then we must pull the plug and assign the resources to those other projects that provide business value opportunities.

We are establishing a metric management team to develop this capabil- ity. The metrics management team will report to the PMO. I expect all of you to assist the team in carrying out their mission if they ask for your input and assistance.

MEETING OF THE MINDS

Al Grey was convinced that he was on the right track in his quest for a metrics management system. Rather than leave the team member assignments to chance, Al personally selected the members of the task force. He knew each of these team members personally and was convinced that they could live up to the challenge. The six team members were:

●John: representing the PMO and the team leader ●Patsy: representing Marketing ● Carol: representing New Business Development ● Allen: representing Engineering

●Barry: representing R&D ●Paul: representing Manufacturing

The team met and began discussing their challenge. The first step was to get a good understanding of what metrics are and how the company can benefit from their use. Everybody seemed to understand that a company cannot manage innovation projects without having good metrics and reasonably accurate measurement capable of providing complete or nearly complete information for the decision-makers. Furthermore, since most of the company’s projects were becoming more complex, it would become harder to determine true progress without effective metrics.

418THE NEED FOR PROJECT MANAGEMENT METRICS (B)

The team prepared a list of the benefits of using metrics. The list included:

●To improve performance for the future

●To improve future estimating

●To validate baselines

●To validate if we are hitting our targets or getting better or worse

●To catch mistakes before they lead to other perhaps more serious mistakes

●To improve client satisfaction

●A means of capturing best practices and lessons learned Although everyone agreed on the benefits of metrics, John expressed his con-

cern that the team must remain focused. John stated:

It takes companies years to achieve all of those benefits. We simply do not have the luxury of doing that. We must focus on our primary mission, which is the establishment of metrics for our innovation process. We need objec- tive, measurable attributes of project performance in order to make informed decisions. We must be able to use the metrics to predict project success and failure. Therefore, we must establish some type of priority for what type of metrics we will develop first.

The team decided that the primary focus should be to establish metrics that can be used as a means of continuous health checks on innovation projects. The metrics has to serve as early warning signs or risk triggers.

But deciding what to do and being able to do it were two separate activities. The business side of the company had been using metrics for some time. These were metrics related to market share, profitability, cash flow, and other such high- level measurements. The innovation metrics to be developed would be more detailed, and this could alienate the culture to the point where there would be more resistance than support. Allen commented:

Engineers do not like constant supervision. They like the freedom to create and I am sure the same holds true in R&D. If we develop metrics that are too detailed and our people believe that the metrics are being used to spy on them, I feel that we will get a lot of resistance.