Module 7 – Regulation Implementation & Management

  • Chapter 3: Measuring Regulatory Excellence, in Listening, Learning, and Leading: Aframework for Regulatory Excellence – Cary Coglianese (2015)

Available via Docushare

  • Responsive Excellence – John Braithwaite (2015)

Willful Blindness? Regulatory Failures behind the Lac-Mégantic Disaster – Bruce Campbell (2014)

Available via Docushare

  • Regulatory Excellence: The Role of Policy Learning and Reputation – David Vogel (2015)

Available via Docushare

  • Chapter 1: What is Adaptive Management? – US Dept of Interior (2012)

Supplementary/Reference:

  • OECD Guiding Principles for Regulatory Quality and Performance
  • Covert Leadership: Notes on Managing Professionals – Henry Mintzberg (1998)Available via Docushare

Module 7 – Executive Summary of Readings

Chapter 3: Measuring Regulatory Excellence, in Listening, Learning, and Leading: A framework or Regulatory Excellence – Cary Coglianese (2015)

Measurement and ratings systems exist to help inform and guide choices. The U.S. government has formally institutionalized performance measurement systems. The World Bank and the OECD have created rating systems that seek to capture the level of government regulation as well as the ease of doing business across different countries.

Measurement is how a regulator gauges how a regulated entity or industry is doing, and identifies areas for improvement. In addition, measurement could be used to gauge a regulator’s overall level of excellence. Regulators need to take a strategic approach to performance measurement and evaluation – they should not simply measure what is easy to measure, and not simply measure for measurement’s sake. Measurement of regulatory performance is not an abstract, academic exercise. It is a tool that can be used actively and intentionally to get feedback, demonstrate progress, identify weaknesses, and serve other internal and external needs for information. Measurement helps leaders learn better what works, identify and track regulatory problems, communicate progress, and ultimately manage toward continuous improvement.

Measurement for excellence refers to the kinds of measures and measurement practices that an excellent regulator would put in place in order to run its operations. That is to say, it is the excellence scorecard developed by the regulator to determine the performance of the regulated entities.Measurement of excellence seeks to determine how well a regulator is fulfilling its role. Measurement of excellence may in some cases rely upon some of the same measures used for becoming excellent; however, since measuring excellence is a distinct purpose, it will necessitate a different measurement approach.

Popular measurement systems succeed because the attributes being measured – and the way they are weighted and aggregated – help inform and fulfill the needs of users. There are eight purposes served by performance measurement in the public sector: to evaluate, control, budget, motivate, promote, celebrate, learn, and to improve.

The needs for performance measures vary across different kinds of regulators. Furthermore, oil and gas regulators in different locations or time periods will need different measures, and the same regulator will need different measures for internal and external needs. The challenge lies in establishing measures that address regulatory goals, are meaningful and reliable, and sensibly linked to incentives.

Regulators seek to maximize a number of objectives that can seem conflicting. A regulator may be charged with reducing risks from an industrial activity while also keeping compliance costs low, promoting distributional equity, and acting transparently, thoroughly, and quickly. If all of the regulator’s operations with respect to each of these objectives could be monetized accurately, then the negative impacts could be subtracted from the positive ones to yield a net-benefits estimate, which would in principle provide a meaningful measure of the regulator’s overall excellence. However, the measurement of overall regulatory excellence will never be easy or tidy. Measurement for excellence therefore should presumably take precedence over measurement of excellence – the aim should be to use measurement to engage in ‘value-oriented’ management that puts the production of public value front and centre while acknowledging the complexities of defining and recognizing value in the public sector. That said, measures of excellence could be used to motivate major transformations throughout an organization, so as to encourage its workforce to improve by measuring how far the regulator is coming along on the regulatory excellence path; after all, “what gets measured gets managed”.

Questions provoked:

  1. Does your organization measure how well it is doing as a regulator?
  1. If your organization does not measure its performance regularly, why is that?

Key takeaways for CPRL:

  • Measurements must be used regularly, to serve varying needs.
  • Measurement for excellence should take precedence over measurement of excellence.

Responsive Excellence – John Braithwaite (2015)

For a regulator aspiring to achieve excellence, responsiveness to opportunities is more important than responsiveness to risk. The excellent regulator scans for cases that offer strategic, macro-opportunities to create public value, even if it involves transforming an industry or an entire economy and society. The business of crisis prevention involves all regulatory players seizing opportunities to eliminate those they can tackle before all those risks come together in another perfect storm.

Responsive regulatory organizations act to honor the principles laid down in enabling statutes, but they are not rule-based machines; legal principles inform a regulatory storybook that can transcend unresponsive regulatory rulebooks. Excellent regulators are entrepreneurial risk-takers in the cause of creating public value.

A limitation might be that the onus is on the regulator to foresee issues. On the other hand, we expect public health officials and physicians to distinguish one-off treatments for patients from responses that might identify a new virus and contain its spread. Braithwaite argues that this could and should become part of our understanding of regulatory excellence.

Regulatory culture is not a rulebook, but rather a storybook. To change how street-level bureaucracy behaves you must encourage stories about how excellent inspection has made a difference.

In Australia, regulatory capture would be the best way to characterize a great many enforceable undertakings that have been negotiated. Enforceable undertaking laws were a creative and powerful option in the hands of the entrepreneurially excellent regulator, a “soft option” in the hands of the captured regulator, and simply a different intermediate option in the middle of the regulatory pyramid for others. Yet the strengths of Australian regulation are encapsulated by greater engagement of the street level activities of regulatory bureaucracies, greater public awareness, and third-party engagement.

Questions provoked:

  1. How important is story-telling in your organization, in defining regulatory excellence?
  1. How can your organization use story telling to promote the behaviours associated with those stories?

Key takeaways for CPRL:

  • A culture of leadership from below can be effective in seizing innovative opportunities to advance a regulatory agency’s objectives.
  • Transformative opportunities seized by regulators can self-enforce, and promote ethics- and compliance-based cultures.

Willful Blindness? Regulatory Failures behind the Lac-Mégantic Disaster – Bruce Campbell (2014)

This report by the former Executive Director of the Canadian Centre for Policy Alternatives (CCPA), Bruce Campbell, details eight key ways in which regulatory failure contributed to the Lac-Mégantic disaster. It chronicles how Canada’s regulatory regime failed- directly and indirectly - to prevent corporate negligence, for which the citizens of Lac-Mégantic paid a terrible price. The author argues, “Canada’s railway regulatory regime—which gives companies primary responsibility for establishing and implementing their own safety systems within the framework of rules set by the government—is itself open to criticism”. Given the existing system, “the ingredients for regulatory failure are readily in place if those rules are too vague or otherwise deficient; if companies are regularly granted exemptions to these rules; if the relationship between regulator and regulated is unduly cozy; if, as three Auditor General reports have found, Transport Canada is not able to provide the necessary oversight and enforcement to ensure its proper functioning.

The eight areas where Canada's federal regulatory system failed Lac-Mégantic are:

  1. Transport Canada's railway operating rules are at times vague and inadequately enforced, giving companies too much latitude and granting too many exemptions.
  2. Senior Transport Canada officials, over objections from its Montreal office and the union, granted permission to Montreal, Maine & Atlantic railway to operate trains carrying massive amounts of crude oil, a dangerous good, with a one-person crew.
  3. Transport Canada, despite multiple warnings, allowed crude oil to be transported in unsafe tank cars.
  4. Transport Canada disregarded concerns about the explosiveness of Bakken crude, had lax testing requirements, and collected insufficient data about the transportation of dangerous goods.
  5. Transport Canada's Safety Management Systems were defective – lacking sufficient oversight and enforcement.
  6. Transport Canada's risk assessment processes and protocols were flawed.
  7. Transport Canada was complacent in light of the oil-by-rail boom, allocating insufficient regulatory resources to cope with the massive surge.
  8. Transport Canada has allowed the industry lobby to become too powerful, compromising public safety.

The foundation of any regulatory regime is set at the top. This is where the tone and expectations, guiding rules and procedures, and budgets—the regulatory culture—are established. According to the author, Lac-Mégantic was not a natural disaster, neither was it an uncontrollable event…. It could have been prevented.Part of the challenge in preventing another Lac-Mégantic, is to keep the spotlight on its root causes—corporate negligence and regulatory failure during a wild-west boom in the transport of oil-by-rail. Government and industry will continue their efforts to rebuild public confidence in rail safety in part by “implanting post-disaster version of events that frames Lac-Mégantic as a learning experience and obscures these causes.”

Questions provoked:

  1. Do you agree with Campbell’s argument that Lac-Mégantic was due to the failure of Transport Canada’s regulatory regime, or with federal Transport Minister, Lisa Raitt’ assertion that the accident was due to the “negligence of certain individuals—not federal regulatory gaps”?
  1. In what ways can your regulatory agencies and departments learn from Transport Canada’s analysis or experience with Lac Mégantic’s railway safety crisis?

Key takeaways for CPRL:

  • Lac-Mégantic is the most devastating Canadian rail disaster in a century.
  • Such a disaster should warrant a thorough review and verificationof performance based systems to ensure their effectiveness and proficiency.

Chapter 1: What is Adaptive Management? – US Dept of Interior (2012)

Adaptive management is a systematic approach for improving resource management by learning from management outcomes. Adaptive management is infrequently implemented, even though many resource-planning documents call for it and numerous resource managers refer to it. A context for resource management involves a decision-making environment characterized by multiple, sometimes competing, management objectives, constrained management authorities and capabilities, dynamic ecological and physical systems, and uncertain responses to management actions. Management thus involves not only predicting how systems are likely to respond to interventions, but also identifying what management options are available, what outcomes are desired, how much risk can be tolerated, and how best to choose among a set of alternatives. The challenge confronting managers is to make “good” decisions in this complex environment, recognizing that the quality of decision making in the face of uncertainty should be judged by the decision-making process as well as progress towards desired outcomes.

It is thought that by monitoring activities and occasionally changing them, one is doing adaptive management. Contrary to this belief, adaptive management is much more than simply tracking and changing management direction in the face of failed policies. An adaptive approach involves exploring alternative ways to meet management objectives, predicting the outcomes of alternatives based on the current state of knowledge, implementing one or more of these alternatives, monitoring to learn about the impacts of management actions, and then using the results to update knowledge and adjust management actions. Adaptive management focuses on learning and adapting, through partnerships of managers, scientists, and other stakeholders who learn together how to create and maintain sustainable resource systems.

The uncertainty about management impacts is often expressed as disagreements among stakeholders who have differing views about the direction and magnitude of resource change in response to management. An adaptive approach explicitly articulates these viewpoints, incorporates them into the decision-making process, and uses management itself to help identify the most appropriate view about resource dynamics. In this way, understanding of the resource can be enhanced over time, and management can be improved.

Question provoked:

  1. How many times through the adaptive management life cycle should a regulation travel? Can the process ever be complete?
  1. Industry generally likes a predictable, stable regulatory environment. Would the industries you work with be amenable to an adaptive management approach?

Key takeaways for CPRL:

  • Adaptive management focuses on gaining experience and knowledge so that future management decisions can be improved.
  • Stakeholders should be involved in assessing a problem and designing activities to solve it. They also can implement and monitor activities, and participate in the evaluation of results.

Regulatory Excellence: The Role of Policy Learning and Reputation – David Vogel (2015)

Policy learning has two key dimensions. First, policy learning requires that a regulator recognize the accomplishments and shortcomings of decisions made by other regulators as well as its own. Second, policy learning requires that the regulator demonstrate responsiveness to new information as it emerges. In this article, David Vogel explores three case studies of policy learning and reputation. Two case studies involve various dimensions of transatlantic regulatory policy learning – or the lack thereof – namely, comparisons of the regulation of ozone depleting chemicals and pharmaceutical products in the United States and Europe. The third case study examines the dynamics of regulatory reputation and learning within a single jurisdiction, focusing on the performance and impact of the agency regulating air pollution in California.

First, excellent regulatory officials must place priority on maintaining public confidence in the mission, work, and decisions of their agencies. It is critical for them to recognize that they are embedded in democratic political systems and that therefore they need to be aware of and broadly responsive to public preferences, especially as these may shift. In the case of ozone depleting chemicals, both American and European authorities were able to retain their reputations precisely because of each of their actions, even though they differed from one another, it was broadly consistent with the risk perceptions of their respective societies. The same has been true of the California Air Resources Board (CARB). CARB has been successful in large measure because Californians have strongly supported its policy goal of improving air quality.

Second, excellent regulatory officials are never complacent: they must be continually open to and engaged in policy learning. Regulators are always faced with a wide range of policy options and often have to act in the face of scientific or technological uncertainty. Some may enjoy better reputations than others. In a sense, all regulatory policies and decisions are experiments: they are always provisional, and they can also be improved. Much of the CARB’s success can be attributed to its ability to engage in continuous policy learning about how mobile source pollutants can be more effectively and efficiently controlled. European pollution control authorities changed their regulations for ozone depleting chemicals as new scientific data emerged, and, over time, the FDA demonstrated its regulatory excellence by exhibiting a willingness to learn from its counterparts in Europe.

Finally, regulatory excellence requires maintaining an appropriate relationship between regulatory officials and business firms affected by their decisions. On one hand, it is critical that the agency avoid becoming the captive of business interests. But on the other, it also must be cognizant of the economic impact of its regulatory policies and be willing to learn from the knowledge and expertise of the private sector. CARB has been so successful precisely because it has been able to strike such a balance: it has both challenged the interests of the major automotive manufacturers, while working closely with them and with other business innovators in pollution control technologies. Too cozy a relationship with industry can undermine an agency’s reputation and legitimacy, but at the same time, so can a relationship that is too adversarial. An excellent regulatory agency must recognize that its effectiveness can be enhanced if it can develop business allies and can demonstrate the ways in which its policies create economic as well as social value.

Question provoked:

  1. In the case of the ozone layer depletions, the Americans and Europeans disagreed on whether CFC’s in consumer products should be restricted (American public considered the risk to be both credible and unacceptable, but the EU public did not trust US regulators as they had a reputation for being “too hasty” with regulations and they regarded the American evidence as “scientifically disputed”). In the end, the American’s were proven right as more evidence proved the link between CFC and skin cancers, but Vogel believes both regions reacted rightly and retained their reputations as they imposed restrictions based on public preferences. Has Vogel given too much weight to public preferences?
  1. In the case of the Pharmaceutical Drug Regulation, after Dr. Kelsey and the FDA successfully averted the Thalidomide disaster, the FDA strengthened its global reputation and policy impact, and became the gold standard for drug regulations. But the FDA’s perceived near policy failure as its previous approval standards placed the public spotlight on the possible risks of them approving unsafe drugs. Consequently, the costs for meeting FDA’s new requirements and the time required for a drug to be approved increased significantly. However in Europe, while regulatory institutions and procedures changed, the drug approval process largely remained the same. Today, due to the increased costs and massive time lag in the US, the FDA is now criticized for denying Americans access to drugs already available in Europe. Critics now argued that the health and safety of Europeans had been enhanced precisely because European drug approval authorities had not emulated the FDA by tightening their drug approval requirements.

How can regulators walk the fine line between respecting public perceptions or public pressures, and regulating based on the best available evidence?