Draft

Is it Time to Consider a New Approach for
Reviewing and Updating the NAAQS?

[List Authors]

1

Draft

Table of Contents

Page

1Introduction

2The NAAQS Framework for Causation Assessment

2.1Epidemiology Studies

2.1.1Exposure Measurement Error

2.1.2Confounding

2.1.3Statistical Models and Model Specification

2.2Experimental Studies

2.3Evidence Integration

3Risk and Exposure Assessment

3.1Exposure Assessment

3.2Risk Assessment

4Regulatory Impact Analyses

4.1Implementation Costs

4.2C-R Model Uncertainties

4.3Co-benefits of Reduced PM2.5 Emissions

4.4Economic Benefits Metrics

5Conclusions

References

1Introduction...... 3

2The NAAQS Framework for Causation Assessment...... 4

2.1Epidemiology Studies...... 4

2.1.1Exposure Measurement Error...... 4

2.1.2Confounding...... 5

2.1.3Statistical Models and Model Specification...... 6

2.2Experimental Studies...... 6

2.3Evidence Integration...... 7

3Risk and Exposure Assessment...... 9

3.1Exposure Assessment...... 9

3.2Risk Assessment...... 10

4Economic Impacts Analysis...... 12

4.1Implementation Costs...... 12

4.2C-R Model Uncertainties...... 14

4.3Co-benefits of Reduced PM2.5 Emissions...... 14

4.4Economic Benefits Metrics...... 14

5Conclusions...... 16

References...... 18

1

Draft

1Introduction

In the US Environmental Protection Agency's (EPA's) 2014-2018 Strategic Plan, one of Administrator McCarthy's objectives under Goal 1, "Addressing Climate Change and Improving Air Quality," is to "[a]chieve and maintain health- and welfare-based air pollution standards and reduce risk from toxic air pollutants and indoor air contaminants" (US EPA, 2014a no acn). This objective is consistent with Sections 108 and 109 of the federal Clean Air Act (CAA), which specify the process for introducing new and reviewing existing National Ambient Air Quality Standards (NAAQS). The NAAQS process includes the following steps:

  • Planning,
  • Conducting an Integrated Science Assessment (ISA),
  • Conducting a Risk and Exposure Assessment (REA),
  • Conducting a Policy Assessment,
  • Having the Clean Air Scientific Advisory Committee (CASAC) review and comment on each of these assessments,
  • Conducting a formal rulemaking, including issuing a proposed rule for public comment, addressing public comments, and issuing a final rule, and
  • Implementing the final rule.

Section 109(a) of the CAA requires EPA to set NAAQS for six "criteria" air pollutants (i.e., ozone, particulate matter [PM], lead, carbon monoxide, nitrogen dioxide, and sulfur dioxide) and review those standards every five years. The statute instructs the agency to set standards such that "the attainment and maintenance of which...are requisite to protect the public health" with "an adequate margin of safety" (42 US Code §7409; US Congress, 2011 211-6848). Regulatory Impact Assessments (RIAs), conducted to examine benefits, costs, and other economic impacts of regulations, are a standard part of the regulatory process. Even though EPA is not permitted to consider costs in setting the NAAQS, the CAA mandates EPA to conduct these analyses to demonstrate that each intended regulation is necessary and the potential benefits of implementation justify its costs (US EPA, 1999 214-9488).

Significant advances in the scientific understanding of air pollution and toxicology have been made since the CAA was promulgated in 1970 and the last major amendments were adopted in 1990. These advances, coupled with the difficulty and cost of attaining ever-tightening standard levels, raise the question: Is it time to consider a new approach for reviewing and updating the NAAQS? The following discussion uses examples primarily drawn from the reviews of the PM and ozone NAAQS, but most of the issues we raised apply to the review of other criteria pollutants as well.

2The NAAQS Framework for Causation Assessment

EPA uses a weight-of-evidence (WoE) frameworkfor causal determination(hereafter, the "NAAQS causal framework")that includes methods for literature searches; study selection, evaluation andintegration; and causal judgments. The NAAQS causal framework has many valuable features, but it could be more explicit in some cases and some features are missing. This has caused it to be applied inconsistently and its use has sometimes led to conclusions that are not supported by the overall WoE. Goodman et al. (2013a 213-7578) identified additions to the NAAQS causal framework that would help align it with best practices for systematic reviews and evidence integration. This includes guidelines for evaluating all of the evidence in a consistent manner using well-specified criteria and determining whether such evidence constitutes support for causation or an alternative hypothesis. The additions that Goodman et al. (2013a 213-7578) identified should be adopted by EPA so that causal evaluations are more thorough,transparent, and scientifically sound, and such that they do not yield a causal determination that may not be warranted. This is particularly important because associations that are considered to be "causal" or "likely causal" are included in EPA's risk assessment.

2.1Epidemiology Studies

Epidemiology evidence figures prominently in EPA's evaluation of causality for most criteria pollutants. Despite significant advances in epidemiology methodsthat have been made in the past 25 years, these studies still have major limitations that often preclude conclusions regarding causality. In general, EPA has focused on epidemiology studies that report very small, but statistically significant, associations between increasingly low levels of air pollutants and health effects, but it has not given the same weight to studies of similar quality that report no associations (e.g., see US EPA, 2013a 211-1526d,b 213-8712). Importantly, although EPA discusses some limitations associated with these studies, it is unclear how it considers them when judging the evidence. These limitations include exposure measurement error, confounding by co-pollutants and other factors, and uncertainty with the statistical models.

2.1.1Exposure Measurement Error

Most epidemiology studies rely on data from central ambient monitoring sites to provide community average ambient pollutant exposure concentrations (e.g., Neaset al., 1999 207-5124; Naeheret al., 1999 207-5122; Mortimer etal., 2002 205-3967; Gent et al., 2003204-0728; Katsouyanniet al., 2009209-6475; Stiebet al., 2009209-5706), and the interpretation of statistical associations is predicated on the assumption that these ambient measurements reflect actual personal exposures. That is, in these studies, individuals are assumed to be exposed to the concentration of each pollutant measured outdoors at the ambient monitor nearest to their home, 24 hours a day, seven days a week. There are several reasons why this assumption does not hold. One is that ambient monitors may be miles away from where individuals work and live, and therefore do not necessarily reflect local concentrations. Another is that being indoors affects a person's exposure to air pollutants. In the case of PM, there are many indoor sources that account for much higher individual exposures (e.g., Long et al., 2000206-7638). Exposure measurement error results when there is poor correlation between the measured exposures used in an epidemiology study and actual individual exposures of the study population.

During the last ozone review process, CASAC highlighted exposure measurement error as a key uncertainty affecting the ozone epidemiology literature, concluding that central-site community monitors that measure ozone in the ambient air are generally poor measures of individual exposures (Henderson, 2006206-4381). Henderson (2006206-4381) further reported that personal ozone exposures are typically much lower than ambient ozone levels and, more importantly, often show little or no correlation with concentrations measured at the central ambient sites. For example, for a Baltimore-based cohort of 56 subjects, Sarnatet al. (2001 201-5841) reported no correlation between ambient and personal ozone measurements for either winter or summertime sampling periods (correlation slopes of 0.00 and 0.01, respectively). In a similarly designed study conducted in Boston, Sarnatet al. (2005206-7663) reported comparable results, finding no correlation between ambient and personal ozone concentrations in winter (correlation slope of 0.04) and only a moderate correlation between ambient and personal ozone concentrations in summer (slope of 0.27).

EPA and others have asserted that exposure measurement error is likely to underestimate risk from air pollutant exposures. For example, in the REA for PM, EPA cited the intra-urban analysis conducted by Krewski et al. (2009209-5135) as support for this assertion. As discussed in detail in Rhomberg et al. (2011a211-7617), exposure measurement error generally leads to an underestimation of risks at high exposures and an overestimation of risks at low exposures; the latter is particularly relevant for the NAAQS. This should be considered when assessing whether epidemiology studies support causal associations.

2.1.2Confounding

Co-pollutants can significantly impact risk estimates, whichis why multi-pollutant models are so important. In the most recent analysis of the American Cancer Society (ACS) cohort, Krewski et al. (2009209-5135) reported associations between several pollutants and mortality in single-pollutant models, but they did not present results from multi-pollutant models. In this study, mortality risks reported for several pollutants (e.g., sulfur dioxide and summertime ozone) were of similar magnitude and statistical significance as fine PM (i.e., PM2.5)(Krewski et al., 2009209-5135). In their earlier re-analysis of the ACS study, Krewski et al. (2000210-5982[a1]) found that adjustment for co-pollutants generally decreased PM2.5 risk estimates. For example, the relative risk (RR) for all-cause mortality from PM2.5 in the ACS cohort was reduced from a statistically significant risk of 1.18 (95% confidence interval [CI]: 1.03 to 1.35) to risks that were not statistically significant when adjusted for sulfur dioxide (RR = 1.03, 95% CI: 0.95 to 1.13) or all four gaseous co-pollutants (RR = 1.06, 95% CI: 0.95 to 1.18).

Temperature and other environmental factors can also confound the relationship between pollutants and health effects. For example, the longitudinal study of ozone and children with asthma by Gent et al. (2003204-0728) only considered same-day maximum temperature, while meteorological variables such as relative humidity may have been potential confounders of respiratory symptoms. Air conditioning use and exposure to tobacco smoke are also important potential confounders of causal associations with respiratory effects, yet they were not accounted for in some key studies (e.g., Mortimer et al., 2002205-3967; Stiebet al., 2009209-5706).

Although EPA evaluated confounders to an extent, EPA should interpret epidemiology study results with a full consideration of how co-pollutants or other environmental confounders (e.g., temperature) impact statistical associations, as well as how results are used in developing air quality standards. Furthermore, a multi-pollutant approach is essential in risk assessment based on epidemiology studies to identify the true risks of pollutant exposure.

2.1.3Statistical Models and Model Specification

Results from air pollution epidemiology studies have been shown to vary depending on the statistical method or model specifications. For example, EPA has interpreted findings of the two prominent cohort studies underlying concentration-response (C-R) functions for PM2.5 (i.e., the ACS cohort evaluation by Krewski et al.[2009 209-5135] and the National Mortality and Morbidity Air Pollution Study [NMMAPS] cohort evaluated by Dominiciet al. [2007 209-1011]) as supportive of a causal relationship. However, Cox (2012a212-0740a)described an analysis he conducted of the NMMAPS dataset, which included census data, daily mortality rates, daily PM2.5 estimates, and meteorological measurementsin more than 100 US cities. He showed that a number of regression models yielded both positive and null associations between PM2.5 exposure and mortality, depending on the treatment of daily temperature, which is a strong confounder in the PM2.5-mortality associations. In addition, he applied Granger causality tests and found that less than 4% of associations between daily PM2.5exposure and all-cause, cardiovascular, and respiratory mortality were significant. He concluded that the results do not suggest a causal relationship.

While the need for causal interpretation of statistical associations has been acknowledged by many in the pollution health effects research field, and tests for assessing potential causation have been developed, these methods have not been generally applied to PM2.5 or ozone and mortality data (Cox, 2012b212-0735). Cox et al. (2013213-6467) suggested three general methodological steps to test for causality:

  • Generate, test, and, if possible, refute plausible alternative (non-causal) explanations for positive associations (also known as a hypothesis-based weight-of-evidence evaluation; see Rhomberg et al., 2010210-8094);
  • Show that the association cannot be explained by using alternative statistical models or other information; and
  • If possible, test whether changes in responses follow (and can be successfully predicted from) changes in individual exposures.

Some of the statistical tests that could be applied to assess causality include conditional independence tests (Freedman, 2004 no acn; Friedman and Goldzsmidt, 1998 no acn), Granger causality tests (Eichler and Didelez, 2010 no acn), change point analysis (Gilmour et al., 2006 no acn; Helfenstein, 1991 no acn), causal network models of change propagation (Dash and Druzdzel, 2008; Hack et al., 2010 no acn), and negative controls for exposures or for effects (Lipsitchet al., 2010 no acn). Such causal tests, rather than simple correlation analyses, should be applied to analyses of epidemiological studies in support of future NAAQS reviews.

2.2Experimental Studies

Traditional toxicology studies evaluate clinically- relevant toxicity or disease, often at high exposure levels (and thus often with questionable relevance to humans; see Goodman et al., 2010 210-7786). Evidence based on mechanistic whole-animal toxicology studies, as well as in vitro studies of tissues, cells, and molecules (e.g., cell-membrane constituents, proteins, DNA), can help identifya mode of action. These studies build on our understanding of chemical toxicology and molecular biology and allow the plausibility of various causal pathways to be explored. The relationship of simple cellular or enzymatic results relative to the whole living organism may be complex, given that they are by their very nature simplifications that may not reflect actual exposure conditions (e.g., inhalation, reaction with extracellular lining fluid, penetration of extracellular matrix proteins and cell membranes) or account for complex whole-organism processes that could mitigate or amplify responses seen in isolated cells. Also, in some cases, a particular biological change may be part of a homeostatic process and so may not be indicative of adverse effects. Despite these limitations, these types of studies can be very informative for understanding risks of criteria pollutants (Goodman et al., 2014a 214-0505).

Mode-of-action studies have shown that antioxidants present within airway lining fluid can prevent ozone-mediated cellular and tissue oxidation (Avissaret al., 2000211-4159; Ballinger et al., 2005211-4157; Cross et al., 1994211-4158; Mudwayet al., 1996200-5612; Samet et al., 2001205-3973), and only ozone exposure of a sufficient duration and concentration can overwhelm antioxidant defenses, allowing oxidative damage to occur in airway epithelial cells (Schelegleet al., 2007210-2194). Similarly, as reviewed by Cox (2012b 212-0735), low levels of PM exposure increase antioxidant generation in the lung, but higher levels induce the generation of reactive oxygen species that overwhelm the lung's homeostatic mechanisms. Together, these studies show that a threshold exists below which antioxidant defenses are sufficient to protect against adverse effects of ozone and PM2.5. It is notable that a mode of action that allows reliable prediction of adverse health impacts at ozone and PM2.5 exposure levels typical of the present-day ambient environment has not emerged.

It is encouraging that EPA has cited more of this type of experimental data in its recent policy reviews. However, moving forward, it is critical that experimental studies, such as those noted above, be given the same weight as those showing a correlation with adverse health effects. This is because these types of studies may imply the existence of a threshold below which more stringent pollutant regulations may not lead to improvements in public health or welfare.

2.3Evidence Integration

The NAAQS causal framework indicates one should look separately at epidemiology, controlled exposure, and animal toxicology evidence, first coming to a synthesizedjudgment for each and then integrating these separate judgments into an overall qualitative statement about causality (US EPA, 2013a211-1526d,b 213-8712). As discussed by Goodman et al. (2013a 213-7578), data evaluation should be integrated across all lines of evidence before coming to judgments based on each realm independently. In this way, interpretation of each line of evidence informs the interpretation of the others.

As an example of how a flawed approach to evidence integration can impact causality determinations, EPA (2013a211-1526d) stated that recent animal toxicology studies of ozone exposures provide stronger evidence for cardiovascular effects than epidemiology studies of ozone exposures and concluded that the evidence was indicative of a "likely" causal” relationship. The key animal studies on which EPA relied were conducted at very high ozone exposure levels and have little relevance to ambient human ozone exposures. Also, EPA considered an increase in heart rate variability as the key indicator of effect in the animal studies, but the epidemiology evidence regarding heart rate variability is inconsistent and does not corroborate the animal data. In this instance, a more effective approach would be to consider the lack of consistency and coherence of evidence across different realms in making a causal determination (e.g., Goodman et al., 2014b214-0323).

Another example can be found in the PM Final Rule, in which EPA stated that "the findings of new toxicological and controlled human exposure studies greatly expand and provide stronger support for a number of potential biological mechanisms or pathways for cardiovascular and respiratory effects associated with long- and short-term PM exposures"(US EPA, 2012 212-9800). A review of the PM ISA, however, suggests that the experimental evidence is inconsistent and not coherent with findings in epidemiology studies. Specifically, the findings of mild and reversible effects in most experimental studies conducted at elevated exposures are not coherent with the more serious associations described in epidemiology studies (e.g., hospital admissions and mortality). Also, both animal and controlled human exposure studies have identified no-effect levels for acute and chronic exposure to PM and PM constituents at concentrations considerably above ambient levels(e.g., Gong et al., 2003209-1090; Holgate et al., 2003 no acn; 203-5116; Schlesinger and Cassee 2003 204-0922). Therefore, a better approach would be to consider experimental findings such as these in light of the high exposure levels and what the relevance may be for ambient exposures.

The current approach of evaluating each realm of evidence separately and then integrating judgments at the end of the process does not allow data from one realm of evidence to influence conclusions from another. Instead, evidence integration that involves an evaluation of how results of particular studies can inform potential similar causal processes in other studies, including studies in other realms of investigation, should be applied. It is the potential for such commonality of causal processes that makes animal data useful evidence for potential effects in humans.

3Risk and Exposure Assessment

EPA includes the health effects it finds to have a “likely causal” or “causal” relationship with the pollutant in its quantitative risk assessment (i.e., the REA) (US EPA, 2013a211-1526d,b 213-8712). In assessing a “causal” relationship, EPA concludes that evidence is sufficient at relevant pollutant exposures, whereas for a “likely causal” relationship, EPA concludes evidence is sufficient, but important uncertainties remain (US EPA, 2013a211-1526d,b 213-8712). EPA conducts REAs to assess risks associated with criteria air pollutants by estimating exposures using air quality or personal exposure models and estimating health risks based on C-R functions derived from the literature. For ozone, EPA conducted two separate risk evaluations: one for lung function decrements based on the results of controlled exposure studies, and one for mortality and morbidity endpoints (e.g., emergency room visits or hospital admissions) based on epidemiology data(US EPA, 2014b214-0638). For PM, estimated risks of mortality and morbidity endpoints are based only on epidemiology studies (e.g., US EPA, 2009 209-1431). All of thhese risk evaluations are based on a number of critical assumptions that can significantly impact the results, yet the uncertainties associated with these assumptions are not fully considered by EPA. The methods EPA uses and some of the issues associated with these methods are described below.