National Academies Press: OpenBook

New Source Review for Stationary Sources of Air Pollution (2006)

Chapter: 5 Econometric Analysis

« Previous: 4 Analytic Framework for Assessing Effects of New Source Review Rule Changes
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

5
Econometric Analysis

INTRODUCTION

This chapter explores the possibility of using econometric analysis to measure the effects of the New Source Review (NSR) rule changes on a variety of outcome measures. As explained in Chapter 4, the best available approach (given data constraints) would be a reduced-form estimation, in which a facility’s outcome measure (Y) is related to an indicator of the type of NSR rules faced by the facility (X) and to other explanatory variables (Z) that could affect the outcome (as in Equation 4-1):

Chapter 4 noted that a reduced-form analysis could identify the variation in NSR regulation in three different ways. A time-series approach would focus on variation in Y for the same facility before and after the NSR rule changes. A cross-section approach would focus on variation in Y across facilities in different states when some of the states had implemented the NSR rule changes and others had not. A difference-in-differences approach would combine the two others, first calculating the changes in Y for each facility over time, and then comparing those changes across facilities that were affected by the NSR rule changes at different times. A comprehensive analysis could include all three approaches, testing for consistency across the different sets of results. As with any econometric analysis, it is important to include a comprehensive set of Z variables to control for other factors that might affect Y. In the case at hand, we are looking at the impact of the 2002 and 2003 changes in the U.S. Environmental Protection Agency (EPA) NSR

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

process, which made it easier for a facility to renovate without triggering NSR. Changing the stringency of NSR requirements could have offsetting effects on overall pollution. If a firm chooses to go through NSR permitting despite having to face the stringent rules, stricter NSR requirements would generate greater emission reductions at the facility. Some firms trying to avoid NSR requirements would also reduce emissions by undertaking other pollution-control projects to avoid an increase in overall emissions or by accepting enforceable limits on facility emissions in the form of a “synthetic minor”1 designation. If a firm facing less strict regulations decides to invest in new equipment that results in a lower emission rate than does its existing equipment (although not as low as NSR might require), the less strict NSR rules could actually decrease emissions. NSR could thus affect both the decision of whether to adopt a new investment project and the final characteristics of the project, and it might cause alterations in other areas of a facility either to meet NSR requirements or to avoid NSR entirely. As outlined below, the recent and ongoing nature of the NSR rule changes, combined with the multiyear lags in the availability of outcome measures, make it impossible at this point to analyze the impact of the NSR rule changes on investment projects and overall emissions—the data are simply not yet available.

IDENTIFYING VARIATIONS IN POLICY

Variations in Policy Timing

To do any econometric analysis, we must be able to measure when and where the NSR rule changes became effective so that we can properly define the X variable in Equation 4-1 for any given facility-year observation. Under the U.S. federal system of environmental regulation, much of the regulatory activity is conducted by state agencies and is subject to federal oversight. In the case of air-pollution regulation, states develop state implementation plans (SIPs) designed to meet federal air-quality standards and conform to various federal requirements. Existing SIPs have been approved by EPA, and changes in a SIP must also be approved by EPA. Thus, if a state’s SIP includes an NSR program that applies to a particular facility, the state has to propose a revision to the NSR program, and EPA has to approve the revision before the NSR rule changes become effective for that facility. EPA gave such states until January 2006 to submit revised SIPs that incorporate the new

1

As discussed in Chapter 2, a source may reduce its potential to emit by agreeing to a legally binding limit on its emissions. If the source agrees to a limit that reduces its potential to emit below the coverage thresholds, it is no longer a major emitting facility and is exempt from the program.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

NSR rules. A few states have already revised their SIPs and submitted them to EPA for approval (see Chapter 2), although at this writing none of the revised SIPs has been formally approved by EPA, so at least in 2005 none of the facilities in these states has been covered by the NSR rule changes.

In contrast, facilities in states without an approved NSR program are subject to NSR rules at the federal level, so changes in the NSR rules for these facilities could be implemented directly by EPA. For such states, EPA made the NSR rule changes effective on March 3, 2003. All areas in nonattainment of the National Ambient Air Quality Standards (NAAQS) in which Part D applies have incorporated the Part D NSR program into their SIPs. A total of 13 states (entire or part) chose not to include a prevention of significant deterioration (PSD) rule in their SIPs, so make up the “implemented” group, where the NSR rule changes took effect first, in 2003.

Table 5-1 shows the list of states in the “implemented” and “nonimplemented” groups. The 2002 NSR revisions took effect on March 3, 2003, for states in the implemented group. States in the nonimplemented group will be subject to the revisions once they submit and EPA approves a revised SIP incorporating the NSR rule changes If a state opposing the NSR changes chooses not to submit such a SIP revision, EPA would have to decide whether to make the SIP changes themselves, which might involve further delay. A few of the states in the nonimplemented group indicated that they have already submitted revised SIPs to EPA, which may result in their NSR changes being approved earlier than those of other states in the nonimplemented group (but still at least 3 years after the implemented group).

One potential measurement difficulty for this analysis is related to identifying the “true” effective date of the regulations. Two sets of NSR rule changes were made final by EPA: in December 2002, changing the calculations that determine whether a given modification results in an important emission increase that will trigger NSR; and in October 2003, exempting routine maintenance, repair, and replacement projects from NSR. The latter set of rule changes are referred to as the equipment replacement provision

TABLE 5-1 Timing of NSR Rule Changes

Implemented group (March 3, 2003; only NAAQS attainment areas)

AZa, CAa, HI, IL, MA, MI, MN, NVa, NJ, NY, PAa, SD, WA

Nonimplemented group (no sooner than 2005-2006)

AL, AKb, AR, CO, CT, DE, DC, FL, GA, IDb, INb, IAb, KS, KYb, LA, ME, MD, MS, MO, MT, NE, NH, NM, NC, ND, OHb, OK, OR, RI, SC, TN, TX, UT, VT, VA, WV, WI, WY

aPartial coverage of state PSD areas in SIP.

bNonimplemented group states that have already submitted SIP revisions to EPA.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

(ERP). The rule changes have been the subject of court challenges and of opposition from some state regulatory agencies. Opponents to the ERP obtained a stay through the courts, so it never went into effect, and has been invalidated. Parts of the 2002 rules were also invalidated by the courts, so those rules are only partly in effect in a few states. Some time may elapse between the initial planning of an investment project and permit approval, so a firm could begin planning for a project, anticipating one set of rules, but face a different set of rules when the permit is finally decided. That uncertainty will tend to make firms conservative in responding to regulatory changes especially when there is a chance that the changes could be reversed. The possibility of reversal makes it difficult to be certain when the rule changes will start to affect investment decisions and thus more difficult to measure the effects of the rule changes. Uncertainty about other regulatory changes could also influence the impact of NSR on investment, including possible changes in NAAQS for ozone and particulate matter, as well as regional regulations, such as the Clean Air Interstate Rule (CAIR) and the proposed mercury rule. Because only the 2002 NSR rules have been implemented, the difficulty of identifying the effects of the two rule changes separately does not arise.

A second measurement difficulty is driven by the “forward-looking” nature of a firm’s investment decisions. An investment project at a facility is likely to affect its production process for many years, so the decision to undertake a particular investment today may depend on the firm’s expectations about regulatory constraints in the future. To the extent that NSR rules are expected to become less strict in the next year or two, firms may postpone investments until the NSR rule changes take effect. Such postponement could lead to a bunching of investment in the first few years after the rule changes, and the short-run response of investment could be considerably larger than the long-run response. Econometrically, that effect might be inferred if investment fell immediately before the NSR rule changes, rose sharply immediately after the changes, and then returned to near previous levels a few years after the changes. Such a measurement difficulty suggests the need to collect several years of outcome data and to examine closely the time pattern of outcome changes relative to policy changes.

Because the NSR rule changes affected some states in 2003 and other states are not affected until 2006 or later, we have some variation in policy timing to support a reduced-form model: seeing similar changes in outcomes (such as investment, pollution, and efficiency) in the nonimplemented states a few years after they occurred in the implemented states. Of course, other things may affect the timing, but the predictable timing of the policy change, combined with the variation across states, will help. We also get differences in policy timing within states based on the differences between NAAQS attainment and nonattainment areas in the implemented states. We would

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

obtain even stronger data for analysis if EPA were to allow some states to retain the prior NSR rules, which would result in long-run differences in investment and pollution outcomes across states. Note also that if the legal challenges to the NSR changes are successful, all facilities in all states will revert to the previous NSR policies, providing yet another policy change for an econometric analysis to work with (although less long-run effect to measure in the future).

Variations in Policy Stringency

So far we have discussed the NSR rule changes as though they had uniform effects on all facilities. The primary effect of the changes was to exempt from the need for an NSR permit some investment projects that would have required a permit under the prerevision rules. However, the stringency of an NSR permit can differ from one facility to another. First, NSR permit stringency may depend on facility location. Facilities in NAAQS nonattainment areas need to meet the lowest achievable emission rate (LAER), which can be more stringent than the best available control technology (BACT) required of facilities in attainment areas, along with other requirements (see Chapter 2). Second, the details of what is required under LAER and BACT may differ across facilities depending on the interpretation of those standards by the regulatory official responsible for approving the NSR permit. Certain states may tend to have more stringent interpretations of these requirements than others. Finally, the NSR permit review process could differ across states in speed and predictability, with slow or uncertain permit approval in some states serving as a major discouragement to investment activity. The importance of delaying the investment process could be especially important for manufacturing facilities, where firms are attempting to respond to rapidly changing business conditions.

Even if the NSR permit standards are the same for two facilities, the effect of the rule changes at each facility will depend on the stringency of any “fallback” permit requirements that the facility faces if it does not need an NSR permit. Depending on local air quality and state regulatory stringency, the application process for a minor permit might be about as stringent as that for an NSR permit or substantially less stringent. Massachusetts, Michigan, and Virginia indicated that their minor-permit programs require BACT in some circumstances, so the NSR rule changes might have less effect for them. Most states indicated only that they had some sort of minor-permit program in addition to NSR (e.g., Zervas 2005), so there does not seem to be enough information available at this time to characterize the stringency of state minor-permit programs. More detailed information about state minor-permit programs was provided by state regulatory agencies—suggesting that a more comprehensive database of state permit data could be developed

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

and collecting such data is in fact one of the committee’s recommendations mentioned later in this chapter.

The existence of different NSR effects among states adds uncertainty to an econometric analysis and makes it more difficult to measure any NSR effect. Suppose that NSR had a big effect in half the states in the implemented group and no effect in the other implemented-group states. The average effect might not be statistically significant, because the no-effect states dilute the effect of the others (the high variance in outcomes in the implemented group would reduce the significance of any difference in average outcomes between the implemented and nonimplemented groups). If we could add variables identifying the no-effect states to the analysis, we could estimate the effect of NSR separately for the high-effect states and improve the overall precision of our estimate of NSR effect and thus we could raise the likelihood of finding statistical significance.

One indirect indicator of a state’s desired level of stringency in NSR permits could come from the legal battles surrounding EPA’s NSR changes. Fourteen states and the District of Columbia brought suit against EPA to stop the rule changes. Those states arguably prefer more stringent NSR rules, so their permit writers would be expected to be stricter in interpreting the NSR requirements. But nine states supported the EPA’s legal position on the NSR rule changes, so permit writers in those states might be expected to be less stringent in their interpretations of NSR. That argument presumes that the variation among states in the policies preferred by state attorneys general who bring the lawsuits is similar to the variation in the policies preferred by the state regulatory-agency staff members who write the permits. Table 5-2 shows the breakdown of the implemented and nonimplemented groups of states by their position on the NSR rule changes. Note that of the implemented states (where the rule changes were implemented first) only South Dakota supported the NSR rule changes.

TABLE 5-2 Legal Challenges to 2002 NSR Rule Changes by States

Challenged NSR Changes

No Position

Supported NSR Changes

Fully and partially implemented group

CAa, IL, MA, NJ, NY, PAa

AZa, HI, MI, MN, NVa, WA

SD

Nonimplemented group

CT, DE, ME, MD, NH, RI, VT, WI, DC

AL, AR, CO, FL, GA, IA, ID, KY, LA, MO, MS, MT, NC, NM, OH, OK, OR, TN, TX, WV, WY

AK, IN, KS, ND, NE, SC, UT, VA

aPartial implementation of state PSD areas in SIP.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

Policy Perception by Industrial Firms

As discussed above, it will be possible to identify when the NSR rule changes became legally valid in different areas, but it may take some time before the affected firms change their investment decisions. Information about the actual timing of the changes should be supplemented by information about how quickly firms recognized those changes. Several approaches may provide useful results to researchers:

  • Discussions about the NSR rule changes with decision makers at regulated firms to identify when investment behaviors changed.

  • Surveys of firms to identify when they recognized the NSR rule changes, perhaps using a series of surveys over the time period that the new NSR rules are being adopted in different states, to see whether firms’ perceptions of the date of change correspond to the actual differences across states in rules change.

  • Discussions with state regulators to confirm information obtained from the respondents at firms. This could also involve collecting information across states to confirm the characterization of different states as early and late adopters of the new NSR rules.

  • Identification of efforts made in each state to inform affected firms of the change, helping to explain why firms in different states responded more or less quickly to the NSR rules change.

Permit Data

It may seem natural to use NSR permit data to identify changes in outcomes related to the NSR rule changes. The permits include information on allowable emissions and required emission reductions. Such permit data have been used in the past (NESCAUM 2004; FLDEP 2005) to predict increases in allowable emissions if less stringent NSR requirements were adopted. These calculations assume that the same set of investment projects would be getting permits and that they would take advantage of any weakening of NSR requirement to increase their emissions as much as possible. As noted earlier, changes in NSR rules could encourage additional investment in new productive equipment that may be cleaner than the older equipment it replaces. It is possible that enough new, emission-reducing investments would occur that overall emissions would be reduced even if a few investment projects would be able to take advantage of the weaker NSR rules to proceed with projects that might entail increases in emissions, or at least smaller emission reductions than they would have needed under the prerevision NSR rules.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

More useful for analysis would be information about minor state construction permits that would still be required of projects that no longer needed NSR permits. If no additional investment activity were needed because of the change in NSR rules, we might see a small increase in minor permits as fewer projects required NSR permits. A substantial encouragement of new investment activity (as anticipated by proponents of the NSR rule changes) would be seen in an increase in minor permits that was considerably larger than the decrease in NSR permits.

The econometric analysis would compare the number and type of permits required in each state, both implemented and nonimplemented, in each year. The analysis would control for other factors that might affect firms’ investment decisions among states or over time (such as changes in economic conditions in the states or nationally). EPA has collected some data on NSR permits, and some states maintain permit databases that could support econometric analyses of their minor permits. The analysis would look for changes in the total number of permits approved each year (NSR and minor) around the time of the rule changes and changes in the relative numbers of NSR and minor permits.

The major hindrance to doing such an analysis is the limited amount of permit data available, especially in the minor state permit databases, as noted in Chapter 3. Table 5-3 shows information from the existing EPA dataset of NSR permit data for 1997-2002. Even over that 6-year period, most states have relatively few NSR permits issued. Many states keep their permit data in paper form, and the final permit document is sometimes accessible electronically as a portable document format (PDF) or word-processing file. State permit databases tend to be idiosyncratic, having been developed by state regulatory agencies with no particular effort to be compatible with other states’ databases. Of the 13 states in the implemented group, only four were clearly described as having electronic permit data. Of the 38 states (and the District of Columbia) in the nonimplemented group, 18 reported some electronic permit data. An example of a compatible database in use is the TEMPO system (an ORACLE database, developed by AMS, that combines all information about a facility with permit, inspection, and compliance data). The TEMPO system is being used by Kentucky, Maryland, and New Mexico, all in the nonimplemented group, although some of the states began using it only recently.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

TABLE 5-3 Permit Data by State (1997-2002)

 

Number of NSR Permits

State Permit Data Availabilitya

 

Total

Electricity-generating Sector

Manufacturing Sectors

Implemented states

AZ

26

26

0

CA

52

31

18

Paper recordsb

HI

13

13

0

Paper/document

IL

195

117

78

Electronic (ORACLE, 1972+)

MA

15

15

0

Paper/document

MI

190

109

54

Electronic (DOS)

MN

182

19

61

NJ

62

62

0

Electronic (ORACLE)

NV

8

8

0

NY

36

32

4

Electronic (AFS)

PA

115

60

22

SD

2

2

0

WA

22

19

3

Paper/documentb

Nonimplemented states

AK

169

6

1

Paper/document

AL

478

196

186

Electronic (new)

AR

229

83

101

Electronic

CO

82

70

0

Paper, electronic (FoxPro)

CT

22

22

0

Paper/electronic

DC

0

0

0

DE

14

8

6

Paper

FL

443

325

118

Paper/document

GA

123

67

56

Paper

IA

80

27

53

Electronic (1995+)

ID

2

2

0

IN

167

78

89

Electronic (ORACLE)

KS

18

3

15

Electronic (ISTEPS, 1997+)

KY

142

43

99

Electronic (Tempo)

LA

348

79

245

Paper

MD

9

5

4

Electronic (Tempo, old-FoxPro)

ME

31

31

0

Paper/document

MO

99

69

30

Paper

MS

140

88

49

Electronic (2000+)

MT

23

5

18

Paper/electronic

NC

226

103

114

Paperb

ND

5

0

5

Electronic (Access)

NE

37

33

4

Electronic (IIS)

NH

0

0

0

Electronic (FoxPro,ORACLE)

NM

51

51

0

Electronic (Tempo, 1998+)

OH

117

37

80

Electronic

OK

90

90

0

Electronic (Access)

OR

15

4

11

Paper

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

 

Number of NSR Permits

State Permit Data Availabilitya

 

Total

Electricity-generating Sector

Manufacturing Sectors

RI

7

7

0

Paper/document

SC

100

28

72

Electronic

TN

101

12

89

TX

250

153

76

UT

20

11

6

Electronic (Access, ORACLE)

VA

198

120

48

Electronic (ORACLE)

VT

1

0

1

WI

302

102

200

Electronic (ORACLE)

WV

19

11

8

Paper/document

WY

287

23

0

Total permits:

5,363

2,505 (47%)

2,024 (38%)

 

a— = No response to State and Territorial Air Pollution Program Administrators’ survey.

bCounty or district responded, but state did not respond.

NOTE: For paper, permit data available as paper files; for paper/document, permit data may be available in electronic text (PDF or word processor); for electronic, permit data available in database (format and years, where available).

Outcome Data

The final data that will be discussed are those that represent the various outcomes, providing the Y variable for estimating Equation 4-1.2 The initial outcomes to be measured are the investment decisions made by the facilities. The benefits claimed for the NSR rule changes are connected to encouraging investment-generating sufficient investment in new capital equipment, which is cleaner and more efficient than the equipment being replaced, increases economic and energy efficiency, and decreases overall emissions. The discussion of the structural and behavioral econometric model in Chapter 4 noted that it would be difficult to get data on the success or failure of individual

2

We do not discuss here the various control variables (Z) that might be included in the estimations—controlling for factors besides the change in NSR rules (X) that might affect the outcome variables (Y). Different outcome variables will require different control variables; for example, the other determinants of investment spending at a plant might include the overall demand for the industry’s output, tax incentives for investment at the plant, and the owning firm’s profitability. Determinants of a plant’s emissions might include the age of its capital stock and other regulatory pressures faced by the plant (inspections and other enforcement activity). Developing detailed models for each outcome variable is a large part of the effort needed for the econometric analysis being recommended here.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

investment projects, especially if we wished to include project proposals that were ruled out by a facility before a permit request was submitted to regulators. However, there are data on overall capital expenditures related to new plant and structures, which may allow identification of any large swings in investment behavior that happened around the time of the NSR rule changes.

Several other outcomes were identified as part of this committee’s charge: emissions of pollutants, effects on human health, investments in pollution-control and -prevention technologies, and efficiency of facility operations. The discussion below focuses on available sources of data for measuring those outcomes at the facility level. Facility-level data are not always available, but because the implemented group consists of facilities in NAAQS attainment areas in 13 states, published aggregate data at the national or even state level do not provide sufficient detail to distinguish between implemented and nonimplemented facilities. Having facility-level data also allows the analysis to include controls for a facility’s industry and size, which can improve the precision of the estimates. Also, the facility-level data can be linked to allow the analysis of several outcomes simultaneously, for example, seeing whether a facility that substantially increased its spending on new capital equipment after the rule changes also achieved increased efficiency or reduced emissions in later years.

It is possible to use models of pollution effects and atmospheric chemistry to calculate the impact of emissions from a facility on the ambient air quality in surrounding areas (see Chapter 7). Having connected facility emissions to changes in ambient air quality, one could add data on population concentrations to calculate the expected health effects from changes in pollution emissions. Such models are discussed in Chapter 7. These models do not involve econometric analysis of a connection between the NSR rule changes and health outcome data. Instead, the results of past studies that identified a connection between ambient air quality and health outcomes would be combined with (noneconometric) atmospheric models of pollution flow to quantify the health effects. Hence, any facility-level measure of the effects of the NSR rule changes on human health would be derived directly from changes in facility-level emissions. If emissions increased, we would expect air quality to worsen and adverse health effects to increase. Complications arise in the assessment if emissions of some facilities increase and emissions of other facilities decrease. Then the details of the emission-health connection (tied to such factors as the relative population densities near the emission-increasing and emission-decreasing facilities) would be used to determine the overall net effect on health.

Table 5-4 provides information on available data sources to measure the outcome variables for manufacturing facilities. Many of the data are collected by the U.S. Census Bureau. The cornerstones of this data collection

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

TABLE 5-4 Types of Outcome Data for Manufacturing Sector

Outcome

Measure

Source

Investments

New capital spending ($)

Census ASM, CM

Pollutant emissions

Amount emitted (tons/year)

EPA National Emissions Inventory, Continuous Emissions Monitoring System

Pollution control

New PACE capital ($)

Census PACE

Technology use

Abatement technology

EPA Aerometric Information Retrieval System Facility Subsystem

Energy efficiency

Output:energy ratio

Census ASM, CM

Materials efficiency

Output:materials ratio

Census ASM, CM

Labor productivity

Output:workers ratio

Census ASM, CM

Total factor productivity

Output:inputs ratio

Census ASM, CM

are the Annual Survey of Manufactures (ASM) and Census of Manufactures (CM) programs. The ASM collects a basic set of data each year from about 55,000 manufacturing facilities. The sample is size-weighted so that very large facilities are included every year, and smaller facilities are rotated in and out of the sample every 5 years. The CM collects a broader range of data on all manufacturing facilities but is conducted only every 5 years (for the purpose of this study, the relevant years would be 2002 and 2007). In addition, the Census Bureau conducts special surveys to collect detailed information on other topics. Most relevant to our project is the Pollution Abatement Costs and Expenditures (PACE) survey, which collects annual data on capital expenditures and operating costs for pollution abatement with some degree of detail that varies among different types of expenditures and pollution media.

The outcome measures available from Census Bureau data include investment spending, pollution abatement spending, and various efficiency measures. Data on investment spending (new capital expenditures) are broken down into equipment and structure investment. These data can be aggregated to generate a measure of a facility’s capital stock for large facilities with continuous ASM data. The ratio of annual investment spending to total capital stock shows what fraction of the capital stock is being replaced at a facility each year and can be used to test whether the replacement rate increases or decreases after the NSR rule changes.

The PACE survey data include information on the amount of new investment in air-pollution abatement capital. These data allow tests for increases or decreases in the amount of air-pollution abatement investment after the NSR rules changed. Some limitations of the PACE data may affect the analysis. First, and most seriously, the PACE survey was not conducted

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

during the 2000-2004 period. (The PACE survey was conducted annually from 1973 to 1994; then it was halted for financial reasons. A revised version of the survey was done for 1999, and annual data collection has been resumed, starting with data for 2005.) This makes it difficult to benchmark the pollution abatement investment to the pre-rule-change period and limits the number of observations on the implemented group (losing data for 2003 and 2004). If the PACE data collection resumes on schedule with the collection of data for 2005, there will be 2 years (2005 and 2006) of differences between nonimplemented and implemented groups for analysis followed by a transition (in 2007 or later) of the nonimplemented states into the implemented group—possibly in different years in different states.

The Census Bureau data are particularly helpful in calculating various efficiency measures, including energy efficiency, the focus of most of the discussion of efficiency in other chapters of this report. The energy efficiency of manufacturing firms would be measured as the quantity of fuels and electric energy consumed divided by the real production at the facility. An efficiency index related to life-cycle pollution-prevention outcomes could be calculated in terms of material efficiency as real material input per unit of real production at the facility. The Census Bureau data also include sufficient information to calculate more general efficiency measures, such as labor productivity (real output per production worker hour) or total factor productivity (real output per unit of total input). The measure of total factor productivity, for which total input is a weighted average of all inputs (including capital, materials, and labor) provides an overall indicator of the effect of the NSR rule changes on the efficiency of production and measure of the overall costs (or benefits) of the rule changes.3

Through the efforts of the Census Bureau’s Center for Economic Studies, the ASM and CM data have been linked at the facility level in the Longitudinal Research Database, as described in McGuckin and Pascoe (1988). The census data also include facility-level links to the PACE survey data, and this link allows investment in pollution-control equipment to be included in the analyses. These data have been used by numerous researchers in recent years to measure the effects of environmental regulatory pressures on a variety of business outcomes (including Levinson [1996] and Becker and Henderson [2000], cited in Chapter 4).

The Census Bureau facility-level data are confidential and are available only to researchers on approved projects and accessible only through the network of Census Research Data Centers (RDCs). The cost of running the network requires that projects pay laboratory fees to an RDC. Considerable time is needed to prepare a research proposal and get approval, in addition

3

Gray (1987) discusses the use of total factor productivity to measure the net benefits (or costs) of regulatory changes.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

to the efforts required to merge the required data and carry out the analyses. Any research project that would use the Census Bureau data would therefore have to be appropriately budgeted with respect to both time and money. One possibility is to incorporate the research into an existing project if there is sufficient lead time and researcher interest.

Similar economic production data on individual electricity-generating plants are collected by the U.S. Energy Information Administration (EIA). Access to the EIA data is easier to arrange than access to the Census Bureau data because much of the electricity-generating sector has been regulated as a public utility, and investment and production have become matters of public record as part of rate-setting deliberations by state utility boards. Because of the public nature of the EIA data, many researchers have used databases compiled at the facility level to analyze the effects of environmental regulations on production, investment, and productivity at electricity-generating facilities (Maloney and Brady [1988] and Nelson et al. [1993], as cited in Chapter 4).

Information on economic outcome measures for nonmanufacturing, nonutility facilities4 is much less complete. The Census Bureau collects some data on nonmanufacturing industries in its Economic Census every 5 years, but data on most industries outside manufacturing are not collected in the intervening years. That would make it relatively difficult to perform econometric analyses aimed at identifying differences between facilities in the implemented and nonimplemented groups during the period 2003-2006. Fortunately, the NSR permit numbers presented in Table 5-3 (and discussed in more detail in Chapter 3) show that 85% of existing NSR permit activity occurs at either electricity-generating facilities or manufacturing facilities, so this data limitation should not be a serious impediment in measuring the overall effects of the NSR rule changes.

Emission data at the facility level is collected by EPA in the National Emissions Inventory (NEI). The NEI data are collected in great detail on both large and small sources of pollution every 3 years (1999, 2002), but some of the data focus on the larger facilities, updated annually. In addition to the NEI data-collection effort, emission data in recent years on some major sources (notably large electricity-generating facilities) are collected in the Continuous Emissions Monitoring System and should provide especially accurate emission measures. Data on the pollution-abatement equipment in place at a facility are also available in EPA databases, such as the Aerometric Information Retrieval System Facility Subsystem database.

Those outcome measures are not immediately available, so it would be some time before analysis could begin. The Census Bureau datasets take about 2-3 years to become available to researchers: information from the

4

SIC 13—oil and gas extraction—has the largest number of NSR permits in this area.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

2002 Economic Census is being released in 2004-2006; the most recent ASM available at this writing (in late 2005) is from 2003, the first year in which changes might be observed. EPA’s NEI data take a similar period before being released.5 Because the NSR rule changes will not affect facilities in the nonimplemented group of states until 2007, it seems reasonable to project that a complete analysis will be feasible some time in 2009 or 2010.

GAO (2003) noted EPA’s lack of data for measuring the emission effects of the NSR rule changes. The report recommended that EPA work with state and local agencies to identify data sources and monitor emissions to better measure impacts on emissions in the future. The report also noted that EPA agreed with the report’s recommendations.

UNCERTAINTY AND STATISTICAL POWER

The committee’s charge included providing estimates of the amount of uncertainty associated with the estimated effect of the NSR rule changes. One advantage of an econometric model is that it provides both a point estimate of an effect and a measure of its statistical precision. The precision will depend on the number of data points in the sample and the variability in the outcome measures across the implemented and nonimplemented groups, which results in a type of model uncertainty that must be considered. The underlying Census Bureau and EPA databases contain information on thousands of facilities, but considerable data variability may not be covered by the explanatory variables (Z) in the model. With many kinds of facilities in the database (which vary by industry and size), some care may also be needed in deciding whether to apply a single set of estimated coefficients to all facilities.

An example of the calculations of sample variability is given in Gray (1993) as part of a discussion of using Census Bureau data to measure the effects of Occupational Safety and Health Administration (OSHA) regulatory activity on investment spending and plant efficiency. That study focused on the costs of complying with new OSHA regulations, increasing capital investment and lowering productivity at the affected plants, but the underlying assumption was similar: some plants are affected by regulatory changes, and others are not. The statistical calculation used is a test for the difference between the means of the affected and unaffected groups:

5

On the basis of information posted in September 2005 on EPA’s Web site, the 2005 NEI data update will receive less effort than usual, so that resources can be focused on a re-engineering of the NEI data-collection process aimed at the 2008 NEI (EPA 2005d). States will still be expected to submit emission data on large sources, but the data will be less standardized by EPA. The 2005 NEI data are projected to be available by December 2006.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

where m is the mean, s is the variance, and n is the number of observations in each group. If the test statistic, T, exceeds 2, it is taken as evidence of a statistically significant difference between the means of the two groups6 and that would indicate that the OSHA regulation had a significant effect on compliance costs as measured by investment spending or productivity levels.

An alternative use of the T calculation is to estimate how precisely a given dataset could distinguish between the means of two groups. Gray (1993) assumed that the variance in each group is equal to S and calculated how large the difference M between the group means would need to be to give a significant T test (at the 95% level):

Estimates of S were obtained from Census Bureau plant-level data by looking at the unexplained variability from econometric models of investment and productivity. The analyses were done in logarithmic form, so S was expressed in terms of percentage variation. Investment was much more variable than productivity across plants: S was 1.5 for investment and 0.2 for productivity. If there were 500 plants in each group, the value of M for investment would be 0.19 = (2)(1.5)[(1,000/(500)(500)]1/2. A significant value of T would be obtained if the investment was 19% greater in affected plants than in unaffected plants; the additional investment would presumably be driven by the added compliance costs of the regulation. For productivity, a significant T would result from affected plants having productivity 2.5% lower than unaffected plants.

Combining the variance information with estimates of the numbers of plants in the affected and unaffected groups, Gray (1993) was able to calculate which of several OSHA standards were likely to have significant effects on compliance costs, given ex ante estimates of the magnitude of compliance costs for each standard. In the case of the NSR rule changes, there is a wider array of outcome variables to consider, but the fundamental nature of the statistical tests is the same: get estimates of the amount of variation in the data for a variable and then see whether the difference between the affected and unaffected plants in their average outcomes exceeds the critical amount of variation for the T test.

We focus our attention here on the same investment and productivity measures as examined in Gray (1993) to use the same estimates of variation in investment and productivity. The change in the calculation comes in the number of plants in the affected and unaffected groups. For the NSR rule

6

If the true means of the two groups were the same and we drew na and nu observations at random from the two groups, the T statistic as calculated here would exceed 2.0 less than 5% of the time.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

changes, the calculation involves the number of plants that are large enough to be affected by the rule changes and their allocation into the two groups (implemented and nonimplemented). Using data provided by EPA on the number of major sources in each state, we calculate a total of 7,890 plants with emissions of any criteria pollutant exceeding 100 tons/year (tpy). (Strictly speaking, only facilities in 28 source categories face a 100-tpy cutoff, and others face a 250-tpy cutoff—but those 28 categories contain nearly all major facilities.) Of those, 1,888 were in the implemented group. We assume that only half the plants (in both groups) would have Census Bureau data available. For investment, we get

so we would need an increase in investment of about 11% to observe a significant difference between affected and unaffected plants (for productivity, the comparable value for a significant difference is 1.5%).

CONCLUSIONS AND RECOMMENDATIONS

The best econometric approach to measure the effects of the NSR rule changes appears to be to estimate a reduced-form model, comparing outcomes (such as investment spending and pollution emissions) across sets of facilities in states that differed in the effective date of the NSR rule changes. The data for such analyses will not become available until some years after the fact, and this will delay analyses. The NSR rule changes began to be implemented in 2003 but will not take effect in most states until 2007 or later. A complete econometric analysis may not be feasible until 2009 or 2010. Furthermore, any such analysis will be subject to measurement error because of concerns about the uncertainty of whether the rule changes will hold up to court challenge and the possibility that anticipation of the rule changes could affect the timing of investment decisions. Diversity among states in the timing and magnitude of the NSR rule changes will help researchers to get a better measurement of the effects of the changes, and this will make possible a reduced-form analysis of the effects of the changes on investment and emissions.

Carrying out such an analysis efficiently will require preparation. First, the committee recommends that data be collected for each state on the date when the NSR rule changes become applicable for facilities in that state in both attainment and nonattainment regions. That will require tracking EPA’s approval of revised SIPs. In addition to the precise legal dates, qualitative data need to be collected, through surveys or interviews with firms and regulators, to identify when firms recognize the NSR rule changes and incorporate them into their investment decision making. It would be especially helpful to gather such information continuously beginning soon,

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×

so that analysis need not rely solely on retrospective surveys after the NSR rule changes are firmly entrenched (when firms’ and regulators’ recollections about what they knew and when they knew it may be colored by their knowledge of what eventually happened).

Second, the committee recommends that a suitable database on NSR and minor state permits be collected. Perhaps EPA could work with state agencies to develop a consensus on the information such a database should include facilitating the development of a national permit database and permitting cross-state analyses for the impacts of NSR rule changes as well as analyzing other regulatory activity.

EPA has done some updating of its NSR permit database beyond the initial 1997-1999 data collection and should be encouraged to continue the updating. Although that database cannot provide complete measures of the effects of the NSR rule changes on outcomes, it constitutes a useful description of where NSR-covered activity continues. Much greater effort will be needed to assemble a useful collection of data on minor state permits. States that have not yet developed permit databases of their own could be encouraged to adopt a common database layout or at least to design their database to make it easy to export permit information to a compatible national permit database. States that do have permit databases should develop conversion programs to export their data into a national permit database.

Third, the committee recommends that resources be made available for analyses of the effects of the NSR rule changes on investment behavior and other outcome measures. Census Bureau data appropriate for facility-level analyses are already being collected, but funding would need to be made available for researchers with Census Bureau-approved projects in secure RDCs to be able to analyze the effects of rule changes on the basis of facility-level data. There is enough time to develop a research protocol before adequate data are available, and these analyses could be an important element in evaluating NSR and related regulations.

Finally, the limitations of an econometric approach should be recognized. If firms respond fairly quickly (within a year or two) to the NSR rule changes with a considerable expansion in investment activity, the change in investment should be noticeable. Conversely, it may be possible to provide upper bounds for the effects of the NSR rule changes on investment (we might observe an average increase in investment of 2% and be able to rule out an impact greater than 20%). Still, it is likely that there will be relatively wide bounds on the estimated effects, especially in the initial years in states where relatively few facilities are being affected by the rule changes.

Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 140
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 141
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 142
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 143
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 144
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 145
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 146
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 147
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 148
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 149
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 150
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 151
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 152
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 153
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 154
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 155
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 156
Suggested Citation:"5 Econometric Analysis." National Research Council. 2006. New Source Review for Stationary Sources of Air Pollution. Washington, DC: The National Academies Press. doi: 10.17226/11701.
×
Page 157
Next: 6 Assessing Potential Effects on the Electricity-Generating Sector »
New Source Review for Stationary Sources of Air Pollution Get This Book
×
Buy Paperback | $88.00 Buy Ebook | $69.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Clean Air Act established a pair of programs—known as New Source Review (NSR)—that regulate large stationary sources of air pollution, such as factories and electricity-generating facilities. Congress then asked the National Research Council to estimate the effects of NSR rule changes made in 2002 and 2003 in terms of the effects on emissions and human health, and changes in operating efficiency (including energy efficiency), pollution prevention, and pollution-control activities. New Source Review for Stationary Sources of Air Pollution provides insights into the potential effects of the rule changes on national emissions from the electric power industry. Although this book focuses on the 2002 and 2003 rules, its analytic framework applies to other possible changes in NSR and to other regulatory contexts. Helpful, in that it outlines the data-collection efforts needed to assess the impact of the NSR rules, the book recommends EPA and other government agencies undertake and sustain the recommended methods.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!