Page 53
PART II
Panel Report
Following the April 2000 Workshop on Formulas for Allocating Program Funds, described in Part I of this report, the Panel on Formula Allocations was established within the Committee on National Statistics to conduct a study of formulas used to allocate federal and state funds. 1 The study focuses on the statistical estimates used as inputs to formulas, data and methods for estimating these inputs, the features of the formulas, and how estimates and formula features interact in ways that affect outcomes. The panel study is considering in greater breadth and depth how the properties of estimates, such as sampling error and response quality, can affect their use in formulas that have a variety of features (e.g., thresholds for eligibility and hold-harmless provisions).
The purpose of the study is to provide a detailed assessment and several illustrations of how formula features can interact with estimator properties in ways that affect the likelihood of program goals being met; it is not designed to recommend changes in existing formulas, new formulas, or the use of particular datasets. The work of the panel will include analyses of allocations with a variety of program provisions and estimators for programs that cover a range of areas, such as education, community development, public health, and others.
1Formulas play a central role in cost of living escalator clauses, labor contracts, child support rules, income tax collection, and congressional apportionment. Although these can serve as interesting and relevant analogies, we do not consider them in this report.
Page 54
In this initial report, the panel summarizes themes identified in the workshop and in its first three meetings, highlights the principal issues it intends to address, and outlines anticipated activities. Themes and issues are presented in Chapter 5 and anticipated activities are described in Chapter 6.
Page 55
5
Themes and Issues
THE FORMULA ALLOCATION PROCESS
Programs that allocate federal funds to states and localities address three principal goals: delivering funds to the right places, implementing programs and delivering services, and producing the desired outcomes (e.g., health improvement, educational attainment). Formulas have a major, direct role in achieving the first goal; a substantially smaller, indirect role in achieving the second goal; and essentially no role in achieving the third goal, except through the first two goals.
There are three models for the way that Congress interacts with Executive Branch agencies in designing programs to allocate federal funds: (1) Congress can legislate the exact amount to be received by each state or other political subdivision; (2) Congress can pass legislation that includes a detailed formula for determining allocations; or (3) the legislation can describe program objectives and leave it to the Executive Branch agency to determine the formula or other process for allocation. Occasionally, an agency receives funds for research on improving the formula inputs.
An example of the first approach is the Capitalization Grants for State Revolving Funds Program of the Environmental Protection Agency; the Medicaid program is an example of the second. The Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) and Title I of the Elementary and Secondary Education Act provide two examples of the third approach. In recent years, the U.S. Department of Education has
Page 56
improved the accuracy and timeliness of the estimated numbers of school-age children in poverty for Title I allocations (via the U.S. Census Bureau's Small-Area Income and Poverty Estimates Program [SAIPE]). However, the programmatic effects of these improvements have been largely negated by legislated hold-harmless provisions. Such post hoc modifications may serve short-term political goals. Nevertheless, when total program funding is approximately level, avoidance of funding reductions to some political subdivisions can be accomplished only at the expense of other jurisdictions that would have received additional funds.
When a program is authorized (or reauthorized), the explicit nature of a formula facilitates the legislative process and increases actual and perceived fairness. Scenarios can be evaluated and the formula specification adjusted. A formula can make the allocation process more transparent and promote full disclosure. Once implemented, a formula may be fine-tuned or considerably modified. Communicating the rationale for and evaluating an allocation process without a formula is far more difficult.
In the evaluation of programs and allocations, technical statistical issues are important, but so are many other administrative and political aspects of a program. If a program fails to attain its stated goals, the fault may lie in requirements dictated by the legislation, in the data inputs to the allocation formula, in the allocation process, or in the program services. There is a complicated interaction between formula inputs and features, with the possibility for unintended and unanticipated, cascading consequences of various combinations. For example, hold-harmless provisions attempt to balance the goals of changing allocations when necessary and maintaining funding stability. As discussed in Part I, Chapter 3, hold-harmless provisions may systematically and persistently help or harm areas based on the statistical properties of formula inputs rather than changes in true need. The absolute and relative impacts of hold-harmless provisions depend on whether total funds for a program are capped or can adjust to the formula-based financial need.
DESIGN AND EVALUATION OF FORMULA ALLOCATION PROGRAMS
As the foregoing indicates, design, implementation, and evaluation of a funds allocation program is considerably more complicated than a superficial view would suggest, entailing synthesis of statistical information from several sources and complex processing of the results. Many programs use
Page 57
formulas with several elements, such as need, capacity, and cost of services. Furthermore, as pointed out by Zaslavsky and Schirm (2000), there is a degree of arbitrariness in what constitutes the inputs and what constitutes the formula. Legislation can direct that “appropriate” statistical analyses be used to prepare inputs (as is the case for Title I Education) or such analyses can be built into the formula (as is the case for Medicaid). For example, the law might specify that allocations are to be based on a three-year moving average and that each year's estimate is to be based on a single year's data. However, the same effect would be obtained if the formula called for an estimate for a single year but, based on a statistical assessment, the estimate for that year was calculated as a three-year moving average.
This complexity calls for a framework for evaluating program performance. It should include measures of monetary allocation success (possibly including loss functions that compute equity/inequity measures), effective use of funds for program-specified services, and beneficial impacts of services, and it should be used to assess performance of current programs and to recommend changes. Developing such a framework will be challenging. Challenges include defining equity, especially when formulas include multiple elements of need, fiscal capacity, and effort, and balancing equity with efficiency and political considerations. Although evaluation of an individual allocation program is of primary importance, evaluating how different federal allocation programs interact and how they impact programs that allocate state funds for similar purposes is also important.
During the legislative process, the U.S. General Accounting Office and the Congressional Research Service, when requested by congressional staffs, provide simulations that evaluate formula options. There are institutional mechanisms for obtaining additional expert input during the development, evaluation, and modification of allocation formulas. In those instances when it is left to an Executive Branch agency to determine the formula or other process for allocation, the agency will need similar capacity to implement and evaluate formulas and to work through the public comment process. State agencies have similar needs, both in developing their own allocation formulas and in understanding how the federal formula allocation programs work. The parties to these processes do interact, sharing problems and solutions, but additional communication and coordination could be beneficial.
Many take for granted that efforts to improve the quality of formula inputs (accuracy, conceptual relevance, timeliness) are desirable. But the benefits may not justify the costs. For example, the U.S. Census Bureau's
Page 58
small-area income and poverty estimates are updated on a two-year schedule. The cost of annual updating may not be justified by the consequent improved performance; such trade-offs need to be evaluated. To get started, one can consider the situation wherein there is no sampling error and the model is correct and evaluate the improvement relative to the current situation. Performance measures will be needed to conduct such an evaluation.
DATA SOURCES
A few key data sources such as the decennial census, the Current Population Survey, and Internal Revenue Service records are widely used to support formula allocation programs. In addition to these data sources currently in use, new information sources have the potential to improve formula inputs. The American Community Survey (ACS), which is intended to replace the decennial census long form, would be a major new data source that might be used in estimating inputs if the survey is implemented as planned. With data from census 2000 becoming available in stages and the ACS pending, an immediate and high priority should be given to developing recommendations on how to make a smooth transition to these and other data sources and how to evaluate the impact on allocations of introducing new data sources.
Guidance is also needed on the statistical limits of available information. No formula will work perfectly, especially at a fine level of spatial, temporal, and demographic disaggregation. Inevitably, there will be short-term statistical fluctuations in estimates used as formula inputs. While reduction of these statistical fluctuations is an important objective, considerable attention must be paid to identifying and reducing persistent biases for identifiable geographic and demographic subgroups to keep allocations aligned with true needs (see National Research Council, 2000b, for examples).
In some circumstances a lower limit to the population size of areas for which the federal agencies are required to produce population and income estimates has been proposed. For example, the 1980 Panel on Small-Area Estimates of Population and Income recommended that the U.S. Census Bureau not provide postcensal population estimates for places with population below a (to be determined) threshold (National Research Council, 1980). However, the Title I education program now mandates allocations to school districts, and many school districts have small populations. Since some process (formal or informal, sophisticated or naive) will be used to determine school district allocations, it may be preferable to base them on a
Page 59
standardized approach using the best available information and statistical analysis. Properties of the estimates and their effect on allocation programs should be assessed and, in the spirit of openness, the estimates should be made public.
Administrative data are an important element in developing model-based statistical estimates used as formula inputs (for example, Internal Revenue Service and food stamp records are used in the U.S. Census Bureau's SAIPE Program). Evaluation of the potential for modifications in the collection and storage of program and administrative data to improve their performance as inputs to allocation formulas will be beneficial. For examples of possible modifications, see National Research Council (2000a:Ch.5).
THE ROLE OF THE STATES
While policy makers, program administrators, statisticians, and others at the federal level give their attention to formulas and formula inputs, some recipients of federal funds, especially states, are spending significant resources on efforts to increase their allocations. As discussed in Chapter 2, California changed its definition of school attendance to achieve a per pupil expenditure estimate that increased its share of Title I education funds, and the state protested as improper the use of manufacturing wages in the mental health/substance abuse block grants. These examples show that states will challenge definitions and data sources to maintain or increase their allocations. It is therefore important to document what individual states are doing to improve their shares of formula funds and to consider what federal agencies can do to ensure a level playing field.
Further consideration of the role of states in fund allocation processes suggests that adjuncts to the use of national databases should be considered for developing formula inputs. For example, a program could require that a state or county produce “best estimates” and use these in a formula. This approach has the potential to produce more timely and targeted estimates. In fact, many funds allocation programs currently allow flexibility in state-provided information, especially for distributing federally allocated funds within the state. If similar flexibility is allowed in providing inputs to the formula for allocating funds among states, inputs should be as immune as possible from manipulation of definitions or data sources.
Formulas also figure importantly in the allocation of state funds to counties, cities, school districts, and other jurisdictions. State funding of
Page 60
elementary and secondary education substantially exceeds the contribution from the Title I and other federal education programs. Since the early 1970s, perceived disparities between school districts have led to many efforts, in state legislatures and in the courts, to develop more equitable allocation processes (National Research Council, 1999b). A recent court decision by a New York State judge declared the state's method of financing public schools illegal and set a September 2001 deadline for the state to revise its formula (Goodnough, 2001).