Of these, integrity, interpretability, and comparability are largely covered by other characteristics on our longer list, so under the heading of “quality” we focus on utility and accuracy.

Cost The benefits from improvements in conceptual fit or other aspects of data quality have to be weighed against the costs. Even when existing data sources are used, it is desirable to avoid incurring significant costs of obtaining data in a format suitable for the allocation process.

Fairness The allocation formula should be perceived as being fair. By fair, it is generally meant that the data used in the formula should be free from perverse manipulations, open to review, and should distribute resources equitably across governmental units. The formula itself should be replicable (see “Transparency,” below).

Stability The data should be relatively stable over time. They should not be subject to extreme volatility or to large, unexplainable variation. However, in the context of Title III allocations, an appropriate balance needs to be struck between the stability of the data series and responsiveness to real annual changes in the size and characteristics of the ELL or immigrant population.

Insensitivity to Policies and Methodological Differences The data series should be relatively insensitive to differences that arise from administrative practices and policy differences between agencies and jurisdictions that provide the data and that benefit from those differences. In the context of allocations for Title III funding, if states X and Y have the same distribution of English language proficiency, but state X sets standards for program entry and exit that result in a larger fraction of its students designated as eligible for Title III services, the data series should take those differences into account so that they do affect the allocations. (This is not to suggest that states are or could be “gaming” the system. In fact, there is little incentive to game the system, since the cost to states and local education authorities of administering and conducting the ELL program generally exceeds the funds received from the federal government. In each year of the reauthorized Title III Program, the amount of money involved was small enough that it didn’t create an incentive to states to this kind of strategic behavior.)

Transparency Users should be able to have access to and be able to understand the assumptions, methods, and results so that a knowledgeable user could readily reproduce the information, within the constraints of protecting the confidentiality and privacy of the subjects (see U.S. Census Bureau, 2010, p. 167).

Comparability The methodology by which the estimate is derived should be similar across geographical units.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement