Designing a Market Basket for NAEP: Summary of a Workshop (2000)

Summary indicators are used in many contexts other than education. The Committee on NAEP Reporting Practices was interested in learning more about them and the experiences of other fields in making the results of complex summary measures understandable to the public. For example, although few people know how the Dow Jones Industrial Average Index of 30 “blue-chip” U.S. stocks is computed, most recognize it as an indication of the status of the stock market and understand what it means when the Dow Jones goes up or down. Similarly, calculation of unemployment rates is based on complex processes, but the end result is a single number that the public believes has immediate meaning.

Because parallels have been drawn between the CPI and the NAEP market basket, the committee arranged for a briefing on the CPI. At the committee's invitation, Kenneth Stewart from the Bureau of Labor Statistics (BLS) addressed committee members and workshop participants about the processes and methods used for deriving and utilizing the CPI (http://www.stats.bls.gov). Stewart's remarks are summarized below.

Stewart explained that the CPI is a measure of the average change over time in the prices paid by urban consumers in the United States for a fixed basket of goods in a fixed geographic area. The CPI is widely used as an economic indicator and a means of adjusting other economic series (e.g.,

retail sales, hourly earnings) and dollar values used in government programs. It is the most widely used index for measuring inflation and aids in the formulation of fiscal and monetary policies and in economic decision-making. Stewart noted that the CPI measures the rates of changes in prices, not absolute levels.

The BLS develops the CPI market basket on the basis of detailed information provided by families and individuals on their actual purchases. The market basket is reconstructed every decade using government survey data. The current CPI market basket is based on the Consumer Expenditure Survey conducted between 1993 and 1995. Approximately 30,000 families responded to this survey, providing information on their spending habits through quarterly interviews and by keeping comprehensive diaries of purchases.

Using the information supplied by these families, the BLS classified their expenditures into more than 200 item categories arranged into eight major groups: food and beverages; housing; apparel; transportation; medical care; recreation; education and communication; and other goods and services. The BLS then constructed a market basket of goods and services and assigned each item in the market basket a weight, or importance, based on its share of total family expenditures.

The BLS produces the monthly CPI using a sampling process. First, using decennial U.S. Census data, the BLS specifies a sample for the urban areas from which prices are to be collected and chooses housing units within each area for inclusion in the housing component of the CPI. A second sample of about 16,800 families each year serves to identify the places (outlets) where households purchase various types of goods and services. The final stage in the sampling process involves selecting the specific detailed item within each item category to be priced each month in a particular outlet. This selection is made using a random probability sampling method that reflects an item's relative share of sales at that particular store.

In addition to monthly release of the national CPI estimates, the BLS publishes monthly indexes for the four principal regions of the nation (Northeast, Midwest, South, and West), as well as for collective urban areas classified by population size. The BLS also publishes indexes for 26 local areas on monthly, bimonthly, or semiannual schedules. An individual area index measures how much prices have changed over a specific time interval in that particular area. However, due to the specifics of the design and sampling, indexes cannot be used for relative comparisons of the level of prices or the cost of living in different geographic areas. In fact, the composition of the market basket generally varies substantially across areas because of differences in purchasing patterns.

In response to Stewart's presentation, workshop participants attempted to draw parallels between the CPI and the NAEP market-basket proposal. In doing so, they realized that the construction and measurement of the CPI market basket is somewhat different than that envisioned for the NAEP market basket. Creating a NAEP market basket using procedures modeled after the CPI would involve a process like the following: identify samples of teachers to participate in a survey; collect information from teachers (or schools) on the content and skills that they teach; classify the content and skills and sample from this listing to create the “market basket;” then, test students to determine their level of performance on this market basket of content and skills. This is quite different from the approach planned for the NAEP market basket. While the NAEP frameworks are developed by committees of experts familiar with school-level curricula, they are not based on surveys of what schools actually teach.