PAPERBACK
\$36.75

## 4 Current and Proposed Measures

What was the high school dropout rate last year? What was the graduation rate? Most people believe that quantifying these rates should be a straightforward task. Intuitively, it might seem that calculating the dropout rate simply means dividing the number of students who drop out by the total number of students in the cohort and, similarly, that calculating the graduation rate simply means dividing the number of students who graduate by the total number in the cohort. Intuitively, it might also seem that once one of the rates is obtained, the other can be calculated by subtracting it from 100.

As those who work in this area can attest, calculating the rates is not that simple. The rates can be calculated from a variety of different data sources using a variety of different methods. These differences can lead to discrepant estimates of the rates. For instance, consider the high school completion rates published for the 2005-06 school year. The U.S. Department of Education reported that approximately 73 percent of public high school students graduated on time that year (Snyder, Dillow, and Hoffman, 2009, p. 3), and Editorial Projects in Education (2008, p. 28) reported a similar figure of 69 percent. However, the U.S. Department of Education also reported a dropout rate for 16- to 24-year-olds of only 9 percent in 2006 (Snyder, Dillow, and Hoffman, 2009, Table 105), and the Annie E. Casey Foundation (2009, p. 64) reported a dropout rate for 16- to 19-year- olds of only 7 percent. How should a completion rate of between 69 and 73 percent be interpreted in light of a dropout rate below 10 percent?

Why do estimates of the high school completion rate and the high school dropout rate differ so much from one another? How can these disparate estimates be reconciled? The disparities can be traced to three sources:

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 43
4 Current and Proposed Measures W hat was the high school dropout rate last year? What was the gradu- ation rate? Most people believe that quantifying these rates should be a straightforward task. Intuitively, it might seem that calculating the dropout rate simply means dividing the number of students who drop out by the total number of students in the cohort and, similarly, that calculating the graduation rate simply means dividing the number of students who graduate by the total number in the cohort. Intuitively, it might also seem that once one of the rates is obtained, the other can be calculated by subtracting it from 100. As those who work in this area can attest, calculating the rates is not that simple. The rates can be calculated from a variety of different data sources using a variety of different methods. These differences can lead to discrepant estimates of the rates. For instance, consider the high school completion rates published for the 2005-06 school year. The U.S. Department of Education reported that approximately 73 percent of public high school students graduated on time that year (Snyder, Dillow, and Hoffman, 2009, p. 3), and Editorial Projects in Education (2008, p. 28) reported a similar figure of 69 percent. However, the U.S. Department of Education also reported a dropout rate for 16- to 24-year- olds of only 9 percent in 2006 (Snyder, Dillow, and Hoffman, 2009, Table 105), and the Annie E. Casey Foundation (2009, p. 64) reported a dropout rate for 16- to 19-year- olds of only 7 percent. How should a completion rate of between 69 and 73 percent be interpreted in light of a dropout rate below 10 percent? Why do estimates of the high school completion rate and the high school dropout rate differ so much from one another? How can these dispa- rate estimates be reconciled? The disparities can be traced to three sources: 43

OCR for page 43
44 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES (1) differences in what various estimates are designed to accomplish, (2) dif- ferences in the conceptual and technical definition of both the numerator and the denominator used to calculate the rates, and (3) differences in the accuracy of the data used to produce them. Understanding these three sources of differ- ences is key to making sense of the resulting rates. This chapter first discusses the different purposes of the estimates and the sources of data used in calculating the rates. We then discuss the different types of rates that can be calculated. We close the chapter by discussing the impor- tance of aligning the choice of a rate with the purpose it will serve. The chapter draws extensively from papers prepared for the workshop by Rob Warren, with the University of Minnesota, and Elaine Allensworth, with the Consortium on Chicago School Research (Allensworth, 2008; Warren, 2008). DIFFERENT PuRPOSES OF THE ESTIMATES One major source of differences in estimates of dropout and completion rates is the question they were designed to answer. Analysts operationalize high school completion and dropout rates in different ways because they have differ- ent conceptual or practical reasons for making those measurements. There are three chief uses of high school completion and dropout rates. The first use is to describe the amount (or lack) of human capital in a popu- lation. In this case, the rates characterize an attribute of society: they quantify the share of people who bring particular credentials and skills to the labor force. The second use is to describe the “holding power” of schools. In this case, the rates answer questions about schools; they characterize their success at moving young people from the first day of high school to successful completion (see Hartzell, McKay, and Frymier, 1992). The third use is to describe students’ success at navigating high school from beginning to end. For this purpose, the rates answer questions about individual students themselves; they measure how successful students are in progressing from the first day of high school to successful completion. If the goal of the rate is to describe the amounts of human capital in a population, the timing of high school completion—how long ago or at what age people completed high school—is not of critical importance. Nor, for some purposes, does it matter exactly how young people complete high school—by obtaining a diploma, a General Educational Development (GED) credential, or a certificate of completion, completing an adult education program, or some other way. For other purposes, however, how students complete high school is critical because research suggests that students who fail to earn a regular high school diploma are less competitive in the labor market compared with gradu- ates (Heckman and Rubinstein, 2001; Tyler, 2003). Any young person who has completed high school is considered to have acquired marketable capital, regardless of his or her age at the time of crossing that educational threshold.

OCR for page 43
45 CURRENT AND PROPOSED MEASURES For the latter two uses, however, both the timing of high school completion and the manner in which young people complete high school can be impor- tant. For instance, schools may be deemed successful at moving young people through to completion only if they obtain regular diplomas “on time,” typically within four years. Given these differences in intended purpose, it becomes less puzzling to read in the Digest of Education Statistics that “73.4 percent of public high school students graduated on time,” despite the fact that only 9 percent of 16- to 24-year-olds were dropouts in 2006 (Snyder, Dillow, and Hoffman, 2009, p. 3 and Table 109). The former estimate is explicitly intended to describe the share of a cohort of students that has completed high school on time and by obtain- ing a diploma—essentially an attribute of schools. The latter estimate is clearly intended to describe the share of young people who are not gaining the human capital associated with high school completion. Presumably many of the 26.6 percent of ninth graders in fall 2002 who did not go on to graduate from high school with a diploma by spring 2006 were still enrolled or will compete high school later, via a GED or another alternative credential.1 Given the reported 9 percent dropout rate, one might presume that eventually about 91 percent of young people will eventually complete high school one way or another. 2 DIFFERENT DATA SOuRCES Another source of differences in the rates is the data used in the calcu- lations. A number of available data sources can be used for calculating the rates. These data were collected for different reasons using different types of designs—cross-sectional sample surveys, longitudinal sample surveys, cross- sectional administrative data, and longitudinal administrative data. The col- lection method and the reasons for collecting the data can affect the rates that are calculated. In this section, we describe the major data sources used in this country to compute high school dropout and completion rates and discuss their strengths and weaknesses in relation to the three purposes listed above. Cross-Sectional Sample Surveys The data most widely used for measuring high school dropout and comple- tion come from the Current Population Survey (CPS),3 the U.S. decennial 1100 percent – 73.4 percent = 26.6 percent. 2100 percent – 9 percent = 91 percent. 3Technically, CPS data are not cross-sectional, although they are often used as if they were. That is, if a subject is selected to respond to the CPS, the person is surveyed 8 times over 16 months, which makes the data longitudinal. Using the data longitudinally is technically very difficult, so they are rarely used in this way. We think this has contributed to the misconception that the data are cross-sectional rather than longitudinal.

OCR for page 43

OCR for page 43
47 CURRENT AND PROPOSED MEASURES based on young people who live in an area but who may not have gone to high school in that area.” To the extent that young people move from state to state after age 18, estimates of state high school dropout rates based on CPS data—particularly status dropout rates based on 16- to 24-year-olds—may be of questionable validity (see also Kaufman, McMillen, and Bradby, 1992). Fourth, there are concerns about population coverage with the CPS, par- ticularly for racial/ethnic minorities. The CPS is representative of the civilian, noninstitutionalized population of household residents in the United States, so young people who are incarcerated, in the military, or homeless are not represented. To the extent that these populations differ from the rest of the population with respect to frequency and method of high school completion, there is the potential for CPS-based estimates to differ from those based on other data sets that capture these populations. Finally, substantial changes over time in CPS questionnaire design, administration, and survey items have made year-to-year comparisons difficult (Hauser, 1997; Kaufman, 2001). It is possible to overcome some, but not all, of these limitations of the CPS by using data from the ACS or the decennial census. Sample sizes are larger, enhancing the reliability of state- and urban-level estimates. Both the ACS and the decennial census include individuals who are institutionalized or in the mili- tary, which enhances the generalizability of the reported statistics. However, it is still not clear how accurately ACS respondents report whether they obtained GEDs or regular high school diplomas. In addition, the ACS and the decennial census share with the CPS the limitation that sampled young people are not asked to indicate the state(s) in which they attended high school or the school they attended. As a result, the ACS and the census are useful for constructing rates that describe the human capital of populations; however, measures derived from the CPS, the ACS, and the census are not well suited to describing schools’ holding power, because they never refer to specific schools at all, or to young people’s success in navi- gating the secondary school system. Longitudinal Sample Surveys Although a number of longitudinal sample surveys are used for construct- ing dropout and completion rates (e.g., the National Longitudinal Surveys), the most widely used are those conducted periodically by the Bureau of Labor Statistics (see http://www.bls.gov/nls/) and the National Center for Education Statistics (NCES) (see http://nces.ed.gov/surveys/SurveyGroups. asp?group=1). The NCES surveys include the following: · he 1972 sample of seniors in the National Longitudinal Study of the T High School Class of 1972 (NLS).

OCR for page 43
48 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES · he 1980 and 1982 samples of sophomores and seniors in High School T & Beyond (HS&B). · he sample of eighth graders in the 1988 National Educational Lon- T gitudinal Study (NELS). · he sample of sophomores in the 2002 Educational Longitudinal T Study (ELS). Of these four data sources, NELS has been at the center of a great deal of research and debate on the measurement of high school dropout and com- pletion rates in recent years (e.g., see Greene, Winters, and Swanson, 2006; Kaufman, 2004; Mishel, 2006; Mishel and Roy, 2006). Thus, we focus on NELS in the discussion below. NELS is a longitudinal survey of the grade 8 student cohort of 1988. In the base year, the sample included approximately 25,000 randomly selected students in 1,000 public and private schools across the United States. In addi- tion to the data collected from student interviews, NELS contains information from parents, school administrators, teachers, and student transcripts. The initial student cohort has been resurveyed on four occasions, in 1990, 1992, 1994, and 2000. Students who dropped out of school between surveys were also interviewed. In the early follow-up surveys, the sample was “freshened” with new sample members in order to make the first and second follow-up surveys cross-sectionally representative of 1990 sophomores and 1992 seniors, respec- tively. The content of the surveys includes students’ school, work, and home experiences; educational resources and support; parental and peer influences; educational and occupational plans and aspirations; delinquency; and many others (Curtin et al., 2002). For the purposes of measuring high school dropout and completion rates, the key feature of NELS (and of other longitudinal sample surveys as well) is that it includes information about whether and when cohort members dropped out of school and whether and how they obtained secondary school creden- tials. A key design feature of NELS is the availability of transcript data on high school enrollment, dropout, and completion. In the absence of coverage bias and nonparticipation, NELS data would provide very accurate estimates of high school dropout and completion rates at the national (but not state or district) level—albeit for a single cohort of young people. Despite the advantages associated with its longitudinal design, a number of technical issues raise questions about the accuracy of dropout and completion rates based on NELS (Kaufman, 2004); these issues also arise in the context of other longitudinal surveys based on samples. First, the base-year NELS sample excluded many students with limited English proficiency or mental or physical disabilities. NCES gathered supplementary information from these students later, but it is not clear how often this supplemental information is used in calculating NELS-based dropout and completion rates. Second, as noted by

OCR for page 43

OCR for page 43

OCR for page 43
51 CURRENT AND PROPOSED MEASURES of 16- to 24-year-olds were status completers; that is, they were not enrolled in high school and held some sort of high school credential. The numerator of the status dropout rate reflects the number of people who have not obtained any high school credential and are not working toward one. The fact that many dropouts subsequently re-enroll in high school, obtain a GED credential, or earn high school credentials in other ways is immaterial in calculating the rate, as is the age at which young people complete high school. Status dropout and completion rates are usually calculated using cross- sectional data on individuals in the target population. All that is required is information about individuals’ ages, enrollment status,8 and high school completion status. All status dropout and completion rates are measures of the amount (or lack) of human capital in a population. Status rates do not differen- tiate between those with a diploma and those with a GED or other credentials, however, and do not consider when the credential was earned. As such, they are poor measures of schools’ holding power or of young people’s success at navigating the secondary school system and persisting in school. A low status dropout rate may reflect very high holding power of schools, or it may obscure a situation in which schools have very low holding power and many young people obtain alternate credentials in their late teens or early twenties. Moreover, status rates do not account for the location of schools. A geo- graphic area may have a low status dropout rate because its schools have high holding power, or the area may attract people who have high school credentials. For instance, counties with high technology industries or large postsecondary institutions tend to have relatively low status dropout rates. This probably says more about the human capital of people who move to those counties than about the holding power of the schools there. Event Rates An event rate reports the fraction of a population that experiences a par- ticular event over a given time interval; by definition, everyone in the popula- tion is at risk of experiencing that event during the period. The most frequently reported example is the event dropout rate—the proportion of students who exit school during a given academic year without completing high school. Event dropout and completion rates can be calculated using either cross- sectional or longitudinal data. All that is required is information about indi- viduals’ enrollment status in two consecutive academic years, their comple- tion status in the second of those years, and (under some formulations) age. Enrollment status is typically measured at the beginning of each academic year 8Itis important to note that enrollment status can be a problematic concept, in that individu- als may be enrolled full-time or part-time and in a regular school, a continuation school, or night school.

OCR for page 43
52 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES so the rate can more clearly represent the incidence of dropping out during a well-defined period of time. For instance, Kominski (1990) developed an event dropout rate that could be calculated from data collected in the October School Enrollment Supple- ment of the Current Population Survey; that rate could be estimated separately at grades 10, 11, and 12 or combined across all three grade levels. Research on national trends and differentials in the event dropout rate was undertaken by Hauser (1997) and by Hauser, Simmons, and Pager (2004). The rates estimated in those studies fall well below those of the more extreme (high) estimates of status dropout and, perhaps partly for that reason, have received little public attention. Misinterpretation of event rates as cohort rates often leads people to believe that dropout rates are lower than they really are. Because event rates are sometimes reported for students at all grade levels and ages, who have very different risks of dropout, and because the CPS provides very small samples at the relevant ages, they are rarely sufficiently sensitive for gauging the effects of changes in school practices, even at the state or regional level. Another dis- advantage of this rate, shared by the status dropout rate when estimated from the CPS, is that it excludes those in the institutionalized population, such as students in prison or in the military. There are also population coverage prob- lems in the CPS, especially for minorities. Because they are measures of the share of a population that experiences a particular event over the course of a specific time interval, event dropout and completion rates can be used to describe schools’ holding power or young people’s ability to successfully navigate the school system. Whether an event dropout rate is a fair characterization depends on (a) how “success” and “fail- ure” are defined in the numerator and (b) how the population is defined in the denominator. If the goal is to measure schools’ holding power, the numerator is determined by how schools define success (e.g., they are explicit about whether GEDs and other alternative credentials are treated as equivalent to regular diplomas), and the denominator is restricted to those continuously residing in a well-defined geographic area (typically a school district or state). The result- ing event dropout rate thus describes the experiences of only those students for whom the school district or state is formally responsible. If the goal is to measure the rate at which students’ succeed in navigating the secondary school system, the denominator need not be restricted to those who continuously reside in a particular geographic area. Individual Cohort Rates Individual cohort rates are derived from longitudinal (or retrospective) data on individuals, all of whom were the same age or in the same grade at a certain point in time (e.g., students entering high school in a given year). Individual cohort rates report the fraction of individuals who transition into a

OCR for page 43

OCR for page 43

OCR for page 43

OCR for page 43

OCR for page 43

OCR for page 43

OCR for page 43
59 CURRENT AND PROPOSED MEASURES recall errors and selective mortality, we suggest that these items be asked only of individuals between the ages of 16 and 45. In the past few years, dropout and graduation rates have received much attention, in part because of discrepancies in the reported rates. These discrep- ancies have arisen as a result of different ways of calculating the rates, different purposes for the rates, and different ways of defining terms and populations of interest. The federal government can do much to help ameliorate the confusion about the rates. For instance, in 2008, it provided regulatory guidance about how the rates were to be calculated and reported to meet the requirements of NCLB. The National Governors Association’s definition of graduation rates provides a good starting point for standardizing practice in the way that these rates are determined. However, the definition is not specific enough to ensure that rates are comparable across states. We therefore recommend: RECOMMENDATION 4-5: The federal government should continue to promote common definitions of a broad array of dropout, graduation, and completion indicators and also to describe the appropriate uses and limita- tions of each statistic.

OCR for page 43