Decisions Required to Compute the Indicators

**D**ropout and completion rates are among the most basic indicators of the effectiveness of a school, a school district, or a state education system. Although calculating the actual rates may seem simple, judgments must be made about who to include in the base group of students being tracked (i.e., the cohort), who to count as a graduate or a dropout, how many years to follow these students, and how to construct the formula. Each decision substantially affects the resulting rates. A number of reports have contrasted popular calculations in terms of their accuracy, bias, and ease of computation (e.g., Mishel and Roy, 2006; National Institute of Statistical Sciences and the Education Statistics Services Institute, 2005; Swanson, 2003; Warren, 2004). The technical definitional issues that are embedded in those calculations are at least as deterministic of bias and accuracy as the computations.^{1}

Dropout and completion rates are based on a ratio, and one of the first decisions that needs to be made is how to define the numerator. Depending on the type of rate, the numerator represents dropouts, completers, or graduates. Identifying the students who are graduates is somewhat straightforward—they are the diploma earners. Identifying the students who fall into the other categories and distinguishing between graduates and other completers are more difficult. The second key decision is how to define the denominator, the full cohort of students with the potential of graduating or dropping out. Decisions

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.

Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 25

3
Decisions Required to
Compute the Indicators
D
ropout and completion rates are among the most basic indicators of
the effectiveness of a school, a school district, or a state education
system. Although calculating the actual rates may seem simple, judg-
ments must be made about who to include in the base group of students being
tracked (i.e., the cohort), who to count as a graduate or a dropout, how many
years to follow these students, and how to construct the formula. Each decision
substantially affects the resulting rates. A number of reports have contrasted
popular calculations in terms of their accuracy, bias, and ease of computation
(e.g., Mishel and Roy, 2006; National Institute of Statistical Sciences and the
Education Statistics Services Institute, 2005; Swanson, 2003; Warren, 2004).
The technical definitional issues that are embedded in those calculations are at
least as deterministic of bias and accuracy as the computations. 1
Dropout and completion rates are based on a ratio, and one of the first
decisions that needs to be made is how to define the numerator. Depending on
the type of rate, the numerator represents dropouts, completers, or graduates.
Identifying the students who are graduates is somewhat straightforward—they
are the diploma earners. Identifying the students who fall into the other cat-
egories and distinguishing between graduates and other completers are more
difficult. The second key decision is how to define the denominator, the full
cohort of students with the potential of graduating or dropping out. Decisions
1Kaufman (2001), for example, found that differences in dropout and completion rates between
the National Education Longitudinal Study and the Current Population Survey derived more from
differences in definitions of populations and dropout/completion than from differences in their
methods.
25

OCR for page 25

26 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES
about how to include transfer students and students retained in grade in the
cohort can have a major impact on its size and on the rate that is calculated.
At the workshop, Elaine Allensworth, with the Consortium on Chicago
School Research, made a presentation about the kinds of decisions that must
be made and the ways that they affect the calculation of the rates. This chapter
is drawn from her paper (Allensworth, 2008).
DECISIONS AFFECTINg THE NuMERATOR
Distinguishing Between graduation and Completion
There is generally little ambiguity about who has obtained a high school
diploma—lists of graduates must be maintained. However, there are multiple
means of completing school besides earning a regular diploma, and not all
students complete high school in the same time span. Accounting for differ-
ent forms of completion has become increasingly complicated. In addition to
granting diplomas, schools may offer programs that award General Educational
Development (GED) credentials, or students may receive GEDs outside a
K-12 school system.2 Some districts offer alternative schools for students who
have dropped out or are at risk of dropping out, and these schools may grant
diplomas based on easier standards than the regular high school standards. 3 A
number of states offer different levels of diplomas or grant certificates of atten-
dance for students who do not pass state-required examinations or meet other
criteria (Laird et al., 2008). Growth in home schooling and distance learning
further complicates the ways in which completion is defined (National Institute
of Statistical Sciences and the Education Statistics Services Institute, 2005).
There are a number of valid reasons for producing graduation rates that
include only students who earn a regular high school diploma. There is little
evidence that alternative methods of completion benefit most students who
pursue these pathways to finishing high school. As noted in Chapter 1, GED
recipients do not usually realize the same economic outcomes as high school
graduates and are less likely to pursue higher education. Also, there is substan-
tial variability across states and districts in the extent to which other forms of
completion are available and recorded in an accessible manner.4 Constraining
2Pallas (1989) provides a good description of the GED, as well as issues around alternative
credentials and the different paths through which students may eventually obtain a diploma or
credential or drop out.
3This is the case in Chicago, where alternative schools require students to complete the require-
ments set by the state for a diploma, rather than the much more demanding requirements set by
the district.
4Data on alternative diplomas, GEDs, are not always recorded if provided by entities other than
regular K-12 schools. This can be seen in the Common Core of Data, which allows three categories
of completers to be reported (diploma recipients—included in the graduation rate, high school

OCR for page 25

27
DECISIONS REQUIRED TO COMPUTE THE INDICATORS
completion rates to diploma recipients provides some degree of consistency
across states and districts with different policies, programs, and data-recording
procedures. It also helps maintain stability over time; that is, more comprehen-
sive completion rates might increase over time simply because of an expan-
sion in the availability of alternative forms of completion.5 For purposes of
accountability, counting only diploma recipients prevents perverse incentives
for schools to push students out of regular programs into alternative or GED
programs.
Constraining completion indicators to diploma recipients also has some
shortcomings. Diploma requirements are not equivalent across states and dis-
tricts, and they may change over time, making them not equivalent across differ-
ent cohorts of students, even within the same school district.6 In addition, when
used for accountability, rates that include only graduates provide no incentive
to districts to offer recovery programs for students not likely to obtain a regular
diploma. Diploma-based rates also cannot be calculated for schools that offer
alternative credentials, and thus, their effectiveness cannot be compared with
schools that offer regular diplomas.
Timing of Completion
It may seem desirable to include all completers in the rate, regardless of
how long it took them to complete their education. If the rates are being used
as a tool to encourage good practices, it makes sense to allow a long time for
completion so that schools do not have an incentive to give up on students
who do not complete high school in four years. Time limitations for comple-
tion are problematic for students with disabilities: those whose Individualized
Educational Plans (IEPs) specify that they remain in school beyond age 18 or
who are ungraded and whose time in high school is not clearly defined. Time
limitations may exclude summer graduates or students who take just one extra
term to complete school. Practically, each extra year that is incorporated into
the statistic means an additional year’s wait before the completion rate can be
reported. Whether the statistics are used for accountability, program evaluation,
or assessment of the current state of educational attainment—it is generally
critical to get timely information.
equivalency recipients, and other). The variability in defining completers is notable across states
(Mishel and Roy, 2006).
5For example, based on the Common Core of Data, Warren (2005) noted a rise in regular di-
plomas in California from 1996 to 1997, but much less of a rise in the total number of high school
completers—a fact he attributed to changes in how diplomas were classified.
6In recent years, many states have raised their graduation requirements or have developed plans
to raise them. Currently, the standards for graduation vary widely across states. There are also dif-
ferent types of diplomas in some states, and some states allow parental waivers from the default
curriculum (Achieve, Inc., 2008).

OCR for page 25

28 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES
Time limits bring questions about how to count students still enrolled in
school. Counting them as noncompleters maintains the definition as a four-year
rate. However, this may not accurately represent the true state of high school
graduation in the school or region. For instance, at the workshop, Mel Riddile,
with the National Association of Secondary School Principals, noted that in
some school districts, a student who does not graduate in June but finishes
incomplete coursework over the summer, so that the diploma is awarded in
July or August, is not always counted as a graduate in four-year rates. An
alternative is to exclude students still enrolled from the indicator—removing
them from both the numerator and the denominator. This allows the comple-
tion rate to be the opposite of the dropout rate, which is conceptually easier
for a wide audience to understand. This also avoids any incentives to push out
students who can no longer be counted as graduates. However, excluding large
numbers of students from the calculation can be difficult to justify if there is
concern that all students be counted. Furthermore, rates have the potential to
be manipulated by delaying classification of students as dropouts. 7
Distinguishing Between Dropouts and Transfers
Determining which students are dropouts is much more difficult than
determining which students are graduates or completers. While students can
graduate or complete a program only once, they can drop out of school repeat-
edly and from different school systems, making it difficult to know how to
classify them at any given point in time.
Students who drop out often have high absentee rates. It is not always clear
at which date these students should be considered dropouts, since they may
have been absent for long periods of time before officially being dropped from
the enrollment list. Some students never show up for high school. If a student
finished grade 8 but does not officially enroll in high school, should the student
be considered a dropout from the high school he or she never attended? The
National Center for Education Statistics (NCES) requires schools to define
enrollment on October 1. How are students classified if they leave grade 9
during September, after less than a month of school?
It is challenging to track down students who simply stop showing up at
school. The student may have transferred to another school system, but verify-
ing the transfer is difficult and requires administrative resources. Schools may
miscode dropouts as transfers, even though their status is actually unknown,
7It may also, potentially, give a more accurate picture of the percentage of students who eventu-
ally graduate—if the same percentage of students remaining in school eventually graduate, some-
thing that can be tested empirically. In Chicago, this is the case, and excluding students still enrolled
after four years produces rates that are similar to those that would be produced if students were
allowed extra years to graduate.

OCR for page 25

29
DECISIONS REQUIRED TO COMPUTE THE INDICATORS
a practice that leads to underestimates of the true dropout rate. Of particular
concern is the way that schools code students who leave school to pursue a
GED credential. Some may miscode them as transfers, particularly if they leave
to enroll in a GED-granting institution, and as a result may eliminate them from
both the numerator and the denominator when calculating rates, a practice that
also leads to underestimates of the true dropout rate. Generally, students who
leave school to pursue a GED credential should only be considered as transfers
if they enroll in a program that grants diplomas (and in this case removed from
the denominator of the rates). This tendency may be more prevalent when the
rates are used for accountability purposes. For this reason, the National Forum
on Education Statistics (2005) encourages states to require validation of trans-
fers to verify (usually through a transcript request) that the student has enrolled
in another school that grants regular diplomas.
Policies that require unverified student leavers to be coded as dropouts
provide a substantial incentive for schools to obtain valid records of transfers.
However, validation costs money,8 and schools that have the greatest demand
for validation of transfers—those with large numbers of mobile students—may
not have the necessary resources. In general, requiring validation will inflate
dropout rates and decrease completion rates, because students will be clas-
sified as dropouts until their transfer status can be verified. For instance, at
the workshop, Rob Curtin, with the Massachusetts Department of Education,
described a policy change in his state that required that all students coded as
in-state transfers to be verified or, if unverified, to be coded as dropouts. This
policy change produced a half-percent increase in the dropout rate in the first
year it was implemented. Statewide data systems with individual student identi-
fiers are useful for in-state validation, but they do not remedy the problem for
across-state and out-of-country transfers.
Another issue is how to classify students who are incarcerated or otherwise
institutionalized. It may seem logical to classify these students as transfers since
the decision to leave their school was not voluntary, and the receiving institu-
tions should provide them schooling. However, there is often no verification
that incarcerated students are enrolled in a program leading to an accredited
high school diploma or that they plan to re-enroll in a regular high school when
released. Therefore, they are sometimes counted as dropouts, unless they re-
enroll in school within the time frame being studied.
Finally, decisions must be made about how to classify students whose
leave status changes over time. There are many potential paths to eventually
becoming a dropout or a completer (Pallas, 1989). Most problematic is that
students who have already been counted as dropouts may re-enroll and drop
8For a discussion of some of the issues around validation and data requirements, see the final re-
port of the National Institute of Statistical Sciences and the Education Statistics Services Institute’s
(2005, Chapter 6 and Appendix D) task force on graduation, completion, and dropout indicators.

OCR for page 25

30 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES
out a second or third time. For instance, at the workshop, Robin Taylor, with
the Delaware Department of Education, reported that in her state 3 percent of
the dropouts left school multiple times.
It is also unclear how to count dropouts who have re-enrolled in a differ-
ent school or entered a school system through an alternative school. Since they
enrolled as dropouts from another place, it may not be fair to the receiving
school to count them as regular transfers into the new school or system. Prob-
lems of re-enrollment overlap with issues of transfer between schools, and the
ways in which they can be handled depend on the way that the group or cohort
being measured is defined in the first place.
DECISIONS AFFECTINg THE DENOMINATOR
Defining the Base grade of the Cohort
The cohort represented by dropout and completion indicators is almost
always defined by grade, such as students entering grade 9 for completion
statistics and students in grades 9-12 for dropout statistics. One complication
posed by defining the base group by grade level is that students repeat grades.
Grade 9, in particular, is frequently repeated, particularly by students who
eventually drop out.9 Thus, statistics defined by grade level often double- or
triple-count students in rates representing different cohorts or years, and the
students who are double-counted are those who are least likely to complete
school. If students who are repeating a grade are not removed from the cal-
culations, the statistics will be biased against schools with high levels of grade
repetition. Because these schools are also likely to have high dropout rates, this
accentuates the real differences in dropout/completion rates across schools and
can make issues of equity appear more extreme than they really are (Mishel and
Roy, 2006; Warren, 2004).
Even grade retention prior to high school can introduce problems in
calculating the rates because a student’s probability of dropping out is highly
correlated with age.10 If larger numbers of students enter high school at age
16 than in the past, dropout rates will rise simply because a larger percentage
of the students are at an age when they are permitted to drop out. These age
9This can be seen, for example, in Abrams and Haney (2004), Nield and Farley (2004), and
Allensworth and Easton (2005).
10For example, dropping out becomes legal at age 16, pregnancy becomes more likely at older
ages, and full-time employment becomes more feasible as students get older. Data from Chicago
show a positive relationship between grade level and the probability of dropping out, but it exists
only because of the relationship between age and grade level. Holding age constant, there is no
positive relationship between grade level and dropping out (the relationship becomes negative
because of grade retention effects). Holding grade level constant, there is still a strong positive
relationship between age and dropping out.

OCR for page 25

31
DECISIONS REQUIRED TO COMPUTE THE INDICATORS
effects occur beyond any effects of grade retention, which itself increases the
probability of eventually dropping out of school (Alexander, Entwisle, and
Dauber, 2003; Allensworth, 2005a).
Changes in grade retention patterns prior to high school can also affect the
comparability of grade 9 cohorts over time by delaying the entry of the academi-
cally weakest students. If larger numbers of students are retained in elementary
school one year (e.g., because of the implementation of promotion standards),
the dropout rate for one freshman cohort may improve simply because many
of the cohort’s low-achieving students have been moved into the next cohort.
These kinds of policies affect the comparability of grade-based cohort rates
over time and over cohorts.
Schools that have unusual grade configurations may not always be included
in rates that define cohorts by grade. For example, middle schools that contain
grade 9 but not grade 12 would not produce a graduation rate, even though
they have a full grade 9 class. Incorporating ungraded special education stu-
dents into grade-based cohorts is also problematic.11
One way to deal with these issues is to define cohorts by age instead of
grade level. Age-based cohorts are not affected by patterns of grade retention
in years prior to or during high school. This makes the rates more sensitive,
accurate, stable, and unbiased than when the cohort is defined by grade level. In
addition, students who never make it to high school can be included in the rates
with their age peers, allowing the statistics to include middle grade dropouts.
Special education students can also be included with their age peers, even if
they are in ungraded classrooms. For districts and states, age-based rates can
be more useful than grade-based rates to gauge trends over time and to assess
the current state of education. However, they are not useful as indicators of
school performance, since schools, parents, and the public generally want to
know how many of the students who attend a school in a given year graduate
or drop out later.
Specifying Enrollment Timelines
Defining the cohort also requires consideration of when the student
enrolled in school. Because students could enroll in a school after a set date
or leave school before that date, it is more inclusive to define the group as
including any student enrolled over the period of a year (e.g., from October 1
of one year to September 30 of the next year) than to choose one date in the
year for the base enrollment (e.g., enrolled by the 30th day of school). Most
often, schools define school year enrollment from either October to October
11NCES includes ungraded students by prorating the total number across grades in the denomi-
nator proportional to known graded enrollment rates, with ungraded dropouts included in the
numerator (Laird et al., 2008).

OCR for page 25

32 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES
or June to June. Choosing one over the other has little effect on the resulting
rates, but it does affect their interpretation—as students who dropped out in
the school year versus students who were enrolled one year and did not return
by the second year.12
Indicators that base their denominator on students enrolled on a spe-
cific date are particularly susceptible to undercounting students who transfer
between schools. Those that compare the number of dropouts or completers
over a set period (e.g., the school year, the next four years) to the enrollment at
the set date will include students in the numerator who are not in the denomi-
nator. For instance, any student who enrolled in a school after the date that
defined the cohort and then dropped out will be included in the numerator but
not in the denominator. This leads to the potential for dropout rates that exceed
100 percent. In contrast, students could be included in the denominator who
left the school soon after the cohort was defined and have no chance of being in
the numerator, deflating the statistic.13 If in- and out-mobility rates are similar,
and if the students who leave have the same probability of completing/dropping
out as the students who enter, then this does not result in bias or inaccuracy.
However, these latter conditions do not always hold.
Adjustments can be made to account for in- and out-mobility so that
the numerator remains a subset of the denominator. However, some of these
adjustments require demographic assumptions that, if untrue, can introduce
additional inaccuracy into the rates. Warren (2004) demonstrates how inaccu-
racies can be introduced when assumptions about in- and out-mobility, cohort
size, and grade retention do not hold. In addition, adjustments that account
for demographic instability must also assume that students who leave a school
are similar to students who enter in terms of their probability of completion/
dropout. This is often not the case when schools have different enrollment poli-
cies. If the students who leave a school are qualitatively different from the stu-
dents who enter in a way that is systematically related to dropout/completion,
the rates will be biased, even after adjustments are made to the denominator.
An alternative to defining the cohort on a specific date is to include all
students enrolled at any time during the school year, or at any time over the
four years of high school, for completion rates (with students entering at higher
12For the Common Core of Data dropout rates, some schools use the official October-to-October
definition to define dropouts in the numerator, and some states use a June-to-June definition. Dis-
tricts that use June-to-June dropout rates tend to report slightly higher numbers of dropouts, but
the differences are small (Winglee et al., 2000).
13For instance, a student may be counted in the September enrollment but transfer in Novem-
ber. Unless adjustments are made for out-mobility, that student will be in the denominator of the
indicator for the first school but will not be included in the numerator of the indicator for the first
school. This will be true even if the student drops out in the spring, because the student’s final
status is known to the new school.

OCR for page 25

33
DECISIONS REQUIRED TO COMPUTE THE INDICATORS
grades in each year after grade 9).14 However, the data requirements are higher
for this more inclusive definition; that is, schools must maintain individual-
level records of students’ dates of enrollment and departure. Furthermore, this
definition results in students being counted at multiple institutions; if statistics
are aggregated to higher level units (such as the district or state level), mobile
students will be double-counted in the aggregate rates.
Decisions also must be made about whether to include students who are
enrolled for a fraction of a year. Should schools be held responsible for stu-
dents who enter and drop out within a week? If no minimum time period is
set, including all students enrolled at a school in the denominator will gener-
ally increase dropout statistics because mobile students are more likely to drop
out than other students. However, if a sizeable minimum time period is used
to define who is a student (e.g., all students who enter after October 1 and
remain through the end of the year), then dropout rates will be depressed,
since, by definition, new students are included in the indicator only if they
stay in school. In the end, issues of transfer and mobility are made no easier by
including students enrolled over a period of time versus a snapshot; while the
indicators are more inclusive, issues of aggregation and data requirements are
more problematic.
Handling Transfer Students
When a student transfers between schools, it is not clear which unit should
take responsibility for that student. In theory, there are four options: attribu-
tion to the receiving school, the sending school, both, or neither. In practice,
schools tend to mix classification decisions depending on students’ final sta-
tus. For example, the graduation rate for the state of Illinois counts transfer
students who graduate with their receiving school but those who drop out to
neither school. Most of the graduation rates based on aggregate counts of stu-
dents (including the Cumulative Proportion Index and the graduation rate in
the Common Core of Data) attribute transfers who drop out to their sending
school, but those who graduate to both schools—as a graduate at their receiving
school and a nongraduate at their sending school. Mixing attribution inherently
produces bias in the statistics.
Attribution to Both Schools
Counting students at both their sending and receiving schools is the most
inclusive method for dealing with transfers. It ensures that all students are
14For example, students entering as grade 10 are grouped with students who started in grade 9
the year before; those entering as grade 11 are grouped with students who started grade 9 two
years earlier, etc.

OCR for page 25

34 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES
counted. However, it also ensures that transfer students will be counted at least
twice—and more than twice for highly mobile students. This is problematic
when school-level statistics are aggregated to produce district- or state-level
rates because the mobile students (who are most at risk of dropping out) will
be overcounted. Counting students at both schools also requires that schools
share information or that information on graduation/dropout is maintained in
a way that students’ final status can be attributed to their first school.
An alternative might be to weight students’ contribution to their school’s
rates by their time enrolled. This alternative is even more data intense, since
it would require information about students’ dates of entry and departure at
each school to be available and shared between schools. It also would require
strict guidelines for defining a dropout date for students with many absences
to define the percentage of time enrolled at the final school. It is doubtful that
such a method could be practically implemented.
Attribution to Neither School
Counting transfer students at neither the sending nor the receiving school
has the obvious disadvantage of excluding all mobile students, which would
produce an overly optimistic picture of the state of education at all levels of
aggregation. This option eases calculation and data requirements, because no
knowledge of final status is necessary. There are also political advantages, since
schools do not feel they are being held accountable for problems created in
other schools. In practice, only nongraduates are at risk of being excluded from
the statistics of all schools—schools tend to want to count all of their graduates.
This biases graduation rates upward, leading to more bias than if all transfer
students were simply excluded.
Attribution to the Receiving School
Counting students with their final/receiving school makes practical sense
because the final status is known to the school to which the student is attrib-
uted. It also makes intuitive sense to group students with the school from which
they graduate or drop out. Schools generally feel that they should get credit for
all of the students they graduate.
Although there are many practical advantages to counting transfers with
their receiving school, doing so produces rates that are biased in favor of
schools that can control their enrollment, and it can provide perverse incentives
to transfer students at risk of not completing school. Attributing dropouts to
their receiving school, instead of their sending school, assumes the problems
that led students to drop out occurred at the final school. Yet, as discussed in
Chapter 5, students’ performance in their first year of high school is extremely
predictive of eventual graduation (Allensworth and Easton, 2007). If a student

OCR for page 25

35
DECISIONS REQUIRED TO COMPUTE THE INDICATORS
leaves a school because of problems at that sending school, it may not be fair
to credit that student’s eventual withdrawal to the receiving school, especially
if the student spent little time at the receiving school. This results in bias in the
rates when schools have different enrollment policies. Schools that can choose
whether to enroll a transfer student can boost their graduation rate by accepting
students who have already shown some success in high school (e.g., on-time
eleventh or twelfth graders).15 This can happen, for example, if a school uses
a lottery to pre-enroll eighth graders into grade 9 and then uses lotteries for
grades 10-12; such a system precludes grade 9 repeaters. Likewise, schools with
open enrollment can seem to be doing worse than they should since they accept
students who have been unsuccessful in other places. This is a flaw inherent
with the National Governors Association (NGA) graduation rate; it is biased
in favor of schools that have some control over their enrollment.
Attribution to the Sending School
The final option is to count students with their first (sending) school. This
is the method used for graduation rates at the college level in the NCES Gradu-
ation Rate Survey. Given the strong relationship between first-year course per-
formance and eventual graduation, it makes sense from a conceptual standpoint
to count students with their first school. It also prevents schools from benefiting
by encouraging poorly performing students to transfer, particularly to schools
that may not best serve their interests. It may seem unfair to count a student
who transferred and then dropped out against the graduation rate of their origi-
nal school. However, unless students are systematically leaving a school for a
particular reason (e.g., because of safety concerns at the school that are forcing
students out), the dropout and graduation rates of the school’s former students
should balance each other and not substantially affect the dropout rate. Using
the first school also allows schools without a grade 12 (e.g., a middle school
with grade 9) to be compared with other schools.
The problem with this approach is that students’ final outcomes may not be
known to the sending school. It can be done if student-level data are available
at a district or state level, but there will still be out-of-district or out-of-state
transfers that cannot be included in the calculations. The NCES college gradu-
ation rate circumvents this problem by counting all transfer students as non-
graduates from their original college. This results in statistics that are unbiased
and accurately represent the graduation rate of students who have matriculated
at a given college, thus circumventing the problem of overcounting transfer
students if statistics are aggregated. The disadvantage is that it underrepresents
15Studentswho transfer in after grade 9 tend to boost graduation rates because they have already
shown enough success to move on past grade 9 and because they are followed for fewer years than
students who enroll in grade 9.

OCR for page 25

36 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES
the percentage of students who graduate overall; that is, it indicates only the
percentage of students who graduate from their initial institution.
For purposes of accountability, this method does not produce perverse
incentives to push students out of a school, but neither does it provide an
incentive to ensure that students who transfer into a school are well served
by the receiving school. Politically, it is difficult to convince schools that their
graduates should not be counted in their completion indicators.
Choosing Among Methods
Each method of defining the numerator and denominator in a dropout/
completion statistic has advantages and disadvantages. No method will produce
a statistic that is ideal for all purposes, and decisions about who to include or
exclude will have an impact on the rate that is produced. Likewise, rules about
who is included and excluded can produce incentives for school officials to act
in ways that are not always in the best interest of students. For instance, a school
can lower its dropout rate if it is allowed to classify students as transfers rather
than dropouts when they leave school and enroll in a GED-granting program.
A sole focus on the on-time (four-year) graduation rate can produce incentives
for schools to push out students who are likely to take longer than four years
to finish high school, particularly if they are allowed to remove these students
from both the numerator and the denominator of the rate.
There are no optimal ways for handling these decisions that overcome all
the potential disadvantages. However, steps can be taken to ensure that the
impact of the decisions is understood. For instance, requiring school officials
to document the status of transfer students and the decisions made in calculat-
ing the rates (i.e., who is included in the numerator and the denominator) is
critical both for interpreting the rates and for revealing unintended incentives
in the rules. Reporting a variety of rates—such as both on-time and eventual
graduation rates, completion rates that include other methods of finishing high
school, rates that do not remove transfer students or incorporate new students,
and age-based rates—can provide a more comprehensive picture of schools’
effectiveness at graduating students. Transparency with regard to the method-
ology underlying the statistics that are created and acknowledgment of their
strengths and weaknesses is essential for understanding.
COMPuTINg DISAggREgATED RATES
Reporting disaggregated dropout and completion rates requires decisions
about who should be included in each group. The No Child Left Behind
(NCLB) Act requires schools to provide separate rates for students grouped
by race/ethnicity, disability status, and English language proficiency. Yet it is
not always clear who should be included in each group. For example, students

OCR for page 25

37
DECISIONS REQUIRED TO COMPUTE THE INDICATORS
can now self-identify their ethnicity and race using multiple categories (U.S.
Department of Education, 2008), and students may change their race/ethnicity
designation(s) from year to year. This can make classification difficult and
inconsistent across schools. Changes in the ways students are classified result
in rates that are not comparable over time, which can lead to false conclusions
about the rates for particular groups.
English Language Learners
Calculating dropout and completion rates for English language learners
(ELLs) is particularly difficult for a number of reasons. First, definitions of
English proficiency are not always consistent across districts and states (Abedi
and Dietel, 2004; National Research Council, 2004, in press). States have dif-
ferent criteria for initially classifying students as ELL and for reclassifying them
(i.e., removing them from the ELL classification because they have acquired
sufficient proficiency in English). This means that cross-state and/or cross-
district comparisons of dropout and graduation rates for this subgroup may
not be valid. More importantly, students’ English proficiency status changes
over time as they acquire sufficient English language skills to be removed from
the ELL classification. Many students who are initially classified as ELL in the
elementary grades are no longer classified as such in high school.
Students who are still classified as ELL in high school—the years when
dropout and completion statistics are calculated—are generally a select group.
They may be students who have had difficulty learning English and have
remained in the classification longer than is typical. Or they may have been
placed in the classification because they recently moved to the United States.
Both of these factors are related to completing school. That is, students’ rate
of acquisition of English is correlated with their academic achievement and
their socioeconomic status,16 factors that are correlated with dropping out
and completion. Immigration at older ages is also related to the likelihood of
graduation, as children who move to the United States at older ages tend to
obtain fewer years of education than children who immigrate when very young
(Allensworth, 1997). Highly mobile students are also more likely to remain
classified as ELL longer than students who are less mobile because they tend
to have long periods of “service breaks” due to absence or transfer; mobility
is another characteristic correlated with high school completion. For these
reasons, students still in the bilingual or English as a second language program
16Hakuta, Butler, and Witt (2000) found that academic English proficiency takes at least four
to seven years, and that the rate of acquisition is related to students’ economic status. Abedi and
Dietel (2004) note that high-performing English language learners get redesignated as they obtain
proficiency. As a result, low-achieving students are increasingly concentrated in the subgroup,
together with students newly classified as English language learners.

OCR for page 25

38 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES
in high school would be expected to have higher dropout rates than students
not in these programs.
In order to obtain the most accurate estimates of schools’ success rates
at graduating ELL students, it is essential to know which students have been
classified as ELL at any time during their schooling, particularly during the
primary grades. This requires data systems to include records on ELL status
from the primary grades onward that remain constant as students progress to
higher grades—a requirement not easily met, since many states do not have
access to primary grade records for these students. These primary grade records
would need to accompany students who transfer, so that receiving schools can
correctly classify students who were formerly classified as ELL. Otherwise, sta-
tistics that are aggregated from school records will fail to include former ELL
students who transferred out of their original school.
Students with Disabilities
Unlike ELL status, students’ disability status does not typically change over
time. That is, having a hearing, visual, emotional, or cognitive impairment or a
learning disability is usually not a temporary condition. However, classification
criteria for these students are not always consistent across schools, districts,
or time (National Research Council, 2004), and thus their status as a recipient
of special education services may change over time. The classification criteria
depend, in part, on the resources available in the school and the district for
diagnosis, as well as district policies about designating students as disabled. 17
Because the criteria for placing students into the category differ across schools
and districts, the dropout and completion rates for this group of students may
not be comparable across units.
Furthermore, students who have achievement well below grade level
are sometimes classified as learning disabled simply on the basis of their low
achievement, even though they may not meet the other criteria for the learn-
ing disability classification. These students may be low-achieving because they
actually have a learning disability, or their low achievement may be due to
other factors that are related to the likelihood of completion, such as years
of low engagement in school. Classifying students into the group based on a
17Eligibilityfor special education services depends on referral, evaluation, and decisions of school
staff and parents. These decisions may differ across schools. School policies and resources may af-
fect whether students are referred or determined to be eligible. Chicago, for example, placed limits
on the percentage of students who could be determined eligible for special education services, in
part, to keep schools from excluding too many students from the accountability policy. When grade
promotion policies were put into place in grades 3, 6, and 8, there was a substantial rise in the
identification of students in those grades. This seemed to be due to uncertainty as to how to handle
students whose achievement remained low after repeatedly failing the exams and being held back
one or more years (Miller and Gladden, 2002).

OCR for page 25

39
DECISIONS REQUIRED TO COMPUTE THE INDICATORS
characteristic (low achievement) that itself is highly correlated with dropping
out can produce misleading results when the indicator is calculated. That is,
the dropout/completion indicator may look worse for this group because it
includes students who may not belong in the category and are at a higher risk
of dropping out due to the characteristic that caused them to be placed in the
category.
In addition, the time element in cohort/group definition is also particularly
problematic for students with disabilities, as some of them are given more than
four years to complete high school in their Individualized Education Plan or
are to remain in the school system past age 18.
AggREgATINg RATES
An additional issue to consider when designing indicators is whether they
will be aggregated to represent completion or dropout rates at higher levels of
analysis. Rates that are accurately defined at the school level are subject to a
number of errors when aggregated to the district, state, or federal level. 18 For
instance, the ways that transfer students are handled can result in their being
omitted or double-counted in aggregated rates. Having individual-level student
data can alleviate these problems. However, subgroup definitions (e.g., defining
who is an ELL student or a special education student) become more problem-
atic at the state level, where there may be less detailed longitudinal information
on individual students.
Students still enrolled after the date they should have graduated are often
counted multiple times, particularly if the rates are calculated from aggregate
numbers of students instead of longitudinal student records. They are initially
counted as noncompleters in their own cohort; they are counted again in the
cohort that follows, either as completers or noncompleters. This introduces
inaccuracy in the rates, but whether they are inflated or deflated depends on
the specific way in which students are double-counted and whether adjust-
ments are made to the denominator along with the numerator. Incorporating
students who remain in school an additional year also makes the indicator less
sensitive to changes in dropout and completion that may be occurring in the
school, which in turn makes them less useful for evaluating the effects of new
policies and practices.
18For example, dropout rates in a district with two high schools might represent all students en-
rolled at each school over a period of four years. If there were high student mobility rates between
the schools, so that many students transfer from one to the other, the resulting dropout rates for the
schools would still be correct estimates of the percentage of students ever enrolled at those schools
who dropped out. But the aggregate dropout rate for the district, based on an average of the two
schools, would double-count students who had enrolled at each school. This would lead the district
statistic to be inaccurate, even though based on accurate school statistics.

OCR for page 25

40 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES
CONCLuSIONS AND RECOMMENDATIONS
Graduation and dropout rates are critical outcome measures of secondary
schooling and should be incorporated into accountability systems. However,
there are inherent weaknesses in any method of calculating those rates. Even
the NGA graduation rate, which we encourage states to adopt, is biased in
favor of schools that can control their enrollments. However, if the limitations
of the calculations are made explicit, alternative rates could be calculated to
verify any conclusions that are made with the statistics. For example, if charter
schools were judged to be more effective at retaining students based on the
NGA rate, it would be prudent to calculate alternative statistics not biased by
school’s enrollment policies to verify this conclusion. This would require data
to be available to calculate the complementary statistics. Thus, it is important
to produce other indicators of the outcomes of secondary schooling and the
development of a robust data system that can produce different variations of
the completion rate estimates.
To the greatest extent possible, the methods for computing the rates should
meet the following criteria: (1) provide the most accurate assessment possible of
how many students actually complete or drop out of school, (2) not be biased in
favor of certain types of schools, (3) be inclusive of all students but not double-
count them, (4) be stable enough to validly track trends over time, and (5) be
sensitive to real changes in student outcomes. All of this must be done within
the constraints of the data systems available to schools and given the purpose
for which the statistics are to be used. No indicator is perfect, and trade-offs
among the criteria are usually required. Ultimately, decisions about the indica-
tors and the ways to calculate them should reflect the intended purpose and
uses. The trade-offs, strengths, and shortcomings should be made explicit. We
therefore recommend:
RECOMMENDATION 3-1: The strengths and weaknesses of dropout
and completion rates should be made explicit when the rates are reported.
RECOMMENDATION 3-2: Rates should be accompanied with documen-
tation about the underlying decisions that were made regarding students
who transfer from one school to another, are retained in grade, receive a
GED credential or an alternative diploma, and take longer than four years
to graduate.
Because decisions about how to handle various groups of students can
affect the rates, we think it is important to supplement dropout and comple-
tion indicators with information to help users accurately interpret them. As
described in this chapter, schools and states have different policies for handling
transfer students. Documentation of how transfers were handled is critical for
interpreting school-level rates. Also useful is an estimate of the transfer/leave

OCR for page 25

41
DECISIONS REQUIRED TO COMPUTE THE INDICATORS
rate and supplementary graduation/dropout rates that do not remove transfers
or incorporate new students. This additional information would allow examina-
tion of the ways in which schools’ policies for handling transfer students affect
the reported rates.
Policies for grade retention also vary across schools, districts, states, and
time. These policies, and particularly year-to-year changes in these policies, can
cause trends in the rates to fluctuate over time. Age-based cohort rates can pro-
vide information to help users understand and evaluate trends in grade-based
rates. Age-based rates have the advantage that they are unaffected by patterns
in grade retention that may have affected one cohort differentially from another.
They are also more inclusive, in that they can include students who never make
it to high school and can include special education students with their peers. If
the limitations associated with a reported rate are made explicit, supplemental
rates can be calculated to verify any conclusions that are based on the statistics,
although this would require data to be available to calculate the supplemental
statistics. We therefore recommend:
RECOMMENDATION 3-3: To the extent possible, data should be made
available to allow supplementary rates to be calculated that compensate for
the limitations in reported rates and help users to further understand the
rates. Types of supplementary information include transfer rates, rates that
do not remove transfer students or incorporate new students, age-based
rates, and the percentage of students with unknown graduation status.
The federal government requires states and districts to produce 4-year
graduation rates that include diploma recipients only. As discussed above,
there are compelling reasons for using this statistic as the primary indicator of
high school completion. However, there are also legitimate reasons to produce
more inclusive completion indicators that allow students more time to complete
high school and include other forms of completion, such as GEDs and alterna-
tive diplomas. Unless there is a common definition, such statistics will not be
comparable across districts and states and over time. We therefore recommend:
RECOMMENDATION 3-4: In addition to the standard graduation rate
that is limited to 4-year recipients of regular diplomas, states and districts
should produce a comprehensive completion rate that includes all forms of
completion and allows students up to six years for completion. This should
be used as a supplemental indicator to the 4-year graduation rate, which
should continue to be used as the primary indicator for gauging school,
district, and state performance.
As part of the NCLB regulations, states and districts are expected to report
disaggregated graduation rates, such as for students grouped by low-income

OCR for page 25

42 HIGH SCHOOL DROPOUT, GRADUATION, AND COMPLETION RATES
eligibility, disability status, and English language learner status, and to track
their progress over time. These subgroup statistics are often not comparable
across schools, districts, or states due to differing methods and rates of iden-
tification and reclassification into and out of the subgroup. Furthermore, the
methods by which students are placed into subgroups can lead to inaccurate
judgments about educational efficacy in a school system for members of the
subgroup. For English language learners, inaccuracies are introduced because
classification into the subgroup changes over time, and the rate of reclassifica-
tion is correlated with dropping out. For students with disabilities, underiden-
tification of disabilities and different methods of classifying disabilities result in
lack of comparability. Furthermore, because some students with disabilities are
expected to remain in school for more than four years, the subgroup statistics
for students with disabilities will disproportionately be affected by decisions
about the number of years allowed for graduation in the indicators (e.g., four-
year versus five-year rates).
The main purpose of subgroup statistics is to gauge the degree to which
schools, districts, and states are serving particular groups of students. To make
these judgments fairly, alternative statistics should provide supplemental infor-
mation for subgroups. With regard to graduation rates for these subgroups,
we recommend:
RECOMMENDATION 3-5: To improve knowledge about graduation
rates among subgroups, alternate statistics should complement conven-
tional indicators. Alternative graduation rates for English language learners
(ELLs) should include former ELL students as well as students currently
classified in this category. Thus, records on ELL status should accompany
students as they progress through grades, change ELL status, and trans-
fer across districts. Alternative graduation rates for special education stu-
dents and English language learners should allow additional years toward
graduation.