Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 61
4
Advancing the Conceptual Framework
This chapter presents a framework intended to provide a starting point for
measuring productivity in higher education. Chapters 2 and 3 presented argu-
ments for why productivity measurement in higher education is exceedingly dif-
ficult and why, in turn, the panel cannot simply prescribe a fully defined metric.
Nonetheless, because governments and many other stakeholders insist on, and in
fact need, an aggregate measure of productivity change, it is important to begin
developing the best measure possible.
The measure proposed involves a number of important assumptions and ap-
proximations, which are elaborated below. Chief among these is the lack of an
agreed-upon measure of educational quality. Productivity should be defined as
the ratio of quality-adjusted outputs to quality-adjusted inputs, but the needed
quality adjustments are not currently possible in higher education and are not
likely to become possible any time soon. We recognize the problem, but believe
it is important to extract as much information as possible from the (quantitative)
data that can be measured. We will describe later how the risks associated with
the lacuna of measures of quality can be minimized including, for example, how
entities can use university and third-party quality assurance methods to ensure
that focusing on the quantitative inputs and outputs does not trigger a "race to
the bottom" in terms of quality.
4.1. CHAPTER OVERVIEW
The productivity measure proposed here is consistent with the methodology
practiced by the Bureau of Labor Statistics (BLS), and offers several significant
advantages over the ad hoc approaches that have been used to date. In particular:
61
OCR for page 62
62 IMPROVING MEASUREMENT OF PRODUCTIVITY IN HIGHER EDUCATION
ˇ The measure is a multi-factor productivity index. It captures output in
physical units (credit hours, degrees) and, unlike cost studies, measures
direct labor inputs in terms of full-time equivalents (FTEs). Labor pro-
ductivity can be derived from the multi-factor results if desired.
ˇ Outputs include credit hour production and degree attainment, both of
which have been shown to be important in labor market studies. Most if
not all the measures currently in use (e.g., credit hour production alone
or graduation rates) depend on one or the other but not both, and there-
fore miss a critical output dimension.
ˇ The measure does not vary along with the proportion of part-time stu-
dents, except to the extent that being part-time might require different
student services or contributes to wasting credits or dropping out. This
feature sidesteps the problem of comparing graduation rates and aver-
age times to degree among schools with different numbers of part-time
students.
ˇ Credits not on the mainline path to a degree, including those due to
changes in major and dropouts, are counted and thus dilute the degree
completion effect. In other words, programs with a heavy dropout rate
will have more enrollments per completion, which in turn will boost
resource usage without commensurate increases in degrees. Productiv-
ity could thus increase with the same number of credit hours if more
students actually complete their degrees. Credit earned, however, is not
treated as entirely wasted just because a degree was not awarded.
ˇ The measure allows differentiation of the labor and output categories,
although doing this in a refined way will require significant new data.
ˇ The measure readily lends itself to segmentation by institutional type,
which is important given the heterogeneity of the higher education
sector.
ˇ The measure can in principle be computed for institutions within a state,
or even single institutions. However, the incentives associated with
low-aggregation level analyses carry the risk of serious accuracy deg-
radation and misuse unless it is coupled with robust quality assurance
procedures. Until quality adjustment measures are developed, the panel
advises against using the productivity metric described in this chapter
for institution-to-institution comparisons (as opposed to more aggregate
level, time series, or perhaps state-by-state or segment analyses).
ˇ Data collection, including data beyond the Integrated Postsecondary
Education Data System (IPEDS) and the proposed special studies, ap-
pears to be feasible.
We emphasize again that the proposed measure follows the paradigm of ag-
gregate productivity measurement, not the paradigm for provision of institution-
level incentives and accountability. As stressed in Chapter 3, institutions should
OCR for page 63
ADVANCING THE CONCEPTUAL FRAMEWORK 63
be prepared to resist inappropriate initiatives to improve productivity as measured
by applying the formula below to their particular data, and to buttress their resis-
tance with their own internal data about quality.
Section 4.2 presents our base model. It is a "multi-factor productivity model"
in that it uses output and input quantities and includes all categories of inputs.
Section 4.3 proposes a segmentation scheme, which is important because of the
heterogeneity of higher education. The section also discusses how the model
can be computed at the state and single-institution level but, again, we stress
that this will be dangerous without a robust quality assurance system. Section
4.4 enhances the base model by differentiating among labor categories. This is
important because of the fundamental difference between academic and nonaca-
demic labor, and the difference between tenure-track and adjunct faculty. Section
4.5 differentiates among output categories, which again is important because of
institutional heterogeneity and the fact that production of degrees at different
levels and in different fields involves different production functions. Finally,
Section 4.6 presents the rationale for using the model in conjunction with quality
assurance procedures.
Nearly all the data required for calculating values using the model sketched
out here can be obtained from the U.S. Department of Education's IPEDS or
other standard public sources (though this would not be the case for the fully
specified "ideal"). Adding the model refinements outlined in Section 4.3 requires
a modest amount of additional information. Data requirements for the enhance-
ments described in Section 4.4 can be approximated from IPEDS, but proper
implementation will require additional data collection. The panel's recommended
changes to IPEDS are discussed in detail in Chapter 6. The new data that are
called for would break useful ground not only for productivity analysis, but also
for institutional planning and resource allocation. This is important because an
institution's use of data for its own purposes makes data collection more palatable
and improves accuracy.
4.2. A BASELINE MULTI-FACTOR PRODUCTIVITY
MODEL FOR HIGHER EDUCATION
Following the concepts defined in Chapter 2, the model calculates the ratio
of changes in outputs (credit hours and degrees) to inputs (labor, purchased ma-
terials, and capital). The focus is on instructional productivity, with inputs being
apportioned among instruction, research, and public services prior to calculating
the productivity ratio. As emphasized throughout this report, our model involves
only quantitative factors. It will be reliable only to the extent that input and output
quality remains approximately constant, or at least does not decline materially.
Currently, quality measurement--of both inputs and outputs--is largely beyond
the capacity of quantitative modeling; but, because quality should never be taken
for granted, we return to the issue at the end of the chapter.
OCR for page 64
64 IMPROVING MEASUREMENT OF PRODUCTIVITY IN HIGHER EDUCATION
4.2.1. Multi-Factor Productivity Indices
Multi-factor indices relate output to a bundle of inputs; that is, they show
how much of an industry's or firm's output growth can be explained by the
combined changes in its inputs. The panel has concluded that a multi-factor pro-
ductivity index is appropriate for measuring higher education productivity at the
segment and sectoral levels. Other kinds of productivity models--for example,
those which estimate educational production functions--are of course possible
and worthwhile. However, the panel was not charged with recommending such
models. Nor was it charged with developing strategies for improving productivity.
Our proposed productivity model is based on the methodology for multi-
factor productivity indices used by BLS, the OECD, and other U.S. and foreign
agencies that produce sectoral productivity statistics (Bureau of Labor Statistics,
2007; Schreyer, 2001). For example, BLS uses this methodology to calculate
productivity indices for aggregate manufacturing and some eighteen manufactur-
ing industries.
The BLS method uses what is known as a Törnqvist index. This differs in
important ways from the method of simply calculating weighted averages of the
variables in the numerator and denominator and then taking the ratio of the two
averages. The key ideas behind the Törnqvist index are as follows (from Bureau
of Labor Statistics [2007:6-7]):
1. The figures for input and output are calculated as weighted averages
of the growth rates of their respective components. Weighting average
growth rates avoids the assumption, implicit in directly averaging the
variables, that the inputs are freely substitutable for one another. It also
removes issues having to do with the components' dimensionality. Both
attributes are important when comparing variables like adjusted credit
hours with labor and other inputs.
2. The weights are allowed to vary for each time period in which the index
is calculated. This means the index always represents current informa-
tion about the relative importance of the variable in question while
maintaining the requirement (discussed in Chapter 2) that the weights
move more slowly than the variables themselves.
3. The weights are defined as the means of the relative expenditure or
revenue shares of the components for the two data periods on which the
current index is based. This method brings relative wages and prices into
the equation because they affect total expenditures.
The Törnqvist scheme has often been the indexing structure of choice for
describing multi-factor productivity change under fairly broad and representative
assumptions about the nature of production: specifically, that the production func-
tion can be represented by a translog generalization of the familiar Cobb-Douglas
OCR for page 65
ADVANCING THE CONCEPTUAL FRAMEWORK 65
function with mild regularity conditions on the parameters (Caves et al., 1982). A
technical description of the Törnqvist methodology is provided in the appendix
to this chapter.
Determining appropriate indices embodying the general ideas put forward in
Chapters 2 and 3 remains a task for future work. As such, it would be premature
for the panel to commit to a specific approach. However, because of its wide-
spread use in other applications--a Törnqvist index is used here for expository
purposes. The denominator of our baseline higher education productivity index
uses a Törnqvist structure to represent the composite growth rates of labor and
capital inputs. The numerator also takes the form of a Törnqvist index, though in
this case with only one element. Section 4.4 extends the index in the denomina-
tor to include more than one labor category, and Section 4.5 uses multiple output
categories in the numerator. The final productivity index is the ratio of the indices
in the numerator and the denominator.
4.2.2. Outputs
On the output side, the model uses two data elements that can be obtained
from IPEDS:
1. Credit Hours: 12-month instructional activity credit hours summed over
undergraduates, first-year professional students, and graduate students;
2. Completions: awards or degrees conferred, summed over programs,
levels, race or ethnicity, and gender.1
Illustrative data for a four-year private university are shown in Table 4.1. 2 The
data cover three years: 2003, 2006, and 2009 (it is best to aggregate over multi-
year periods to reduce volatility associated with noise in the data, but the illustra-
tion ignores that refinement). For reasons explained earlier, both credit hours and
degrees (or completions) are included as outputs. Whatever their flaws, these are
the standard unit measures of instruction in American higher education.
The model uses adjusted credit hours as its measure of output, defined as
follows:
Adjusted credit hours = Credit hours + (Sheepskin effect × Completions)
1For broader use, definitions become more complicated. For example, as discussed in the next
chapter and elsewhere in the report, "completions" defined as certificates and successful transfers
become relevant in the community college context. Nondegree seekers (e.g., summer transients) also
come into play at many kinds of institutions.
2The data are based on an actual institution, but certain adjustments were made to make the
illustration more coherent.
OCR for page 66
66 IMPROVING MEASUREMENT OF PRODUCTIVITY IN HIGHER EDUCATION
TABLE 4.1 Illustrative Data from IPEDS for the Base Model
Period 1 Period 2 Period 3
Enrollments and Completions
Credit hours 578,815 574,176 602,000
Completions 2,154 2,310 2,500
Adjusted credit hours 638,435 643,476 677,000
Total Number of Staff
Full time 6,265 6,656 6,826
Part time 683 4,949 2,250
Labor FTEs 6,493 8,306 7,576
Finance: Core Expenditures
Wages and Fringe Benefits
Instruction $421,534 $525,496 $641,749
Research 295,814 531,759 424,075
Public service 5,339 5,500 5,700
Student services 39,178 50,113 62,626
Administration and support services 488,969 563,969 534,924
Intermediate Expenditures
Instruction $161,142 $328,987 $427,833
Research 436,824 332,909 424,075
Public service 463 450 450
Student services 19,643 31,374 62,626
Administration and support services 491,953 366,841 534,924
Total Cost
Instruction $582,676 $854,483 $1,069,582
Research 732,638 864,668 848,149
Public service 5,802 5,950 6,150
Student services 58,821 81,487 125,251
Administration and support services 980,921 930,810 1,069,847
Finance Balance Sheet Items
Land improvements; ending balance $233,698 $238,269 $269,551
Buildings; ending balance 2,370,981 2,455,427 2,940,552
Equipment, including art and library; 1,150,228 1,191,801 1,372,257
ending balance
Total Capital $3,754,907 $3,885,497 $4,582,360
NOTE: FTE = full-time equivalent, IPEDS = Integrated Postsecondary Education Data System.
SOURCE: This, and all other tables in Chapter 4, were calculated by the panel and staff.
The "sheepskin effect" represents the additional value that credit hours have when
they are accumulated and organized into a completed degree. The panel believes
that a value equal to a year's worth of credits is a reasonable figure to use as a
placeholder for undergraduate degrees.3 Additional research will be needed to
determine the sheepskin effect for graduate and first professional programs.
3Jaegerand Page (1996) suggest something more than an additional year for the sheepskin effect.
They conclude "Sheepskin effects explain approximately a quarter of the total return to completing 16
OCR for page 67
ADVANCING THE CONCEPTUAL FRAMEWORK 67
4.2.3. Inputs
Inputs consist of the following variables, which can be calculated mainly
from IPEDS data as shown in Section 4.2.4.
a) Expenditures on Labor (LE): nominal value of salaries and wages plus
fringe benefits, used as the weight of L when aggregating the input.
b)Labor (L): the quantity measure for labor input, approximated by full-
time equivalent (FTE) employees. Both academic and nonacademic
employees are included in the calculation (this assumption, driven by
the limitations in IPEDS data categorization, is relaxed in Section 4.4).
FTE figures are calculated from total full- and part-time employees, with
a part-time employee counting as one-third of a full-time employee, as
assigned in IPEDS (this, too, could be adjusted with empirical justifica-
tion). Labor is the biggest input into higher education instruction.
c) Expenditures on Intermediate Inputs (IE): nominal cost of materials
and other inputs acquired through purchasing, outsourcing, etc. (the
sum of the IPEDS "operations & maintenance" [O&M] and "all other"
categories). These nominal values are used in calculating weights for
intermediate inputs.
d) Intermediate Inputs (I): Deflated nominal expenditures (IE) are used to
represent the physical quantities.
e) Expenditures on Capital (KE): opportunity cost for the use of physical
capital; also called rental value of capital. Expenditures equal the IPEDS
book value of capital stock times an estimated national rate of return
on assets, where book value of capital stock equals the sum of land,
buildings, and equipment.4 Overall, the book value reported in IPEDS
is likely too low; however, it does include buildings that may not be
specifically allocated to teaching, which offsets the total to an unknown
degree.5 These nominal capital values are used in calculating the capital
weights.
years of education and more than half of the return to completing 16 years relative to 12 years. . . . The
marginal effect of completing a Bachelor's degree over attending `some college' is 33%, conditional
on attending school for 16 years." Park (1999) found the sheepskin effect to be somewhat lower. Wood
(2009) provides a review of the literature. See also Section 5.1.1.
4Book value is typically defined as the original cost of an asset adjusted for depreciation and
amortization.
5An alternative option was considered: current replacement value, the cost to replace an asset or
a utility at current prices. This figure is available in IPEDS estimates of current replacement value
for educational institutions and also calculated by Sightlines, a private company (see http://www.
sightlines.com/Colleges-Universities_Facilities.html [June 2012]). Sightlines calculates current re-
placement value based upon the age, function, and technical complexity of each building. Current
replacement value is defined as the average cost per gross square foot of replacing a building in kind
in today's current dollar value. The Sightlines figures reflect the total project cost, including soft
OCR for page 68
68 IMPROVING MEASUREMENT OF PRODUCTIVITY IN HIGHER EDUCATION
f)Capital (K): For the quantity of capital input, the book value is deflated
by the Bureau of Economic Analysis's investment deflator for gross
private domestic investment.
The deflators for intermediate expenditures and capital are, respectively, the
Producer Price Index (PPI) and the index for Gross Private Domestic Investment:
Chain-Type Price Index (GPDICTPI). These figures cannot be obtained from
IPEDS but they are available from standard sources.6
4.2.4. Allocations to Education
The first step in the allocation process is to isolate inputs for the institutions'
educational function from those attributable to the research and public service
functions. Because IPEDS does not break out the FTE and capital variables by
function,7 our approach is to allocate these variables proportionally to expen-
ditures by function--which are available in IPEDS. Here, "education" means
"Education and Related Cost" (E&R), as defined by the Delta Project on Post-
secondary Education Costs as "Instruction plus Student Services" (Delta Cost
Project, 2009).
The allocation formulas are:
L = FTE(EdShAllLE + EdShDirLE × AdShAllLE)
LE = DLEI + DLES + DLEA × EdShDirLE
I = IE/PPI
IE = DIE + DIS + DIA × EdShDirIE
K = KE/GPDICTPI
KE = Stock × ROR(EdShAllTot + EdShDirTot × AdShAllTot)
costs, and are adjusted for architectural significance and region. Of course there is scope for further
exploration and refinements in this estimate which is best left to the judgment of college/university
authorities. As per e-mail exchange between one panel member and Jim Kadamus (vice president of
Sightlines), the company's staff conducted some preliminary comparisons between current replace-
ment values as calculated by IPEDS and Sightlines. The Sightlines estimates (for comparable space)
are 70-100 percent greater than the value reported in IPEDS. The entity charged with implementing
the productivity model will have to decide which estimate to use.
6 The PPI is available at http://www.bls.gov/ppi/ and the GPDICTPI is available from http://www.
bea.gov via the GDP and personal income interactive data link.
7The IPEDS Human Resources section provides a functional breakdown for direct teaching,
research, and public service staff, but only an occupational breakdown for nonteaching staff. This
scheme does not map into our model, and in any case the functional breakdown may be unstable due
to inconsistencies in institutional classification schemes.
OCR for page 69
ADVANCING THE CONCEPTUAL FRAMEWORK 69
Additional variables (beyond those on p. 67) are defined as:
ˇ DLEI = "Direct labor expenditures for instruction" as given by IPEDS; no-
tations follows this format for student services (DLES) and administration.
ˇ EdShAllLE is "Education's share of all labor expenditures"; AdShAllLE is
"Administration and support services' share of all labor expenditures";
notation follows this format for intermediate expenditures and capital.
Finally, EdShDirLE is "Education's share of direct labor expenditures,"
and similarly for all the other shares. The difference between EdShAllLE
and EdShDirLE is that the former's denominator includes labor expen-
ditures for administration and support services whereas the latter's does
not.
ˇ IE is expenditures on intermediate inputs; DIE is direct nonlabor ex-
penditures on instruction; DIS is direct nonlabor expenditures on stu-
dent services; DIA is direct nonlabor expenditures on administration;
EdShDirIE is education share of direct nonlabor expenditures.
ˇ Stock is capital stock as shown on institutional balance sheets, ROR is
the national rate of return on capital, PPI is the producer price index,
and CPDICIPI is the price index for gross private domestic investment
(both price indices are suitably normalized).8
Faculty time that is separately budgeted for institutional service is included in
administration and support services, and unbudgeted faculty service time (e.g.,
departmental administration) is included in instruction.
4.2.5. Illustrative Productivity Calculations
Table 4.2 shows the productivity calculation for the institution referred to
above. The calculation can be broken down into four steps:
1. Allocate the quantity and expenditure data to the Education function:
Apply the formulas above. For example, adjusted credit hours in the
three periods equal 638,435, 643,476, and 677,000.
2. Calculate the change in the quantity data from period to period: The
change for adjusted credit hours equals the current value divided by the
8See Hodge et al. (2011:25, Table 2). The rate of return to the net stock of produced assets for other
nonfinancial industries is used as a proxy for the rate of return to higher education land, buildings
and equipment. Other industries includes agriculture, forestry, fishing and hunting; transportation
and warehousing; information; rental and leasing services and lessors of intangible assets; profes-
sional, scientific, and technical services; administrative and waste management services; educational
services; health care and social assistance; arts, entertainment, and recreation; accommodation and
food services; and other services, except government.
OCR for page 70
70 IMPROVING MEASUREMENT OF PRODUCTIVITY IN HIGHER EDUCATION
TABLE 4.2 Base Model Productivity Calculations
Period 1 Period 2 Period 3
Step 1: Allocations to Education
Outputs
Adjusted credit hours (ACHs) 638,435 643,476 677,000
Input Quantities
Labor FTEs (L) 3,926 4,296 4,705
Intermediate expenditures $324,680 $486,147 $643,599
Rental value of capital (K) $261,834 $267,507 $348,033
Input Expenditures
Wages and fringe benefits $756,399 $867,311 $1,036,594
Intermediate expenditures $324,680 $550,921 $777,193
Retail value of capital (K) $261,834 $301,954 $400,791
Total Cost $1,342,913 $1,720,186 $2,214,578
Step 2: Quantity Changes Period 1 Period 1 2 Period 2 3
Output Change
Adjusted credit hours 1.000 1.008 1.052
Input Change
Labor FTEs 1.000 1.094 1.095
Real intermediate expenditures 1.000 1.497 1.324
Real capital stock 1.000 1.022 1.301
Step 3: Input Index
Weights (average)
Wages and fringe benefits 53.4% 48.6%
Normal intermediate expenditures 28.1% 33.6%
Real capital stock 18.5% 17.8%
Weighted geometric average 1.180 1.204
Step 4: Multi-Factor Productivity
Productivity index 0.854 0.874
Productivity change 2.3%
NOTE: FTE = full-time equivalent.
previous one, with the first value being initialized at one. The ratios for
Periods 2 and 3 are 1.008 and 1.052, for example, which indicate growth
rates of 0.8 percent and 5.2 percent.
3. Calculate the input index: For inputs, the composite index ("weighted
geometric average") is equal to the geometric average of the indices for
the individual variables using the arithmetic average of the successive
periods' nominal expenditure shares as weights (no averaging is needed
for outputs because there is only one output measure).9 This calculation
9The index for the more complicated models presented later is based on the geometric average of
output changes, the same as for inputs.
OCR for page 71
ADVANCING THE CONCEPTUAL FRAMEWORK 71
BOX 4.1
Productivity and Quality
The productivity measure here does not take account of quality changes.
Instead, it depends on the market to police quality erosion. Normally such polic-
ing is done through the price mechanism, although sometimes products such as
computers whose quality is increasing over time do become cheaper. As argued
earlier in this report, higher education prices generally are not set in competitive
markets. Hence the conclusion, "Productivity has increased by `x' percent" must
be taken as tentative until the constancy of quality has been verified, for example,
through a separate quality assurance procedure. A master artist produces 10
paintings in a month; her student also produces 10 paintings in the same period,
for example. We sense that the quality of the artists is different, and that this dif-
ference should be reflected in the final product. But we cannot tell the difference in
productivity just by counting the hours and the paintings (nor can we quantify the
quality difference just by looking at them)--though we may eventually get some
evidence by tracking the price that the paintings sell for or whether they sell at all.
We return to the question of output quality in Section 4.6.
comes from the third equation in the Technical Appendix. The results are
1.180 and 1.204, which indicate average input growths of 18.0 percent
and 20.4 percent.
4. Calculate the productivity index: This is expressed as the ratio of the
index for the change in outputs to the index for the change in inputs--
i.e., 1.008/1.180 = 0.854 for the first period and 1.052/1.204 = 0.874
for the second. The last line, "productivity change," is the ratio of the
productivity indexes for the two periods: (0.854/0.874 1), or 2.3
percent.
An alternative but equivalent calculation illuminates Step 4. Notice that the out-
put index grew by 1.052/1.008 = 1.044 (4.4 percent) and the input index grew
by 1.204/1.180 = 1.020 (2.0 percent) between the second and third periods.
Dividing the output growth by the input growth yields 1.044/1.020 = 1.023, or
2.3 percent, the same as in the table. Put another way, the output index grew 2.3
percent faster than the input index--which represents productivity improvement
(see Box 4.1).
4.3. INSTITUTIONAL SEGMENTATION
AND DISAGGREGATIVE INDICES
Having established the basic productivity index, we now present refine-
ments intended to make it more useful. This section describes how indices can
OCR for page 76
76 IMPROVING MEASUREMENT OF PRODUCTIVITY IN HIGHER EDUCATION
TABLE 4.3 Additional IPEDS Data for the Differential Labor Market
Period 1 Period 2 Period 3
Quantities
Number of FTEs--instruction, research, and public 1,564 1,601 1,1601
service staff with faculty status
Number of PT/PI staff 2 2 2
Number of other staff 4,928 6,704 5,974
Expenditures
Expenditures on wages and salaries for instruction, $529,572 $701,791 $874,051
research, and public service
Expenditures on fringe benefits for instruction, 193,203 361,064 193,373
research, and public service
Expenditures on wages and salaries for other 380,610 410,296 482,139
functions
Expenditures on fringe benefits for other functions 147,449 203,686 119,510
Average salary for FT instructional, staff 108,200 114,464 122,508
NOTE: FT = full-time, FTE = full-time equivalent, PT/PI = part-time/primarily instruction.
come directly from IPEDS, with those for other functions obtained by subtrac-
tion. Average salary for full-time instructional staff comes from the corresponding
table in IPEDS. The very small PT/PI share may be an artifact of the particular
data used in the example, but because the number is growing across the sector
we believe this variable remains worthy of consideration.
Only one required data element is unavailable in IPEDS: the ratio of PT/PI
salaries per FTE to those of regular faculty. It is possible (though perhaps not
cost-effective) that IPEDS could be expanded to get this information; absent the
change it may be adequate to assume the compensation ratio or determine it by
special study.11 Finally, the expenditures for tenure-track and adjunct faculty
expenditures are subtracted from total labor expenditures to get the figure for
other labor.
Table 4.4 illustrates the calculations. The first step is to allocate the quantities
and expenditures to the Education function. The figures for full-time tenure-track
faculty and other staff are portioned using the education share variable computed
in the base model. PT/PI staff needs no allocation because they are "primarily
instruction" to start with. The FTE figures for Period 3 have been adjusted to
demonstrate the effect of substituting PT/PI staff for full-time tenure-track fac-
ulty (discussed below): specifically, we subtracted 100 instructional FTEs from
tenure-track faculty and added them to PT/PI.
11One-third may be a reasonable approximation. We obtained this figure by assuming that (a) PT/
PI staff get about $5,000 per course with no benefits, (b) average annual faculty salary plus benefits
is $90,000, and (c) the average full-time faculty member teaches 6 courses a year. In this case the
calculation is $5,000/($90,000/6) = 1/3.
OCR for page 77
ADVANCING THE CONCEPTUAL FRAMEWORK 77
TABLE 4.4 Differentiated Labor Index Calculations
Period 1 Period 2 Period 3
Allocations to Education
Quantities
Regular faculty FTEs 946 828 828
PT/PI staff FTEs 2 2 2
Other staff FTEs 2,978 3,466 3,875
Expenditures
Effective fringe benefit rate 36.5% 51.4% 22.1%
Regular faculty $139,667 $143,552 $123,891
PT/PI staff $28 $32 $30
Other staff $616,704 $723,726 $912,673
Total expenditures $756,399 $867,311 $1,036,594
Index Calculations
Quantities
FT regular faculty 1.000 0.876 1.000
PT/PI staff 1.000 1.000 1.000
Other staff 1.000 1.164 1.118
Quantity Change Period 1 2 Period 2 3
FT regular faculty 0.876 1.142
PT/PI staff 1.000 1.000
Other staff 1.164 1.118
Weights
FT regular faculty 17.5% 14.3%
PT/PI staff 0.00% 0.00%
Other staff 82.5% 85.7%
Labor Index
Geometric average 1.107 0.985
Change in the average (0.110)
NOTE: FT = full-time, FTE = full-time equivalent, PT/PI = part-time/primarily instruction.
Calculating the expenditures requires the effective fringe benefits rate to be
computed as a preliminary step. (It equals the ratio of fringe benefits to salaries
and wages in Table 4.3.) The expenditure figures can be computed as follows:
Expenditures for regular faculty = (1+ Fringe benefits rate) × Average salary
for FTE instructional staff × Regular faculty FTEs;
Expenditures for PT/PI staff = (1+ Fringe benefits rate/2) × PT/PI salary ratio
× Average salary for FTE instructional staff × PT/PI FTEs; and
Expenditures for other staff = Total wages + fringe benefits (Expenditures
for regular faculty and PT/PI staff).
OCR for page 78
78 IMPROVING MEASUREMENT OF PRODUCTIVITY IN HIGHER EDUCATION
Total "Wages & fringe benefits" is taken from Table 4.1, and readers may notice
that PT/PI staff receive only half the fringe benefits rate.
The final step, calculating the change indices and their weights, proceeds
as in the base model. As shown in Table 4.4, resulting geometric averages show
more variation than did the labor indices in Table 4.2. This produces larger multi-
factor productivity indices, and also a larger change in the index, as shown at the
bottom of the table. Which approach is more accurate may become more clear
during implementation.
4.5. DIFFERENTIATING OUTPUTS
Another potentially key model enhancement is to control for the heteroge-
neity of educational outputs. Institution-level cost data indicate clearly that the
resources required for producing an undergraduate degree vary across fields.
Likewise, the resource requirements for producing bachelor degrees differ sys-
tematically from those for associate, graduate, and first professional degrees. Fail-
ure to control for these differences would risk the kinds of distortions described
earlier. For example, a shift of outputs from the more expensive disciplines of
science, technology, engineering, and mathematics (STEM) to non-STEM disci-
plines would falsely boost the productivity index.
IPEDS provides data on degrees by field and award level. A difficulty arises
only in differentiating the credit hours associated with degree production. It is
not unusual for institutions to track credit hours by the department or broad dis-
cipline in which a course is taught (as required to apply the so-called Delaware
cost benchmarks for example), but these data cannot be mapped directly to degree
production because students take many courses outside their matriculated area.
Researchers have made the necessary correspondences by creating course-taking
profiles for particular degrees, but these matrices are difficult to manage and
maintain on an institution-wide basis. There is a better way, which we outline
below--one that feeds directly into the productivity statistics and produces the
course-taking profiles as by-products.12
For the long run, credit-hour data for productivity analysis should be col-
lected in a way that follows the students, not only the departments that teach
them. The necessary information exists in most institutions' student registration
files and the needed statistics can be extracted as follows:
ˇ Identify the students matriculated in a given degree program ("output
category") as defined by the IPEDS fields and degree levels. Undeclared
students and students not matriculated for a degree would be placed in
separate ("nonattributable") output categories.
12Simply applying a sheepskin effect to each degree category and summing the result before add-
ing to aggregate credit hours is insufficient because shifts in degree production will induce shifts in
credit-hour production, which will produce the kinds of distortion described in the text.
OCR for page 79
ADVANCING THE CONCEPTUAL FRAMEWORK 79
ˇ For each output category, accumulate the credit hours earned by the stu-
dents in that category, regardless of the department in which the course
was offered or the year in which it was taken.
ˇ Allocate credits earned by matriculated but undeclared students in pro-
portion to the credit-hour fractions of declared students for the given
degree. Retain nonmatriculated students in their own separate category,
one that has no sheepskin effect but in other respects is treated the same
as other categories.
The question arises as to whether these data need to be collected by each
institution, or whether generalized weights based on special studies (e.g., using
the Postsecondary Educational Transcripts Study, PETS) could be used. The use
of generalized weights is not inconceivable, but we worry that the heterogeneity
of programs from school to school means that much information would be lost.
Further, institutions may find the data on credit hours by degree program useful
for internal purposes as well as for reporting--for example, in studying student
profiles of course-taking behavior and benchmarking costs per degree program. It
seems likely that, once in hand, these data will open significant new opportunities
for institutional research.
It may be some time before data on student-based credit hour accumulations
can be obtained, but there is a simple interim procedure that can be computed
from the available IPEDS data. It is to allocate total enrollments to fields and
levels based on fractions of completion. While ignoring differences in the course-
taking profiles, the procedure does allow at least some differentiation among
output categories.
Aggregation to a single output index is best accomplished by taking a geo-
metric average of the output category indices using their net student revenue
shares as weights. A geometric average with weights equal to revenue shares
reflects the BLS methodology described earlier. Use of net as opposed to gross
shares appears reasonable because revenue based on net prices (which is what the
institution can spend on operations) is consistent with the underlying Törnqvist
model as presented by Caves, Christensen, and Diewert (1982).
We recognize that getting these data may be problematic for institutions, in
which case an alternative based on reported tuition and financial aid rates might
well suffice. We also should note that it is not necessary to allocate any inputs
across output categories. Such a requirement would disrupt data collection. Like
most sectoral productivity indices, ours simply divides the aggregate output index
for a segment by its aggregate input index.
4.6. VARIATIONS IN OUTPUT QUALITY
The quality of education is the elephant in the room in all discussions about
instructional productivity, and the issue has been raised repeatedly in this report.
OCR for page 80
80 IMPROVING MEASUREMENT OF PRODUCTIVITY IN HIGHER EDUCATION
The panel would have liked nothing better than to propose an explicit quality
adjustment factor for weighted credit hours as part of our conceptual framework.
In our model, an effect will be captured to the extent that higher quality inputs
lead to higher graduation rates (and, in turn, a larger degree bonus in the numera-
tor), but this effect is indirect. For example, if small classes or better teaching
(inputs of different quality) lead to higher graduation rates, this will appear in the
output numerator as a greater sheepskin effect. Similarly, high student and teacher
quality at selective private institutions may offset high input costs by creating an
environment conducive to high throughput rates.
This modest step notwithstanding, full (or even adequate) integration of qual-
ity adjustment into a productivity measure will not be possible any time soon.
Significant progress on quality assessment has been made, but there is a long
way to go before a generally accepted cardinal measure--one that can be used
reliably to adjust weighted credit hours--can be agreed upon. It is possible, and
perhaps even likely, that critics will call for a moratorium on all efforts to measure
instructional productivity until a valid and reliable output quality index can be
developed. We believe this would be unwise, for two reasons.
First, the kind of productivity measures proposed in this report is intended to
deal primarily with changes over time rather than comparisons across institutions.
It is true that an increasing focus on the quantitative aspects of productivity might
trigger a "race for the bottom" in educational quality as competing institutions
make increasingly larger concessions, seeking to boost the numerator and cut
the denominator of the productivity fraction. Pressures on enrollments relative
to budgets make this a danger whether quantitative productivity is properly mea-
sured or not, but there is no doubt that an increasing emphasis on the quantita-
tive elements of productivity could exacerbate the problem. Normalizing the
productivity index for each segment to 1.0, as is done in Section 4.3.1, will help
alleviate this danger--though of course it is always possible to manipulate data
to achieve desired results. Attention to the limitations of the metric for measuring
at low levels of aggregation is also important.
Second, failure to agree on an economically valid and technically robust
quantitative productivity measure will only increase proliferation of the weaker
measures described in Chapter 2. These are even more susceptible to missing
differences in quality than the method proposed here. Furthermore, failure to
implement a good measure would indefinitely defer the benefits achievable from
a better understanding of quantitative productivity even in the absence of quality
adjustment. These considerations suggest a strategy of simply assuming that, ab-
sent evidence to the contrary, the quality of outputs is not declining significantly
over time. The panel believes, albeit reluctantly given our desire for evidence,
that the general approach proposed has much to recommend it.
To broaden the applicability of the measure developed here, additional steps
should be taken. The essential idea is that effective and transparent quality assur-
ance systems should be maintained to insure that output quality does not race for
OCR for page 81
ADVANCING THE CONCEPTUAL FRAMEWORK 81
the bottom (Massy, 2010). These could be based on extant accreditation systems,
on the methods of academic audit being used effectively in Tennessee and certain
overseas venues (Massy, Graham, and Short, 2007), or on the other quality-
reviewing initiatives now being conducted at the state level. From a modeler's
perspective, the approach converts a quantity that would be included in the objec-
tive function if a cardinal measure were available to a "yes-no" constraint that
needs only binary measurement. The binary constraint amounts to what might be
called a "watchdog evaluation": remaining silent if all is well but sounding the
alarm if it is not. The watchdog evaluation would be at root subjective, but based
upon evidence; we return to this idea in Section 5.3.2.
The approach is by no means perfect. However, it allows progress to be made
in measuring the quantitative aspects of productivity while containing the risk of
triggering institutional competition that results in lowering educational quality.
Progress on the development of quantitative productivity measures may boost the
priority for developing a serviceable quality adjustment index.
OCR for page 82
82 IMPROVING MEASUREMENT OF PRODUCTIVITY IN HIGHER EDUCATION
TECHNICAL APPENDIX
The Törnqvist Productivity Index
This Appendix briefly describes the theoretical basis and calculations for the
Törnqvist productivity index used in Chapter 4. The argument follows Caves,
Christensen, and Diewert (1982:1393). As noted in the text, the Törnqvist index
is used by the Bureau of Labor Statistics in calculating multi-factor productivity
change (cf. Bureau of Labor Statistics, 2007). The appendix text is adapted from
Section 1 of Massy (2012). Where applicable, the equations are keyed to Steps
2-4 of Table 4.2.
Input Indices, Distance Functions, and Productivity
We define "Firm k" and "Firm l" as two enterprises whose productivity is
to be compared. Standard usage takes the two to be the same organization at
different time periods, but this is not a requirement of the Törnqvist theory. For
example, the two could be separate enterprises operating in the same or different
periods.
At their Eq. 6, Caves and colleagues (1982) define the Malmquist input index
for Firm k with respect to the inputs of Firm l as:
(1) Dk ( yk , x l )
Qk (x l , x k ) = ,
Dk ( yk , x k )
where yk is an m-element vector of outputs and xk is an n-element vector of inputs
for firm k. The numerator of the right-hand side, called "Firm k's input distance
function with respect to the inputs of Firm l," is defined at Eq. 7 as:
xl
(2) D k ( y k , x l ) = max { : F k y
k ( ) y },
k
l
k k k
where Fk(yk,xk) = 0 is k's production function, y k is the vector {y2 , y3 ,... yn },
and y1 = F ( y
k k
, x ). Maximizing determines the "minimum Firm l input mix
k
required to produce Firm k's output using k's production function"--i.e., it "de-
flates" kl onto k's production function. From this it can be seen that Dk (yk, xk),
the denominator in (1), always must equal one because any firm can produce its
own output using its own inputs. This means Q k ( x l , x k ) = D k ( y k , x l ) 1, which
implies that "the input vector of firm l, xl, is `bigger' than the input vector of firm
k (i.e., k, xk) from the perspective of Firm k's technology" (Caves, Christensen,
and Diewert, 1982:1396). A similar argument applies to Ql (xl,xk).
Suppose now that Firms k and l are in fact the same entity, observed at times
t1 and t2 respectively, and Qk (xl,xk) 1. This means the technology used at time
OCR for page 83
ADVANCING THE CONCEPTUAL FRAMEWORK 83
would have required more inputs to produce the t2 outputs than does the technol-
ogy at t2 (this conclusion does not require the outputs at t1 and t2 to be the same).
In other words, there has been a productivity improvement.
A Practical Method of Computation
This is a powerful result, but the calculation cannot be done without detailed
knowledge of the production function and its parameters--knowledge that is
rarely if ever available. Caves and his co-authors surmount this difficulty by in-
voking classic profit maximization--an assumption that is dubious when applied
to higher education institutions (see below)--together with a modest technical
simplification. In their words:
By making use of a specialized functional form and the assumption of cost-
minimizing behavior, it is possible to compute a geometric average of the two
Malmquist indices and Qk(xl,xk), using only observed information on input pro-
cesses and quantities. We demonstrate this fact for the case in which each firm
has a translog distance function, but the properties of the two translog functions
are allowed to differ substantially. In this case the geometric average of the two
Malmquist indices turns out to be a Törnqvist index. (Caves, Christensen, and
Diewert, 1982:1397).
The technical simplification is that the production functions' cross-product
parameters for inputs and outputs be equal within and across firms. The authors
point out that these restrictions are not onerous because "The translog distance
function is capable of providing a second-order approximation to an arbitrary
distance function. Thus the technologies in the two firms can be virtually arbi-
trary (to the second-order) except for the restrictions" [Caves, Christensen, and
Diewert, 1982:1398]. Nonetheless, research will be needed to assess whether or
not these restrictions are reasonable in the higher education context.
The resulting theorem, Caves Equation (15) reproduced below, shows how
one can compute the geometric average of the two input indices using only ob-
servable data. This is all that's needed to estimate productivity change.
k
×w k
w lj × wn
l
(3)1
2
1
2
1 n
ln Q k ( x l , x k ) + ln Q l ( x l , x k ) =
2 j =1
[wj
w xk k
n
+
w l x l
][ln x l
j j ].
- ln x k
The new symbols wk and wl are vectors of input prices and wk ˇ xk is total ex-
penditure, the dot-product of price and quantity. This means the fractions repre-
sent shares of input expenditure. Hence the right-hand side defines the log of a
Törnqvist index, denoted by: namely, the geometric average of physical inputs
using expenditure shares as weights.
OCR for page 84
84 IMPROVING MEASUREMENT OF PRODUCTIVITY IN HIGHER EDUCATION
The quantity Q(wl,wk,xl,xk) is shown for the transitions from Period 1 to
Period 2 and Period 2 to Period 3, in the second-to-last line of Table 4.2 (the
first line of step 4). Step 2 of Table 4.2 shows the inputs for the quantity-change
(i.e., the last) term in the right-hand side of (3): i.e., in x lj . Step 3 shows the
weights and the resulting geometric averages of the quantity changes. Notice that
the weights are themselves an average of the weights in the two periods being
considered.
The Overall Productivity Measure
Section 4 of Caves, Christensen, and Diewert (1982:1401), points out that
there are "two natural approaches" to the measurement of productivity changes:
differences in maximum output conditional on a given level of inputs ("output-
based" indices) and those based on minimum input requirements conditional on
a given level of output ("input-based" indices). Furthermore, the two approaches
"differ from each other by a factor that reflects the returns to scale of the produc-
tion structure." Without going into the details, it is intuitively reasonable that a
geometric average of the output-based and input-based indices represents a good
overall measure of productivity.13 Therefore, the desired productivity index is
the ratio of the output-based and input-based Törnqvist indices. Changes in pro-
ductivity are obtained by taking the ratios of the indices in successive periods.
Step 4 of Table 4.2 performs these final calculations. The first line is the
productivity index itself: the ratio of the output to the input indices. The second
line shows productivity change: the ratio of the indices in the two successive
periods, minus one.
Applying the Index to Nonprofit Enterprises
The proof of optimality for the Törnqvist index depends on the assumption
of profit maximization. As noted above, this assumption is dubious when applied
to traditional universities (it applies perfectly well to for-profit universities, how-
ever). This leads to two questions that need to be addressed by further research:
(1) Will application of the index to nonprofits produce misleading results? (2)
What modifications to the Törnqvist (or perhaps an entirely different approach)
will likely be better than the traditional index?
Regarding the first question, we note that a lack of optimality is not equiva-
lent to a lack of efficacy. Many algorithms and measures are used, in economics
13Adjustments for decreasing and increasing returns to scale are described in later sections of the
Caves paper. They are interesting but, we believe, not of particular concern to productivity measure-
ment in colleges and universities. While scale economies in higher education are intuitively plausible,
it appears that significant size increases are likely to trigger institutional responses (e.g., scope or
support service increases) that tend to drive up costs (cf., Brinkman, 1990:120). Our model does not
adjust for scale effects.
OCR for page 85
ADVANCING THE CONCEPTUAL FRAMEWORK 85
and elsewhere, whose optimality cannot be proved or that have even been shown
to be suboptimal. The question is an open one which calls for additional research.
Regarding the second question, Massy (2012) has proposed a modification to
Equation (3) to achieve optimality in the nonprofit case. Still, additional research
will be needed to determine how the required new parameters can be estimated
and to identify the conditions under which the new model produces results that
differ materially from the traditional one. The question of material differences
will, in turn, shed light on the answer to question (1).
OCR for page 86
