National Academies Press: OpenBook

Improving Measurement of Productivity in Higher Education (2012)

Chapter: 4 Advancing the Conceptual Framework

« Previous: 3 Why Measurement of Higher Education Productivity Is Difficult
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

4

Advancing the Conceptual Framework

This chapter presents a framework intended to provide a starting point for measuring productivity in higher education. Chapters 2 and 3 presented arguments for why productivity measurement in higher education is exceedingly difficult and why, in turn, the panel cannot simply prescribe a fully defined metric. Nonetheless, because governments and many other stakeholders insist on, and in fact need, an aggregate measure of productivity change, it is important to begin developing the best measure possible.

The measure proposed involves a number of important assumptions and approximations, which are elaborated below. Chief among these is the lack of an agreed-upon measure of educational quality. Productivity should be defined as the ratio of quality-adjusted outputs to quality-adjusted inputs, but the needed quality adjustments are not currently possible in higher education and are not likely to become possible any time soon. We recognize the problem, but believe it is important to extract as much information as possible from the (quantitative) data that can be measured. We will describe later how the risks associated with the lacuna of measures of quality can be minimized including, for example, how entities can use university and third-party quality assurance methods to ensure that focusing on the quantitative inputs and outputs does not trigger a “race to the bottom” in terms of quality.

4.1. CHAPTER OVERVIEW

The productivity measure proposed here is consistent with the methodology practiced by the Bureau of Labor Statistics (BLS), and offers several significant advantages over the ad hoc approaches that have been used to date. In particular:

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
  • The measure is a multi-factor productivity index. It captures output in physical units (credit hours, degrees) and, unlike cost studies, measures direct labor inputs in terms of full-time equivalents (FTEs). Labor productivity can be derived from the multi-factor results if desired.
  • Outputs include credit hour production and degree attainment, both of which have been shown to be important in labor market studies. Most if not all the measures currently in use (e.g., credit hour production alone or graduation rates) depend on one or the other but not both, and therefore miss a critical output dimension.
  • The measure does not vary along with the proportion of part-time students, except to the extent that being part-time might require different student services or contributes to wasting credits or dropping out. This feature sidesteps the problem of comparing graduation rates and average times to degree among schools with different numbers of part-time students.
  • Credits not on the mainline path to a degree, including those due to changes in major and dropouts, are counted and thus dilute the degree completion effect. In other words, programs with a heavy dropout rate will have more enrollments per completion, which in turn will boost resource usage without commensurate increases in degrees. Productivity could thus increase with the same number of credit hours if more students actually complete their degrees. Credit earned, however, is not treated as entirely wasted just because a degree was not awarded.
  • The measure allows differentiation of the labor and output categories, although doing this in a refined way will require significant new data.
  • The measure readily lends itself to segmentation by institutional type, which is important given the heterogeneity of the higher education sector.
  • The measure can in principle be computed for institutions within a state, or even single institutions. However, the incentives associated with low-aggregation level analyses carry the risk of serious accuracy degradation and misuse unless it is coupled with robust quality assurance procedures. Until quality adjustment measures are developed, the panel advises against using the productivity metric described in this chapter for institution-to-institution comparisons (as opposed to more aggregate level, time series, or perhaps state-by-state or segment analyses).
  • Data collection, including data beyond the Integrated Postsecondary Education Data System (IPEDS) and the proposed special studies, appears to be feasible.

We emphasize again that the proposed measure follows the paradigm of aggregate productivity measurement, not the paradigm for provision of institution-level incentives and accountability. As stressed in Chapter 3, institutions should

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

be prepared to resist inappropriate initiatives to improve productivity as measured by applying the formula below to their particular data, and to buttress their resistance with their own internal data about quality.

Section 4.2 presents our base model. It is a “multi-factor productivity model” in that it uses output and input quantities and includes all categories of inputs. Section 4.3 proposes a segmentation scheme, which is important because of the heterogeneity of higher education. The section also discusses how the model can be computed at the state and single-institution level but, again, we stress that this will be dangerous without a robust quality assurance system. Section 4.4 enhances the base model by differentiating among labor categories. This is important because of the fundamental difference between academic and nonacademic labor, and the difference between tenure-track and adjunct faculty. Section 4.5 differentiates among output categories, which again is important because of institutional heterogeneity and the fact that production of degrees at different levels and in different fields involves different production functions. Finally, Section 4.6 presents the rationale for using the model in conjunction with quality assurance procedures.

Nearly all the data required for calculating values using the model sketched out here can be obtained from the U.S. Department of Education’s IPEDS or other standard public sources (though this would not be the case for the fully specified “ideal”). Adding the model refinements outlined in Section 4.3 requires a modest amount of additional information. Data requirements for the enhancements described in Section 4.4 can be approximated from IPEDS, but proper implementation will require additional data collection. The panel’s recommended changes to IPEDS are discussed in detail in Chapter 6. The new data that are called for would break useful ground not only for productivity analysis, but also for institutional planning and resource allocation. This is important because an institution’s use of data for its own purposes makes data collection more palatable and improves accuracy.

4.2. A BASELINE MULTI-FACTOR PRODUCTIVITY MODEL FOR HIGHER EDUCATION

Following the concepts defined in Chapter 2, the model calculates the ratio of changes in outputs (credit hours and degrees) to inputs (labor, purchased materials, and capital). The focus is on instructional productivity, with inputs being apportioned among instruction, research, and public services prior to calculating the productivity ratio. As emphasized throughout this report, our model involves only quantitative factors. It will be reliable only to the extent that input and output quality remains approximately constant, or at least does not decline materially. Currently, quality measurement—of both inputs and outputs—is largely beyond the capacity of quantitative modeling; but, because quality should never be taken for granted, we return to the issue at the end of the chapter.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

4.2.1. Multi-Factor Productivity Indices

Multi-factor indices relate output to a bundle of inputs; that is, they show how much of an industry’s or firm’s output growth can be explained by the combined changes in its inputs. The panel has concluded that a multi-factor productivity index is appropriate for measuring higher education productivity at the segment and sectoral levels. Other kinds of productivity models—for example, those which estimate educational production functions—are of course possible and worthwhile. However, the panel was not charged with recommending such models. Nor was it charged with developing strategies for improving productivity.

Our proposed productivity model is based on the methodology for multi-factor productivity indices used by BLS, the OECD, and other U.S. and foreign agencies that produce sectoral productivity statistics (Bureau of Labor Statistics, 2007; Schreyer, 2001). For example, BLS uses this methodology to calculate productivity indices for aggregate manufacturing and some eighteen manufacturing industries.

The BLS method uses what is known as a Törnqvist index. This differs in important ways from the method of simply calculating weighted averages of the variables in the numerator and denominator and then taking the ratio of the two averages. The key ideas behind the Törnqvist index are as follows (from Bureau of Labor Statistics [2007:6-7]):

  1. The figures for input and output are calculated as weighted averages of the growth rates of their respective components. Weighting average growth rates avoids the assumption, implicit in directly averaging the variables, that the inputs are freely substitutable for one another. It also removes issues having to do with the components’ dimensionality. Both attributes are important when comparing variables like adjusted credit hours with labor and other inputs.
  2. The weights are allowed to vary for each time period in which the index is calculated. This means the index always represents current information about the relative importance of the variable in question while maintaining the requirement (discussed in Chapter 2) that the weights move more slowly than the variables themselves.
  3. The weights are defined as the means of the relative expenditure or revenue shares of the components for the two data periods on which the current index is based. This method brings relative wages and prices into the equation because they affect total expenditures.

The Törnqvist scheme has often been the indexing structure of choice for describing multi-factor productivity change under fairly broad and representative assumptions about the nature of production: specifically, that the production function can be represented by a translog generalization of the familiar Cobb-Douglas

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

function with mild regularity conditions on the parameters (Caves et al., 1982). A technical description of the Törnqvist methodology is provided in the appendix to this chapter.

Determining appropriate indices embodying the general ideas put forward in Chapters 2 and 3 remains a task for future work. As such, it would be premature for the panel to commit to a specific approach. However, because of its widespread use in other applications—a Törnqvist index is used here for expository purposes. The denominator of our baseline higher education productivity index uses a Törnqvist structure to represent the composite growth rates of labor and capital inputs. The numerator also takes the form of a Törnqvist index, though in this case with only one element. Section 4.4 extends the index in the denominator to include more than one labor category, and Section 4.5 uses multiple output categories in the numerator. The final productivity index is the ratio of the indices in the numerator and the denominator.

4.2.2. Outputs

On the output side, the model uses two data elements that can be obtained from IPEDS:

  1. Credit Hours: 12-month instructional activity credit hours summed over undergraduates, first-year professional students, and graduate students;
  2. Completions: awards or degrees conferred, summed over programs, levels, race or ethnicity, and gender.1

Illustrative data for a four-year private university are shown in Table 4.1.2 The data cover three years: 2003, 2006, and 2009 (it is best to aggregate over multiyear periods to reduce volatility associated with noise in the data, but the illustration ignores that refinement). For reasons explained earlier, both credit hours and degrees (or completions) are included as outputs. Whatever their flaws, these are the standard unit measures of instruction in American higher education.

The model uses adjusted credit hours as its measure of output, defined as follows:

Adjusted credit hours = Credit hours + (Sheepskin effect × Completions)

________________

1For broader use, definitions become more complicated. For example, as discussed in the next chapter and elsewhere in the report, “completions” defined as certificates and successful transfers become relevant in the community college context. Nondegree seekers (e.g., summer transients) also come into play at many kinds of institutions.

2The data are based on an actual institution, but certain adjustments were made to make the illustration more coherent.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

TABLE 4.1 Illustrative Data from IPEDS for the Base Model

  Period 1 Period 2 Period 3
Enrollments and Completions

Credit hours

578,815 574,176 602,000

Completions

2,154 2,310 2,500

Adjusted credit hours

638,435 643,476 677,000
Total Number of Staff

Full time

6,265 6,656 6,826

Part time

683 4,949 2,250

Labor FTEs

6,493 8,306 7,576
Finance: Core Expenditures

Wages and Fringe Benefits

Instruction

$421,534 $525,496 $641,749

Research

295,814 531,759 424,075

Public service

5,339 5,500 5,700

Student services

39,178 50,113 62,626

Administration and support services

488,969 563,969 534,924

Intermediate Expenditures

Instruction

$161,142 $328,987 $427,833

Research

436,824 332,909 424,075

Public service

463 450 450

Student services

19,643 31,374 62,626

Administration and support services

491,953 366,841 534,924

Total Cost

Instruction

$582,676 $854,483 $1,069,582

Research

732,638 864,668 848,149

Public service

5,802 5,950 6,150

Student services

58,821 81,487 125,251

Administration and support services

980,921 930,810 1,069,847
Finance Balance Sheet Items

Land improvements; ending balance

$233,698 $238,269 $269,551

Buildings; ending balance

2,370,981 2,455,427 2,940,552

Equipment, including art and library; ending balance

1,150,228 1,191,801 1,372,257

Total Capital

$3,754,907 $3,885,497 $4,582,360

NOTE: FTE = full-time equivalent, IPEDS = Integrated Postsecondary Education Data System.

SOURCE: This, and all other tables in Chapter 4, were calculated by the panel and staff.

The “sheepskin effect” represents the additional value that credit hours have when they are accumulated and organized into a completed degree. The panel believes that a value equal to a year’s worth of credits is a reasonable figure to use as a placeholder for undergraduate degrees.3 Additional research will be needed to determine the sheepskin effect for graduate and first professional programs.

________________

3Jaeger and Page (1996) suggest something more than an additional year for the sheepskin effect. They conclude “Sheepskin effects explain approximately a quarter of the total return to completing 16 years of education and more than half of the return to completing 16

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

4.2.3. Inputs

Inputs consist of the following variables, which can be calculated mainly from IPEDS data as shown in Section 4.2.4.

a)   Expenditures on Labor (LE): nominal value of salaries and wages plus fringe benefits, used as the weight of L when aggregating the input.

b)   Labor (L): the quantity measure for labor input, approximated by full-time equivalent (FTE) employees. Both academic and nonacademic employees are included in the calculation (this assumption, driven by the limitations in IPEDS data categorization, is relaxed in Section 4.4). FTE figures are calculated from total full- and part-time employees, with a part-time employee counting as one-third of a full-time employee, as assigned in IPEDS (this, too, could be adjusted with empirical justification). Labor is the biggest input into higher education instruction.

c)   Expenditures on Intermediate Inputs (IE): nominal cost of materials and other inputs acquired through purchasing, outsourcing, etc. (the sum of the IPEDS “operations & maintenance” [O&M] and “all other” categories). These nominal values are used in calculating weights for intermediate inputs.

d)   Intermediate Inputs (I): Deflated nominal expenditures (IE) are used to represent the physical quantities.

e)   Expenditures on Capital (KE): opportunity cost for the use of physical capital; also called rental value of capital. Expenditures equal the IPEDS book value of capital stock times an estimated national rate of return on assets, where book value of capital stock equals the sum of land, buildings, and equipment.4 Overall, the book value reported in IPEDS is likely too low; however, it does include buildings that may not be specifically allocated to teaching, which offsets the total to an unknown degree.5 These nominal capital values are used in calculating the capital weights.

_________________

years relative to 12 years…. The marginal effect of completing a Bachelor’s degree over attending ‘some college’ is 33%, conditional on attending school for 16 years.” Park (1999) found the sheepskin effect to be somewhat lower. Wood (2009) provides a review of the literature. See also Section 5.1.1.

4Book value is typically defined as the original cost of an asset adjusted for depreciation and amortization.

5An alternative option was considered: current replacement value, the cost to replace an asset or a utility at current prices. This figure is available in IPEDS estimates of current replacement value for educational institutions and also calculated by Sightlines, a private company (see http://www.sightlines.com/Colleges-Universities_Facilities.html [June 2012]). Sightlines calculates current replacement value based upon the age, function, and technical complexity of each building. Current replacement value is defined as the average cost per gross square foot of replacing a building in kind in today’s current dollar value. The Sightlines figures reflect the total project cost, including soft

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

f)   Capital (K): For the quantity of capital input, the book value is deflated by the Bureau of Economic Analysis’s investment deflator for gross private domestic investment.

The deflators for intermediate expenditures and capital are, respectively, the Producer Price Index (PPI) and the index for Gross Private Domestic Investment: Chain-Type Price Index (GPDICTPI). These figures cannot be obtained from IPEDS but they are available from standard sources.6

4.2.4. Allocations to Education

The first step in the allocation process is to isolate inputs for the institutions’ educational function from those attributable to the research and public service functions. Because IPEDS does not break out the FTE and capital variables by function,7 our approach is to allocate these variables proportionally to expenditures by function—which are available in IPEDS. Here, “education” means “Education and Related Cost” (E&R), as defined by the Delta Project on Postsecondary Education Costs as “Instruction plus Student Services” (Delta Cost Project, 2009).

The allocation formulas are:

L = FTE(EdShAllLE + EdShDirLE × AdShAllLE)

LE = DLEI + DLES + DLEA × EdShDirLE

I = IE/PPI

IE = DIE + DIS + DIA × EdShDirIE

K = KE/GPDICTPI

KE = Stock × ROR(EdShAllTot + EdShDirTot × AdShAllTot)

________________

costs, and are adjusted for architectural significance and region. Of course there is scope for further exploration and refinements in this estimate which is best left to the judgment of college/university authorities. As per e-mail exchange between one panel member and Jim Kadamus (vice president of Sightlines), the company’s staff conducted some preliminary comparisons between current replacement values as calculated by IPEDS and Sightlines. The Sightlines estimates (for comparable space) are 70-100 percent greater than the value reported in IPEDS. The entity charged with implementing the productivity model will have to decide which estimate to use.

6The PPI is available at http://www.bls.gov/ppi/ and the GPDICTPI is available from http://www.bea.gov via the GDP and personal income interactive data link.

7The IPEDS Human Resources section provides a functional breakdown for direct teaching, research, and public service staff, but only an occupational breakdown for nonteaching staff. This scheme does not map into our model, and in any case the functional breakdown may be unstable due to inconsistencies in institutional classification schemes.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

Additional variables (beyond those on p. 67) are defined as:

  • DLEI = “Direct labor expenditures for instruction” as given by IPEDS; notations follows this format for student services (DLES) and administration.
  • EdShAllLE is “Education’s share of all labor expenditures”; AdShAllLE is “Administration and support services’ share of all labor expenditures”; notation follows this format for intermediate expenditures and capital. Finally, EdShDirLE is “Education’s share of direct labor expenditures,” and similarly for all the other shares. The difference between EdShAllLE and EdShDirLE is that the former’s denominator includes labor expenditures for administration and support services whereas the latter’s does not.
  • IE is expenditures on intermediate inputs; DIE is direct nonlabor expenditures on instruction; DIS is direct nonlabor expenditures on student services; DIA is direct nonlabor expenditures on administration; EdShDirIE is education share of direct nonlabor expenditures.
  • Stock is capital stock as shown on institutional balance sheets, ROR is the national rate of return on capital, PPI is the producer price index, and CPDICIPI is the price index for gross private domestic investment (both price indices are suitably normalized).8

Faculty time that is separately budgeted for institutional service is included in administration and support services, and unbudgeted faculty service time (e.g., departmental administration) is included in instruction.

4.2.5. Illustrative Productivity Calculations

Table 4.2 shows the productivity calculation for the institution referred to above. The calculation can be broken down into four steps:

  1. Allocate the quantity and expenditure data to the Education function: Apply the formulas above. For example, adjusted credit hours in the three periods equal 638,435, 643,476, and 677,000.
  2. Calculate the change in the quantity data from period to period: The change for adjusted credit hours equals the current value divided by the

________________

8See Hodge et al. (2011:25, Table 2). The rate of return to the net stock of produced assets for other nonfinancial industries is used as a proxy for the rate of return to higher education land, buildings and equipment. Other industries includes agriculture, forestry, fishing and hunting; transportation and warehousing; information; rental and leasing services and lessors of intangible assets; professional, scientific, and technical services; administrative and waste management services; educational services; health care and social assistance; arts, entertainment, and recreation; accommodation and food services; and other services, except government.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

TABLE 4.2 Base Model Productivity Calculations

  Period 1 Period 2 Period 3
Step 1: Allocations to Education Outputs      

Adjusted credit hours (ACHs)

638,435 643,476 677,000
Input Quantities

Labor FTEs (L)

3,926 4,296 4,705

Intermediate expenditures

$324,680 $486,147 $643,599

Rental value of capital (K)

$261,834 $267,507 $348,033
Input Expenditures

Wages and fringe benefits

$756,399 $867,311 $1,036,594

Intermediate expenditures

$324,680 $550,921 $777,193

Retail value of capital (K)

$261,834 $301,954 $400,791
Total Cost $1,342,913 $1,720,186 $2,214,578
 
Step 2: Quantity Changes Output Change Period 1 Period 1 → 2 Period 2 → 3

Adjusted credit hours

1.000 1.008 1.052
Input Change

Labor FTEs

1.000 1.094 1.095

Real intermediate expenditures

1.000 1.497 1.324

Real capital stock

1.000 1.022 1.301
 
Step 3: Input Index

Weights (average)

Wages and fringe benefits

  53.4% 48.6%

Normal intermediate expenditures

  28.1% 33.6%

Real capital stock

  18.5% 17.8%

Weighted geometric average

  1.180 1.204
 
Step 4: Multi-Factor Productivity

Productivity index

  0.854 0.874

Productivity change

    2.3%

NOTE: FTE = full-time equivalent.

previous one, with the first value being initialized at one. The ratios for Periods 2 and 3 are 1.008 and 1.052, for example, which indicate growth rates of 0.8 percent and 5.2 percent.

  1. Calculate the input index: For inputs, the composite index (“weighted geometric average”) is equal to the geometric average of the indices for the individual variables using the arithmetic average of the successive periods’ nominal expenditure shares as weights (no averaging is needed for outputs because there is only one output measure).9 This calculation

________________

9The index for the more complicated models presented later is based on the geometric average of output changes, the same as for inputs.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

BOX 4.1
Productivity and Quality

The productivity measure here does not take account of quality changes. Instead, it depends on the market to police quality erosion. Normally such policing is done through the price mechanism, although sometimes products such as computers whose quality is increasing over time do become cheaper. As argued earlier in this report, higher education prices generally are not set in competitive markets. Hence the conclusion, “Productivity has increased by ‘x’ percent” must be taken as tentative until the constancy of quality has been verified, for example, through a separate quality assurance procedure. A master artist produces 10 paintings in a month; her student also produces 10 paintings in the same period, for example. We sense that the quality of the artists is different, and that this difference should be reflected in the final product. But we cannot tell the difference in productivity just by counting the hours and the paintings (nor can we quantify the quality difference just by looking at them)—though we may eventually get some evidence by tracking the price that the paintings sell for or whether they sell at all. We return to the question of output quality in Section 4.6.

comes from the third equation in the Technical Appendix. The results are 1.180 and 1.204, which indicate average input growths of 18.0 percent and 20.4 percent.

  1. Calculate the productivity index: This is expressed as the ratio of the index for the change in outputs to the index for the change in inputs—i.e., 1.008/1.180 = 0.854 for the first period and 1.052/1.204 = 0.874 for the second. The last line, “productivity change,” is the ratio of the productivity indexes for the two periods: (0.854/0.874 – 1), or 2.3 percent.

An alternative but equivalent calculation illuminates Step 4. Notice that the output index grew by 1.052/1.008 = 1.044 (4.4 percent) and the input index grew by 1.204/1.180 = 1.020 (2.0 percent) between the second and third periods. Dividing the output growth by the input growth yields 1.044/1.020 = 1.023, or 2.3 percent, the same as in the table. Put another way, the output index grew 2.3 percent faster than the input index—which represents productivity improvement (see Box 4.1).

4.3. INSTITUTIONAL SEGMENTATION AND DISAGGREGATIVE INDICES

Having established the basic productivity index, we now present refinements intended to make it more useful. This section describes how indices can

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

be calculated for different segments or subsectors of the postsecondary education universe that have heterogeneous modes of production. Then we build on this description to describe how the index can be calculated for individual institutions or for a subset of institutions (e.g., all institutions in a state’s public system). We emphasize again, however, that such disaggregated indices should not be used as accountability measures unless a robust quality assurance system is operating in parallel.

4.3.1. Institutional Segmentation

The reason for segmenting colleges and universities into groups with similar characteristics is to avoid largely meaningless comparisons between highly differentiated institutions. Research universities, for example, differ in their outputs and production methods from master’s and bachelor’s universities and community colleges. The base model’s procedure for allocating costs among teaching, research, and public service handles some of the heterogeneity, but by no means all of it. As discussed earlier, research universities include a substantial amount of departmental research under the rubric of instructional cost, an intermixing that our allocation methodology cannot tease apart. Therefore, a decline in the share of educational output accounted for by research universities would boost the productivity index as production is shifted from higher-cost to lower-cost institutions, even though productivity within each individual institution remains unchanged. This will represent a true increase in overall higher education productivity only if the educational outputs are substantially similar—an assumption that is suspect.

Productivity indices may be calculated for each of a number of institutional segments and then aggregated to the national level using an appropriate indexing methodology. Total Education and General expenditures for the segments might be used as weights, though this suggestion should be reviewed. The alternative, calculating a single national statistic to start with, would eliminate the possibility of comparing productivity trends across segments. Retaining the ability to compare results should indicate whether productivity changes result primarily from shifts in enrollment and completion among segments, or from intra-segment productivity changes. IPEDS data for individual institutions include potentially useful descriptors, making it possible to formulate and test alternative segmentation schemes.

A natural starting point to defining institutional groups is to follow the approach established for the Delta Cost Project, if for no other reason than a considerable amount of experience in working with them has been accumulated. The Delta Cost Project employs six institutional groups: public research, public master’s, public community colleges, private nonprofit research, private nonprofit master’s, and private nonprofit bachelor’s. To these six we would add for-profit

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

institutions, for a total of seven groups.10 It may be desirable to have more than one for-profit segment and to create separate sub-categories for public and private research universities that do and do not have medical schools, but this is an open questions.

The simplest approach is to base the productivity calculations on aggregate data for all or a sample of institutions in a given segment. Such aggregation is standard practice in sectoral productivity analysis and we see no reason not to use it in higher education. Given that IPEDS reports data for individual institutions, it is almost as easy to do the calculations separately by segment as it is to do them on national aggregates. With the segment indices in hand, it is straightforward to aggregate them to produce a sector-wide index.

Finally, the productivity index for each segment should be normalized to 1.0 in the base period, before the aggregation proceeds. This is a natural step within most any indexing methodology. Moreover, the normalization will emphasize that it is the productivity trends that are being measured and not comparisons of absolute productivity across segments.

4.3.2. State-Level and Single-Institution Indices

The panel’s charge states that we should consider productivity measures at “different levels of aggregation: including the institution, system, and sector levels.” Our proposed model is designed to operate at the sector or subsector (segment) level. Given the IPEDS dataset and the specifics of the calculations, however, there is nothing to prevent researchers or administrators from applying the formulas to individual campuses and, by extension, to any desired set of campuses—say, within a system or state. Indeed, the illustration in Tables 4.1 to 4.4 is based on a single institution.

Two methodological caveats must be noted. First, trend comparisons should be made only with institutions in the same segment as the one being studied. Second, state-level or similar indices should themselves be disaggregated by segment. It makes no more sense to combine the apples and oranges of different segments at the state level than it does at the national level. Aggregation to a single state-level index should use the same methodology as the one described for segment aggregation. However, a question arises regarding the weights to be used for aggregation: should they be the same as for the national aggregation, or

________________

10We indicate for-profit higher education as a separate category because its production methods often differ substantially from those in the nonprofit sector. The for-profit sector has been growing rapidly; additionally, recent concerns about the performance of these schools—including questions about their heavy revenue reliance on federal student loans and issues of quality—make them well worth consideration as part of any serious appraisal of higher education performance. We are not proposing that the productivity of for-profit higher education be measured differently, but rather that it be placed in its own segment for comparison purposes.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

should they reflect expenditure or similar shares at the state level? The question of which weights to use should be examined further through future research.

While we feel obligated by our charge to raise the possibility of single-institution and state-level indices, the panel remains uncomfortable with the prospect that this might invite use of the model for accountability purposes. No such invitation is intended! Single-institution results may exhibit considerable volatility due to short-term variations in key input and output variables and the likelihood of data errors. Such volatility will likely decline as the number of institutions in the set increases. More importantly, as we have already emphasized, is the need to deploy robust quality assurance procedures in any situation where high-stakes quantitative productivity measures are used. In the slightly longer term, these quality assurance procedures should be supplemented by improvement-oriented structural models of the kind discussed in Chapter 2.

4.4. DIFFERENTIATING LABOR CATEGORIES

The first enhancement to the model is to track key labor categories separately from total FTEs. Separate tracking of labor types is typically a feature of sectoral productivity studies, but it is less commonly used to distinguish full-time from part-time employees; clearly, this differentiation is likely to be important in higher education. There are four reasons for this view.

  1. One of the critical assumptions of the conventional productivity model is not viable in higher education. The typical productivity study assumes that, because labor is secured in competitive markets, relative compensation approximates relative marginal products. There is, in such a situation, no need to differentiate full-time from part-time employees. Unfortunately, tenure-track faculty labor may not be linked tightly to marginal product in education because such faculty often are valued for research and reputational reasons and/or protected by tenure, or else locked into institutions because of tenure.
  2. Another assumption is that the market effectively polices output quality, which is manifestly not the case for higher education. Colleges pursue strategies—larger classes or less costly instructors—that reduce costs per nominal output but could dilute quality when taken to extremes. As noted earlier in this report, for example, it may be attractive to employ less expensive, but also less qualified, personnel who are not well integrated into a department’s quality processes. The panel is concerned lest the measurement of productivity add to the already problematic incentives to emphasize quantity over quality in higher education.
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
  1. The distinction between teaching and nonteaching staff is blurring as information technology shifts the modalities of teaching and learning. In some institutions, for example, faculty time is leveraged by modern learning software, a change that may require entirely new kinds of labor inputs. Such technologically driven changes are not unique to higher education, but the pace of change seems unusually brisk at the present time.
  2. Productivity statistics are more likely to weigh heavily in policy debates on higher education than in policy debates on other industries. The U.S. public policy environment includes a significant oversight and accountability component requiring information about productivity. Therefore, it is important that the statistics be as complete as possible on the important issues, including those associated with labor substitution (e.g., between tenure-track and nontenure-track teachers). Of course, proper analysis brings with it the responsibility to monitor and assess the quality of education obtained by students as the mix of inputs change.

These considerations suggest the following three-way labor classification scheme based on IPEDS data.

  • Regular faculty FTEs: approximated from IPEDS data for “Number of full-time Instruction/Research/Public Service staff with faculty status.”
  • Part-time teaching FTEs who are hired on a course-by-course basis: approximated by one-third of the “Number of staff by primary function/occupational activity” listed as “Part-time” and “Primarily instruction” (“PT/PI”). Ideally graduate assistants whose primary function is instruction should be included here. IPEDS contains data on graduate assistants, and they are reported separately under part-time staff.
  • All other FTEs: i.e., the base-model value minus the sum of the above.

The Human Resources/Employees by Assigned Position section of the IPEDS survey questionnaire requires institutions to report number of full-time and part-time staff involved in instruction/research/public service. The values reported for staff under this item will vary from one institution to another. Two-year colleges are more likely to report their entire faculty under “primarily instruction.” Some four-year institutions may change the way they report; for example, all kinds of faculty (irrespective of how much research they are doing) may be grouped under instruction/research/public service, creating biases in period to period trend comparisons.

In Table 4.3, part-time staff are converted to FTEs, and other staff FTEs are obtained by subtracting from total FTEs as used in the base model. Total wages and salaries and fringe benefits for instruction, research, and public service also

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

TABLE 4.3 Additional IPEDS Data for the Differential Labor Market

  Period 1 Period 2 Period 3
Quantities

Number of FTEs—instruction, research, and public service staff with faculty status

1,564 1,601 1,1601

Number of PT/PI staff

2 2 2

Number of other staff

4,928 6,704 5,974
 
Expenditures

Expenditures on wages and salaries for instruction, research, and public service

$529,572 $701,791 $874,051

Expenditures on fringe benefits for instruction, research, and public service

193,203 361,064 193,373

Expenditures on wages and salaries for other functions

380,610 410,296 482,139

Expenditures on fringe benefits for other functions

147,449 203,686 119,510

Average salary for FT instructional, staff

108,200 114,464 122,508

NOTE: FT = full-time, FTE = full-time equivalent, PT/PI = part-time/primarily instruction.

come directly from IPEDS, with those for other functions obtained by subtraction. Average salary for full-time instructional staff comes from the corresponding table in IPEDS. The very small PT/PI share may be an artifact of the particular data used in the example, but because the number is growing across the sector we believe this variable remains worthy of consideration.

Only one required data element is unavailable in IPEDS: the ratio of PT/PI salaries per FTE to those of regular faculty. It is possible (though perhaps not cost-effective) that IPEDS could be expanded to get this information; absent the change it may be adequate to assume the compensation ratio or determine it by special study.11 Finally, the expenditures for tenure-track and adjunct faculty expenditures are subtracted from total labor expenditures to get the figure for other labor.

Table 4.4 illustrates the calculations. The first step is to allocate the quantities and expenditures to the Education function. The figures for full-time tenure-track faculty and other staff are portioned using the education share variable computed in the base model. PT/PI staff needs no allocation because they are “primarily instruction” to start with. The FTE figures for Period 3 have been adjusted to demonstrate the effect of substituting PT/PI staff for full-time tenure-track faculty (discussed below): specifically, we subtracted 100 instructional FTEs from tenure-track faculty and added them to PT/PI.

________________

11One-third may be a reasonable approximation. We obtained this figure by assuming that (a) PT/PI staff get about $5,000 per course with no benefits, (b) average annual faculty salary plus benefits is $90,000, and (c) the average full-time faculty member teaches 6 courses a year. In this case the calculation is $5 000/($90 000/6) = 1/3

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

TABLE 4.4 Differentiated Labor Index Calculations

  Period 1 Period 2 Period 3
Allocations to Education

Quantities

Regular faculty FTEs

946 828 828

PT/PI staff FTEs

2 2 2

Other staff FTEs

2,978 3,466 3,875

Expenditures

Effective fringe benefit rate

36.5% 51.4% 22.1%

Regular faculty

$139,667 $143,552 $123,891

PT/PI staff

$28 $32 $30

Other staff

$616,704 $723,726 $912,673

Total expenditures

$756,399 $867,311 $1,036,594
Index Calculations

Quantities

FT regular faculty

1.000 0.876 1.000

PT/PI staff

1.000 1.000 1.000

Other staff

1.000 1.164 1.118
 

Quantity Change

  Period 1 → 2 Period 2 → 3

FT regular faculty

  0.876 1.142

PT/PI staff

  1.000 1.000

Other staff

  1.164 1.118

Weights

FT regular faculty

  17.5% 14.3%

PT/PI staff

  0.00% 0.00%

Other staff

  82.5% 85.7%

Labor Index

Geometric average

  1.107 0.985

Change in the average

    (0.110)

NOTE: FT = full-time, FTE = full-time equivalent, PT/PI = part-time/primarily instruction.

Calculating the expenditures requires the effective fringe benefits rate to be computed as a preliminary step. (It equals the ratio of fringe benefits to salaries and wages in Table 4.3.) The expenditure figures can be computed as follows:

Expenditures for regular faculty = (1+ Fringe benefits rate) × Average salary for FTE instructional staff × Regular faculty FTEs;

Expenditures for PT/PI staff = (1+ Fringe benefits rate/2) × PT/PI salary ratio × Average salary for FTE instructional staff × PT/PI FTEs; and

Expenditures for other staff = Total wages + fringe benefits – (Expenditures for regular faculty and PT/PI staff).

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

Total “Wages & fringe benefits” is taken from Table 4.1, and readers may notice that PT/PI staff receive only half the fringe benefits rate.

The final step, calculating the change indices and their weights, proceeds as in the base model. As shown in Table 4.4, resulting geometric averages show more variation than did the labor indices in Table 4.2. This produces larger multi-factor productivity indices, and also a larger change in the index, as shown at the bottom of the table. Which approach is more accurate may become more clear during implementation.

4.5. DIFFERENTIATING OUTPUTS

Another potentially key model enhancement is to control for the heterogeneity of educational outputs. Institution-level cost data indicate clearly that the resources required for producing an undergraduate degree vary across fields. Likewise, the resource requirements for producing bachelor degrees differ systematically from those for associate, graduate, and first professional degrees. Failure to control for these differences would risk the kinds of distortions described earlier. For example, a shift of outputs from the more expensive disciplines of science, technology, engineering, and mathematics (STEM) to non-STEM disciplines would falsely boost the productivity index.

IPEDS provides data on degrees by field and award level. A difficulty arises only in differentiating the credit hours associated with degree production. It is not unusual for institutions to track credit hours by the department or broad discipline in which a course is taught (as required to apply the so-called Delaware cost benchmarks for example), but these data cannot be mapped directly to degree production because students take many courses outside their matriculated area. Researchers have made the necessary correspondences by creating course-taking profiles for particular degrees, but these matrices are difficult to manage and maintain on an institution-wide basis. There is a better way, which we outline below—one that feeds directly into the productivity statistics and produces the course-taking profiles as by-products.12

For the long run, credit-hour data for productivity analysis should be collected in a way that follows the students, not only the departments that teach them. The necessary information exists in most institutions’ student registration files and the needed statistics can be extracted as follows:

  • Identify the students matriculated in a given degree program (“output category”) as defined by the IPEDS fields and degree levels. Undeclared students and students not matriculated for a degree would be placed in separate (“nonattributable”) output categories.

________________

12Simply applying a sheepskin effect to each degree category and summing the result before adding to aggregate credit hours is insufficient because shifts in degree production will induce shifts in credit-hour production, which will produce the kinds of distortion described in the text.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
  • For each output category, accumulate the credit hours earned by the students in that category, regardless of the department in which the course was offered or the year in which it was taken.
  • Allocate credits earned by matriculated but undeclared students in proportion to the credit-hour fractions of declared students for the given degree. Retain nonmatriculated students in their own separate category, one that has no sheepskin effect but in other respects is treated the same as other categories.

The question arises as to whether these data need to be collected by each institution, or whether generalized weights based on special studies (e.g., using the Postsecondary Educational Transcripts Study, PETS) could be used. The use of generalized weights is not inconceivable, but we worry that the heterogeneity of programs from school to school means that much information would be lost. Further, institutions may find the data on credit hours by degree program useful for internal purposes as well as for reporting—for example, in studying student profiles of course-taking behavior and benchmarking costs per degree program. It seems likely that, once in hand, these data will open significant new opportunities for institutional research.

It may be some time before data on student-based credit hour accumulations can be obtained, but there is a simple interim procedure that can be computed from the available IPEDS data. It is to allocate total enrollments to fields and levels based on fractions of completion. While ignoring differences in the course-taking profiles, the procedure does allow at least some differentiation among output categories.

Aggregation to a single output index is best accomplished by taking a geometric average of the output category indices using their net student revenue shares as weights. A geometric average with weights equal to revenue shares reflects the BLS methodology described earlier. Use of net as opposed to gross shares appears reasonable because revenue based on net prices (which is what the institution can spend on operations) is consistent with the underlying Törnqvist model as presented by Caves, Christensen, and Diewert (1982).

We recognize that getting these data may be problematic for institutions, in which case an alternative based on reported tuition and financial aid rates might well suffice. We also should note that it is not necessary to allocate any inputs across output categories. Such a requirement would disrupt data collection. Like most sectoral productivity indices, ours simply divides the aggregate output index for a segment by its aggregate input index.

4.6. VARIATIONS IN OUTPUT QUALITY

The quality of education is the elephant in the room in all discussions about instructional productivity, and the issue has been raised repeatedly in this report.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

The panel would have liked nothing better than to propose an explicit quality adjustment factor for weighted credit hours as part of our conceptual framework. In our model, an effect will be captured to the extent that higher quality inputs lead to higher graduation rates (and, in turn, a larger degree bonus in the numerator), but this effect is indirect. For example, if small classes or better teaching (inputs of different quality) lead to higher graduation rates, this will appear in the output numerator as a greater sheepskin effect. Similarly, high student and teacher quality at selective private institutions may offset high input costs by creating an environment conducive to high throughput rates.

This modest step notwithstanding, full (or even adequate) integration of quality adjustment into a productivity measure will not be possible any time soon. Significant progress on quality assessment has been made, but there is a long way to go before a generally accepted cardinal measure—one that can be used reliably to adjust weighted credit hours—can be agreed upon. It is possible, and perhaps even likely, that critics will call for a moratorium on all efforts to measure instructional productivity until a valid and reliable output quality index can be developed. We believe this would be unwise, for two reasons.

First, the kind of productivity measures proposed in this report is intended to deal primarily with changes over time rather than comparisons across institutions. It is true that an increasing focus on the quantitative aspects of productivity might trigger a “race for the bottom” in educational quality as competing institutions make increasingly larger concessions, seeking to boost the numerator and cut the denominator of the productivity fraction. Pressures on enrollments relative to budgets make this a danger whether quantitative productivity is properly measured or not, but there is no doubt that an increasing emphasis on the quantitative elements of productivity could exacerbate the problem. Normalizing the productivity index for each segment to 1.0, as is done in Section 4.3.1, will help alleviate this danger—though of course it is always possible to manipulate data to achieve desired results. Attention to the limitations of the metric for measuring at low levels of aggregation is also important.

Second, failure to agree on an economically valid and technically robust quantitative productivity measure will only increase proliferation of the weaker measures described in Chapter 2. These are even more susceptible to missing differences in quality than the method proposed here. Furthermore, failure to implement a good measure would indefinitely defer the benefits achievable from a better understanding of quantitative productivity even in the absence of quality adjustment. These considerations suggest a strategy of simply assuming that, absent evidence to the contrary, the quality of outputs is not declining significantly over time. The panel believes, albeit reluctantly given our desire for evidence, that the general approach proposed has much to recommend it.

To broaden the applicability of the measure developed here, additional steps should be taken. The essential idea is that effective and transparent quality assurance systems should be maintained to insure that output quality does not race for

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

the bottom (Massy, 2010). These could be based on extant accreditation systems, on the methods of academic audit being used effectively in Tennessee and certain overseas venues (Massy, Graham, and Short, 2007), or on the other quality-reviewing initiatives now being conducted at the state level. From a modeler’s perspective, the approach converts a quantity that would be included in the objective function if a cardinal measure were available to a “yes-no” constraint that needs only binary measurement. The binary constraint amounts to what might be called a “watchdog evaluation”: remaining silent if all is well but sounding the alarm if it is not. The watchdog evaluation would be at root subjective, but based upon evidence; we return to this idea in Section 5.3.2.

The approach is by no means perfect. However, it allows progress to be made in measuring the quantitative aspects of productivity while containing the risk of triggering institutional competition that results in lowering educational quality. Progress on the development of quantitative productivity measures may boost the priority for developing a serviceable quality adjustment index.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

TECHNICAL APPENDIX

The Törnqvist Productivity Index

This Appendix briefly describes the theoretical basis and calculations for the Törnqvist productivity index used in Chapter 4. The argument follows Caves, Christensen, and Diewert (1982:1393). As noted in the text, the Törnqvist index is used by the Bureau of Labor Statistics in calculating multi-factor productivity change (cf. Bureau of Labor Statistics, 2007). The appendix text is adapted from Section 1 of Massy (2012). Where applicable, the equations are keyed to Steps 2-4 of Table 4.2.

Input Indices, Distance Functions, and Productivity

We define “Firm k” and “Firm l” as two enterprises whose productivity is to be compared. Standard usage takes the two to be the same organization at different time periods, but this is not a requirement of the Törnqvist theory. For example, the two could be separate enterprises operating in the same or different periods.

At their Eq. 6, Caves and colleagues (1982) define the Malmquist input index for Firm k with respect to the inputs of Firm l as:

images

where yk is an m-element vector of outputs and xk is an n-element vector of inputs for firm k. The numerator of the right-hand side, called “Firm k’s input distance function with respect to the inputs of Firm l,” is defined at Eq. 7 as:

images

where Fk(yk,xk) = 0 is k’s production function, k is the vector {y2k, y3k,… ynk}, and y1k = Fk (, xk). Maximizing δ determines the “minimum Firm l input mix required to produce Firm k’s output using k’s production function”—i.e., it “deflates” kl onto k’s production function. From this it can be seen that Dk (yk, xk), the denominator in (1), always must equal one because any firm can produce its own output using its own inputs. This means Qk (xl, xk) = Dk (yk, xl) ≥ 1, which implies that “the input vector of firm l, xl, is ‘bigger’ than the input vector of firm k (i.e., k, xk) from the perspective of Firm k’s technology” (Caves, Christensen, and Diewert, 1982:1396). A similar argument applies to Ql (xl,xk).

Suppose now that Firms k and l are in fact the same entity, observed at times t1 and t2 respectively, and Qk (xl,xk) ≥ 1. This means the technology used at time

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

would have required more inputs to produce the t2 outputs than does the technology at t2 (this conclusion does not require the outputs at t1 and t2 to be the same). In other words, there has been a productivity improvement.

A Practical Method of Computation

This is a powerful result, but the calculation cannot be done without detailed knowledge of the production function and its parameters—knowledge that is rarely if ever available. Caves and his co-authors surmount this difficulty by invoking classic profit maximization—an assumption that is dubious when applied to higher education institutions (see below)—together with a modest technical simplification. In their words:

By making use of a specialized functional form and the assumption of cost-minimizing behavior, it is possible to compute a geometric average of the two Malmquist indices and Qk(xl,xk), using only observed information on input processes and quantities. We demonstrate this fact for the case in which each firm has a translog distance function, but the properties of the two translog functions are allowed to differ substantially. In this case the geometric average of the two Malmquist indices turns out to be a Törnqvist index. (Caves, Christensen, and Diewert, 1982:1397).

The technical simplification is that the production functions’ cross-product parameters for inputs and outputs be equal within and across firms. The authors point out that these restrictions are not onerous because “The translog distance function is capable of providing a second-order approximation to an arbitrary distance function. Thus the technologies in the two firms can be virtually arbitrary (to the second-order) except for the restrictions” [Caves, Christensen, and Diewert, 1982:1398]. Nonetheless, research will be needed to assess whether or not these restrictions are reasonable in the higher education context.

The resulting theorem, Caves Equation (15) reproduced below, shows how one can compute the geometric average of the two input indices using only observable data. This is all that’s needed to estimate productivity change.

images

The new symbols wk and wl are vectors of input prices and wkxk is total expenditure, the dot-product of price and quantity. This means the fractions represent shares of input expenditure. Hence the right-hand side defines the log of a Törnqvist index, denoted by: namely, the geometric average of physical inputs using expenditure shares as weights.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

The quantity Q(wl,wk,xl,xk) is shown for the transitions from Period 1 to Period 2 and Period 2 to Period 3, in the second-to-last line of Table 4.2 (the first line of step 4). Step 2 of Table 4.2 shows the inputs for the quantity-change (i.e., the last) term in the right-hand side of (3): i.e., in xlj. Step 3 shows the weights and the resulting geometric averages of the quantity changes. Notice that the weights are themselves an average of the weights in the two periods being considered.

The Overall Productivity Measure

Section 4 of Caves, Christensen, and Diewert (1982:1401), points out that there are “two natural approaches” to the measurement of productivity changes: differences in maximum output conditional on a given level of inputs (“output-based” indices) and those based on minimum input requirements conditional on a given level of output (“input-based” indices). Furthermore, the two approaches “differ from each other by a factor that reflects the returns to scale of the production structure.” Without going into the details, it is intuitively reasonable that a geometric average of the output-based and input-based indices represents a good overall measure of productivity.13 Therefore, the desired productivity index is the ratio of the output-based and input-based Törnqvist indices. Changes in productivity are obtained by taking the ratios of the indices in successive periods.

Step 4 of Table 4.2 performs these final calculations. The first line is the productivity index itself: the ratio of the output to the input indices. The second line shows productivity change: the ratio of the indices in the two successive periods, minus one.

Applying the Index to Nonprofit Enterprises

The proof of optimality for the Törnqvist index depends on the assumption of profit maximization. As noted above, this assumption is dubious when applied to traditional universities (it applies perfectly well to for-profit universities, however). This leads to two questions that need to be addressed by further research: (1) Will application of the index to nonprofits produce misleading results? (2) What modifications to the Törnqvist (or perhaps an entirely different approach) will likely be better than the traditional index?

Regarding the first question, we note that a lack of optimality is not equivalent to a lack of efficacy. Many algorithms and measures are used, in economics

________________

13Adjustments for decreasing and increasing returns to scale are described in later sections of the Caves paper. They are interesting but, we believe, not of particular concern to productivity measurement in colleges and universities. While scale economies in higher education are intuitively plausible, it appears that significant size increases are likely to trigger institutional responses (e.g., scope or support service increases) that tend to drive up costs (cf., Brinkman, 1990:120). Our model does not adjust for scale effects.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

and elsewhere, whose optimality cannot be proved or that have even been shown to be suboptimal. The question is an open one which calls for additional research.

Regarding the second question, Massy (2012) has proposed a modification to Equation (3) to achieve optimality in the nonprofit case. Still, additional research will be needed to determine how the required new parameters can be estimated and to identify the conditions under which the new model produces results that differ materially from the traditional one. The question of material differences will, in turn, shed light on the answer to question (1).

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

This page intentionally left blank.

Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 61
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 62
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 63
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 64
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 65
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 66
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 67
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 68
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 69
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 70
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 71
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 72
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 73
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 74
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 75
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 76
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 77
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 78
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 79
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 80
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 81
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 82
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 83
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 84
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 85
Suggested Citation:"4 Advancing the Conceptual Framework." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page 86
Next: 5 Recommendations for Creating and Extending the Measurement Framework »
Improving Measurement of Productivity in Higher Education Get This Book
×
Buy Paperback | $47.00 Buy Ebook | $37.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Higher education is a linchpin of the American economy and society: teaching and research at colleges and universities contribute significantly to the nation's economic activity, both directly and through their impact on future growth; federal and state governments support teaching and research with billions of taxpayers' dollars; and individuals, communities, and the nation gain from the learning and innovation that occur in higher education.

In the current environment of increasing tuition and shrinking public funds, a sense of urgency has emerged to better track the performance of colleges and universities in the hope that their costs can be contained without compromising quality or accessibility. Improving Measurement of Productivity in Higher Education presents an analytically well-defined concept of productivity in higher education and recommends empirically valid and operationally practical guidelines for measuring it. In addition to its obvious policy and research value, improved measures of productivity may generate insights that potentially lead to enhanced departmental, institutional, or system educational processes.

Improving Measurement of Productivity in Higher Education constructs valid productivity measures to supplement the body of information used to guide resource allocation decisions at the system, state, and national levels and to assist policymakers who must assess investments in higher education against other compelling demands on scarce resources. By portraying the productive process in detail, this report will allow stakeholders to better understand the complexities of--and potential approaches to--measuring institution, system and national-level performance in higher education.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!