National Academies Press: OpenBook
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

Improving Measurement of
Productivity in Higher Education

Panel on Measuring Higher Education Productivity:
Conceptual Framework and Data Needs

Teresa A. Sullivan, Christopher Mackie, William F. Massy, and
Esha Sinha, Editors

Committee on National Statistics

Board on Testing and Assessment

Division of Behavioral and Social Sciences and Education

NATIONAL RESEARCH COUNCIL
                         OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.

www.nap.edu

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

THE NATIONAL ACADEMIES PRESS • 500 Fifth Street, NW • Washington, DC 20001

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

This study was supported by grant number 5793 between the National Academy of Sciences and Lumina Foundation. Support for the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (award number SES-1024012). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organizations or agencies that provided support for this project.

International Standard Book Number-13: 978-0-309-25774-9
International Standard Book Number-10: 0-309-25774-3

Library of Congress Cataloging-in-Publication data are available from the Library of Congress

Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313, http://www.nap.edu.

Copyright 2012 by the National Academy of Sciences. All rights reserved.

Printed in the United States of America

Suggested citation: National Research Council. (2012). Improving Measurement of Productivity in Higher Education. Panel on Measuring Higher Education Productivity: Conceptual Framework and Data Needs. Teresa A. Sullivan, Christopher Mackie, William F. Massy, and Esha Sinha, Editors. Committee on National Statistics and Board on Testing and Assessment, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

THE NATIONAL ACADEMIES

Advisers to the Nation on Science, Engineering, and Medicine

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.

www.national-academies.org

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

PANEL ON MEASURING HIGHER EDUCATION PRODUCTIVITY: CONCEPTUAL FRAMEWORK AND DATA NEEDS

TERESA A. SULLIVAN (Chair), Office of the President, University of Virginia

THOMAS R. BAILEY, Institute on Education and the Economy and Community College Research Center, Teachers College, Columbia University

BARRY P. BOSWORTH, Economic Studies Program, The Brookings Institution, Washington, DC

DAVID W. BRENEMAN, Curry School of Education, University of Virginia

RONALD G. EHRENBERG, Cornell Higher Education Research Institute, Cornell University

PETER T. EWELL, National Center for Higher Education Management Systems, Boulder, CO

IRWIN FELLER, Department of Economics (emeritus), Pennsylvania State University

BARBARA FRAUMENI, Muskie School of Public Service, University of Southern Maine

JULIET V. GARCIA, Office of the President, University of Texas at Brownsville and Texas Southmost College

MICHAEL HOUT, Department of Sociology, University of California, Berkeley

NATE JOHNSON, HCM Strategists, Washington, DC

GEORGE D. KUH, Center for Postsecondary Research (emeritus), Indiana University

WILLIAM F. MASSY, Independent Consultant, Florence, MA

CAROL A. TWIGG, National Center for Academic Transformation, Saratoga Springs, NY

DAVID J. ZIMMERMAN, Department of Economics, Williams College

CHRISTOPHER D. MACKIE, Study Director

STUART ELLIOTT, Senior Program Officer

ESHA SINHA, Associate Program Officer

MICHAEL SIRI, Program Associate

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

COMMITTEE ON NATIONAL STATISTICS 2011-2012

LAWRENCE BROWN (Chair), Department of Statistics, The Wharton School, University of Pennsylvania

JOHN M. ABOWD, School of Industrial and Labor Relations, Cornell University

ALICIA CARRIQUIRY, Department of Statistics, Iowa State University

WILLIAM DuMOUCHEL, Oracle Health Sciences, Waltham, MA

V. JOSEPH HOTZ, Department of Economics, Duke University

MICHAEL HOUT, Survey Research Center, University of California, Berkeley

KAREN KAFADAR, Department of Statistics, Indiana University

SALLIE KELLER, University of Waterloo, Ontario, Canada

LISA LYNCH, Heller School for Social Policy and Management, Brandeis University

SALLY MORTON, Graduate School of Public Health, University of Pittsburgh

JOSEPH NEWHOUSE, Division of Health Policy Research and Education, Harvard University

RUTH PETERSON, Criminal Justice Research Center, The Ohio State University

HAL STERN, Department of Statistics, University of California, Irvine

JOHN H. THOMPSON, NORC at the University of Chicago

ROGER TOURANGEAU, Statistical Group, Westat, Rockville, MD

ALAN ZASLAVSKY, Department of Health Care Policy, Harvard Medical School

CONSTANCE F. CITRO, Director

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

BOARD ON TESTING AND ASSESSMENT

EDWARD HAERTEL (Chair), Jacks Family Professor of Education and Associate Dean for Faculty Affairs, Stanford University

GARY CHAMBERLAIN, Louis Berkman Professor of Economics, Harvard University

MARK DYNARSKI, Researcher, Pemberton Research, LLC

DAVID J. FRANCIS, Hugh Roy and Lillie Cranz Cullen Distinguished Professor, and Director, Texas Institute for Measurement, Statistics and Evaluation, University of Houston

JOAN HERMAN, Director, National Center for Research on Evaluation, Standards, and Student Testing, University of California, Los Angeles

MICHAEL KANE, Messick Chair in Validity, Educational Testing Service

SHARON LEWIS, Director of Research, Council of Great City Schools

ROBERT MARE, Professor of Sociology, University of California, Los Angeles

DIANA C. PULLIN, Professor, Boston College

ANN MARIE RYAN, Professor of Psychology, Michigan State University

BRIAN STECHER, Senior Social Scientist, Education Program, RAND Corporation

JOHN ROBERT WARREN, Associate Professor of Sociology, University of Minnesota

MARK WILSON, Professor of Policy, Organization, Measurement, and Evaluation Cognition and Development, University of California, Berkeley

REBECCA ZWICK, Distinguished Presidential Appointee, Research and Development, Educational Testing Service

STUART ELLIOTT, Director

Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

Acknowledgments

The work of this panel has been immeasurably assisted by the insight and counsel of numerous colleagues. In particular, we wish to acknowledge the reviewers. This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the Report Review Committee of the National Research Council (NRC). The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Julian Betts, Department of Economics, University of California, San Diego; William G. Bowen, President’s Office, Andrew W. Mellon Foundation; Pat Callan, President’s Office, Higher Education Policy Institute, San Jose, California; Charles T. Clotfelter, Center for the Study of Philanthropy and Voluntarism, Duke University; Don E. Detmer, University of Virginia School of Medicine; David N. Figlio, Institute for Policy Research, Northwestern University; Brent R. Hickman, Economics Department, University of Chicago; Michael McPherson, President’s Office, The Spencer Foundation; B. Don Russell, Jr., Department of Electrical Engineering, Texas A&M University; and Burton A. Weisbrod, Institute for Policy Research, Northwestern University.

The review of this report was overseen by Greg Duncan, distinguished professor of education, University of California, Irvine, and Charles Manski, Board of Trustees professor in economics, Northwestern University. Appointed by the NRC’s Report Review Committee, they were responsible for making certain that

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered.

Although the reviewers have provided many constructive comments, and improved the content of the report a great deal, they were not asked to endorse the conclusions or recommendations; nor did they see the final draft of the report prior to its release. Responsibility for the final content of this report rests entirely with the authoring panel and the institution.

Many others generously gave of their time in offering oral presentations at meetings and answering questions from panel members and staff, thereby helping us to develop a clearer understanding of key issues relevant to the measurement of higher education productivity and related issues. The panel thanks Lumina Foundation; they provided financial support for the project and, even more importantly, helped shape the scope of the study. From Lumina, Jamie Merisotis, Kevin Corcoran, Suzanne Walsh (now with the Bill & Melinda Gates Foundation), and Charles (Chip) Hatcher provided insights and guidance in their roles as initiators of the project. Kristin Conklin (HCM Strategists, LLC) kept the panel informed about Lumina grantees’ programs to increase productivity and proposals for measuring the effectiveness of those efforts. The panel benefited from the open discussion of these initiatives.

During meetings and deliberations, the panel heard from a number of subject matter experts. Andrea Bonaccorsi, University of Pisa, Italy; Kevin Carey, New America Foundation; Hamish Coates, Australian Council for Education Research; Bo Hansson, OECD; Jorge Klor de Alva, University of Phoenix; and Donna Sundre, James Madison University informed the panel about efforts to measure higher education productivity, to design and implement accountability systems, and to improve input/output data at different levels of aggregation.

The panel could not have conducted its work without an excellent and well-managed staff. Connie Citro, director of the Committee on National Statistics, and Stuart Elliott, director of the Board on Testing and Assessment, provided expert guidance to the panel about the NRC study process. Program associate Michael Siri provided excellent administrative, editorial, and research support. Esha Sinha, program officer, provided valuable research and analytic assistance with her understanding of higher education data sources. Her knowledge proved especially helpful as we worked through some of the thornier measurement issues. The panel also benefited from the work of Kirsten Sampson-Snyder, Division of Behavioral and Social Sciences and Education, who was responsible for overseeing the review process. Amy Smith provided able editing of numerous drafts.

Christopher Mackie, the panel’s study director, organized our meetings and facilitated communication among panel members, including a lengthy process of chapter revisions. His work required synthesizing and evaluating many disparate points of input, seeking what common ground could be found, and guiding the panel through careful discussion of the points of disagreement. He helped to

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

develop the structure for the panel’s final report, and he shepherded the report through the final review process.

Most importantly, the members of the panel deserve thanks for their patience, creativity, and hard work. There is a reason that higher education productivity is not currently reported in the national accounts. Reaching agreement on conceptual and measurement issues was difficult work. This report reflects the collective expertise and commitment of the individual members of the panel, each of whom brought a unique perspective based upon a scholarly discipline, research experience, and a lifetime of practice. Members were generous with their time and effort, and they struggled to understand and appropriately acknowledge the critical views of others. Our meetings provided many opportunities for panel members to learn from one another.

Teresa A. Sullivan, Chair
Panel on Measuring Higher Education Productivity:
Conceptual Framework and Data Needs

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

3       WHY MEASUREMENT OF HIGHER EDUCATION PRODUCTIVITY IS DIFFICULT

3.1. Beyond the Degree Factory—Multiple Outputs and Joint Production

3.2. Heterogeneity of Inputs and Outputs

3.3. Nonmarket Variables and Externalities

3.4. Quality Change and Variation

3.4.1. Inputs

3.4.2. Outputs (and Outcomes)

3.5. Measurement at Different Levels of Aggregation

3.5.1. Course and Department Level

3.5.2. Campus Level

3.5.3. State or System Level

3.6. Conclusion

4       ADVANCING THE CONCEPTUAL FRAMEWORK

4.1. Chapter Overview

4.2. A Baseline Multi-Factor Productivity Model for Higher Education

4.2.1. Multi-Factor Productivity Indices

4.2.2. Outputs

4.2.3. Inputs

4.2.4. Allocations to Education

4.2.5. Illustrative Productivity Calculations

4.3. Institutional Segmentation and Disaggregative Indices

4.3.1. Institutional Segmentation

4.3.2. State-Level and Single-Institution Indices

4.4. Differentiating Labor Categories

4.5. Differentiating Outputs

4.6. Variations in Output Quality

Technical Appendix: The Törnqvist Productivity Index

5       RECOMMENDATIONS FOR CREATING AND EXTENDING THE MEASUREMENT FRAMEWORK

5.1. The Basic Productivity Measure

5.1.1. Instructional Outputs and Benefits

5.1.2. Instructional Inputs and Costs

5.2. Adjusting for Research Production

5.2.1. Project-Driven Departmental Research

5.2.2. Discretionary Departmental Research

5.3. Dealing with Heterogeneity and Quality Issues

5.3.1. Variation of Inputs

5.3.2. Quality Variation and Change of Outputs

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×

This page intentionally left blank.

Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R1
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R2
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R3
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R4
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R5
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R6
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R8
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R9
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R10
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R11
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R12
Page xiii Cite
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R13
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R14
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R15
Suggested Citation:"Front Matter." National Research Council. 2012. Improving Measurement of Productivity in Higher Education. Washington, DC: The National Academies Press. doi: 10.17226/13417.
×
Page R16
Next: Summary »
Improving Measurement of Productivity in Higher Education Get This Book
×
Buy Paperback | $47.00 Buy Ebook | $37.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Higher education is a linchpin of the American economy and society: teaching and research at colleges and universities contribute significantly to the nation's economic activity, both directly and through their impact on future growth; federal and state governments support teaching and research with billions of taxpayers' dollars; and individuals, communities, and the nation gain from the learning and innovation that occur in higher education.

In the current environment of increasing tuition and shrinking public funds, a sense of urgency has emerged to better track the performance of colleges and universities in the hope that their costs can be contained without compromising quality or accessibility. Improving Measurement of Productivity in Higher Education presents an analytically well-defined concept of productivity in higher education and recommends empirically valid and operationally practical guidelines for measuring it. In addition to its obvious policy and research value, improved measures of productivity may generate insights that potentially lead to enhanced departmental, institutional, or system educational processes.

Improving Measurement of Productivity in Higher Education constructs valid productivity measures to supplement the body of information used to guide resource allocation decisions at the system, state, and national levels and to assist policymakers who must assess investments in higher education against other compelling demands on scarce resources. By portraying the productive process in detail, this report will allow stakeholders to better understand the complexities of--and potential approaches to--measuring institution, system and national-level performance in higher education.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!