Using the American Community Survey for the National Science Foundation's Science and Engineering Workforce Statistics Programs (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Using the American Community Survey for the National Science Foundation’s Science and Engineering Workforce Statistics Programs Panel on Assessing the Benefits of the American Community Survey for the NSF Division of Science Resources Statistics Committee on National Statistics Division of Behavioral and Social Sciences and Education 

THE NATIONAL ACADEMIES PRESS  500 Fifth Street, N.W.  Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Gov- erning Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engi- neering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by National Science Foundation Award Number SRS- 0632055 to the National Academy of Sciences. Support of the work of the Commit- tee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (award number SBR-0453930). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organiza- tions or agencies that provided support for the project. International Standard Book Number-13:  978-0-309-12153-8 International Standard Book Number-10:  0-309-12153-1 Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu. Printed in the United States of America Copyright 2008 by the National Academy of Sciences. All rights reserved. Suggested citation: National Research Council. (2008). Using the American Com- munity Survey for the National Science Foundation’s Science and Engineering Work- force Statistics Programs. Panel on Assessing the Benefits of the American Com- munity Survey for the NSF Division of Science Resources Statistics, Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. 

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal govern- ment on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its mem- bers, sharing with the National Academy of Sciences the responsibility for advis- ing the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr.  Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in pro- viding services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org 



PANEL ON ASSESSING THE BENEFITS OF THE AMERICAN COMMUNITY SURVEY FOR THE NSF DIVISION OF SCIENCE RESOURCES STATISTICS HAL STERN (Chair), Department of Statistics, University of California, Irvine DAN A. BLACK, Harris School, University of Chicago CHESTER (CHET) BOWIE, National Opinion Research Center, Washington, DC BRENDA G. COX, Battelle Memorial Institute, Arlington, VA RANDALL J. OLSEN, Center for Human Resources Research, Ohio State University ROBERT SANTOS, Urban Institute, Washington, DC LOWELL TAYLOR, H. John Heinz III School of Public Policy and Management, Carnegie Mellon University CATHERINE WEINBERGER, Department of Economics, University of California, Santa Barbara THOMAS J. PLEWES, Study Director CARYN E. KUEBLER, Associate Program Officer MICHAEL J. SIRI, Senior Program Assistant  

COMMITTEE ON NATIONAL STATISTICS 2007-2008 William F. Eddy (Chair), Department of Statistics, Carnegie Mellon University Katharine Abraham, Department of Economics and Joint Program in Survey Methodology, University of Maryland William DuMouchel, Lincoln Technologies, Inc., Waltham, MA John Haltiwanger, Department of Economics, University of Maryland V. Joseph Hotz, Department of Economics, Duke University, Durham, NC Karen Kafadar, Department of Statistics, Indiana University, Bloomington Douglas Massey, Department of Sociology, Princeton University Sally Morton, Statistics and Epidemiology, RTI International, Research Triangle Park, NC Vijay Nair, Department of Statistics and Department of Industrial and Operations Engineering, University of Michigan Joseph Newhouse, Division of Health Policy Research and Education, Harvard University Samuel H. Preston, Population Studies Center, University of Pennsylvania Kenneth Prewitt, School of International and Public Affairs, Columbia University Louise Ryan, Department of Biostatistics, Harvard University Roger Tourangeau, Survey Research Center, University of Michigan, and Joint Program in Survey Methodology, University of Maryland Alan Zaslavsky, Department of Health Care Policy, Harvard Medical School Constance F. Citro, Director vi 

Acknowledgments T his report results from contributions and expertise provided by many individuals. First, I acknowledge and thank my fellow panel members, who offered their time and collective expertise in plan- ning the October 2007 workshop, served as moderators and active par- ticipants during the workshop, and were full partners in writing this report. Staff of the National Science Foundation (NSF) and the Census Bureau were incredibly helpful in assisting the panel’s work and providing the information necessary for our making informed and realistic recommen- dations. The panel is grateful for their helpful presentations at the panel’s two open meetings and their timely responses to the panel’s numerous questions about their programs. In particular, we thank Nirmala Kannankutty, who served as liaison between the panel staff and NSF and contributed very useful presenta- tions on the work of the Division of Science Resources Statistics at NSF. Mary Frase and Stephen Cohen at NSF contributed to the discussions at the panel’s meetings and provided helpful presentations on the NSF man- dates and sample design options. The panel is also indebted to Ron Fecso, who prepared the original analysis of survey design options when he was on the staff of the NSF and continued to assist the panel with advice even after he had moved to the Government Accountability Office. Overall, the panel’s work and the assistance we received from the NSF staff would not have been possible without the invaluable participation of Lynda Carlson, director of the Division. vii 

viii ACKNOWLEDGMENTS Susan Schechter, Jennifer Tancreto, and Cheryl Landman from the Census Bureau expertly answered the panel’s questions about the uses of the American Community Survey (ACS) and particular concerns sur- rounding NSF’s and other users’ needs for information on graduates’ fields of degree. Ms. Tancreto’s workshop presentation on the testing for the field-of-degree question and the timeline for testing offered the panel an essential framework for its work on the topic. Several users of the Scientists and Engineers Statistical Data System program, including three panel members, offered insight on the varied options for research with these data at the workshop. The panel is grateful for the outside perspective on the important uses of the National Survey of College Graduates data provided by Donna Ginther of the University of Kansas and Sharon Levin of the University of Missouri at St. Louis. In testimony before the panel, Graham Kalton, Westat, who had chaired the National Research Council (NRC) Panel on the Functional- ity and Usability of Data from the American Community Survey, offered a fresh perspective on the design options under discussion and helped guide the panel to consider a broader range of alternatives. The panel acknowledges the excellent work of the staff of the Com- mittee on National Statistics (CNSTAT) and the NRC for support in devel- oping and organizing the workshop and this report. We are especially grateful for the excellent leadership of Tom Plewes, the panel’s study director. Tom used his experience and wise counsel to facilitate com- munication among panel members and between the panel and the NSF staff and to keep the project on schedule. His attention to the needs of the panel was invaluable in the process of producing this report. Tom was ably assisted by Caryn Kuebler of the CNSTAT staff, and Michael Siri of the CNSTAT staff, who provided administrative support. The panel also benefited from the work of Eugenia Grohman of the Division of Behav- ioral and Social Sciences and Education for editing the report. We are especially indebted to Constance F. Citro, CNSTAT director, for her general expertise and for sharing her extensive knowledge of the ACS. Her participation in the panel’s meetings and deliberations benefited the report in numerous ways. This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the Report Review Committee of the NRC. The purpose of this independent review is to provide candid and critical com- ments that assist the institution in making its report as sound as possible, and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. 

ACKNOWLEDGMENTS ix The panel thanks the following individuals for their review of this report: Barbara A. Bailar, consultant, Washington, DC; Richard Freeman, National Bureau of Economic Research, Cambridge, MA; Donna K. Ginther, Economics Department, University of Kansas; James M. ­Lepkowski, Insti- tute for Social Research, University of Michigan; Sharon Lohr, Depart- ment of Mathematics and Statistics, Arizona State University; Sally C. Morton, Research Triangle Institute, Research Triangle, NC; and Paula E. Stephan, Department of Economics, Andrew Young School of Policy Studies, ­Georgia State University.  Although the reviewers listed above have provided many construc- tive comments and suggestions, they were not asked to endorse the con- clusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Richard Kulka, Abt Associates, Durham, NC. Appointed by the NRC, he was responsible for making certain that the independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of the report rests entirely with the authoring committee and the institution. Hal Stern, Chair Panel on Assessing the Benefits of the American Community Survey for the NSF Division of Science Resources Statistics 



Contents Executive Summary 1 1 Introduction 9 2 The Scientists and Engineers Statistical Data System 13 3 The National Survey of College Graduates 26 4 The ACS and the SESTAT Program 33 5 Adding a Field-of-Degree Question to the ACS 41 6 Using ACS for the NSCG Sample Frame 50 7 The ACS and SESTAT in the Future 68 References 79 Appendixes A Workshop Summary and Agenda 81 B Biographical Sketches of Panel Members and Staff 86 xi