Understanding American Agriculture

Challenges for the Agricultural Resource Management Survey

Panel to Review USDA’s Agricultural Resource Management Survey

Committee on National Statistics

Division of Behavioral and Social Sciences and Education

NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.
www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Understanding American Agriculture Challenges for the Agricultural Resource Management Survey Panel to Review USDA’s Agricultural Resource Management Survey Committee on National Statistics Division of Behavioral and Social Sciences and Education NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES THE NATIONAL ACADEMIES PRESS Washington, D.C. www.nap.edu

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Contract No. 53-3K06-05-1600 between the National Academy of Sciences and the United States Department of Agriculture. Support of the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (award number SBR-0453930). Any opinion, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project. International Standard Book Number-13: 978-0-309-11092-1 International Standard Book Number-10: 0-309-11092-0 Additional copies of this report are available from the National Academies Press, 500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu. Printed in the United States of America. Copyright 2008 by the National Academy of Sciences. All rights reserved. Suggested citation: National Research Council. (2008). Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey. Panel to Review USDA’s Agricultural Resource Management Survey. Committee on National Statistics, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey THE NATIONAL ACADEMIES Advisers to the Nation on Science, Engineering, and Medicine The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council www.national-academies.org

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey This page intentionally left blank.

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey PANEL TO REVIEW USDA’S AGRICULTURAL RESOURCE MANAGEMENT SURVEY BRUCE GARDNER (Chair), Department of Agricultural and Resource Economics, University of Maryland WALTER J. ARMBRUSTER, Farm Foundation, Oak Brook, Illinois DAVID BINDER, Statistics Canada, Ottawa (Retired) RAY D. BOLLMAN, Agriculture Division, Statistics Canada, Ottawa CYNTHIA Z.F. CLARK, Methodology Directorate, Office for National Statistics, United Kingdom FREDERICK CONRAD, Institute for Social Research, University of Michigan ANI L. KATCHOVA, Department of Agricultural and Consumer Economics, University of Illinois at Urbana-Champaign ARTHUR KENNICKELL, Microeconomic Surveys Unit, Board of Governors, Federal Reserve System CATHERINE KLING, Department of Economics, Iowa State University JEAN OPSOMER, Center for Survey Methodology, Iowa State University BOBBY R. PHILLS, Small Fruits Program, Florida Agricultural and Mechanical University STANLEY PRESSER, Sociology Department and Joint Program in Survey Methodology, University of Maryland ROBERT D. TORTORA, Survey Research Center, The Gallup Organization, Washington, DC MICHAEL K. WOHLGENANT, Department of Agricultural and Resource Economics, North Carolina State University THOMAS J. PLEWES, Study Director CARYN KUEBLER, Associate Program Officer MICHAEL SIRI, Senior Program Assistant

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey COMMITTEE ON NATIONAL STATISTICS 2007 WILLIAM F. EDDY (Chair), Department of Statistics, Carnegie Mellon University KATHARINE ABRAHAM, Department of Economics, University of Maryland, and Joint Program in Survey Methodology ROBERT BELL, AT&T Research Laboratories, Florham Park, New Jersey WILLIAM DuMOUCHEL, Lincoln Technologies, Inc., Waltham, Massachusetts JOHN HALTIWANGER, Department of Economics, University of Maryland V. JOSEPH HOTZ, Department of Economics, University of California at Los Angeles KAREN KAFADAR, Department of Mathematical Sciences, University of Colorado at Denver and Health Sciences Center DOUGLAS MASSEY, Department of Sociology, Princeton University VIJAY NAIR, Department of Statistics and Department of Industrial and Operations Engineering, University of Michigan JOSEPH NEWHOUSE, Division of Health Policy Research and Education, Harvard University SAMUEL H. PRESTON, Population Studies Center, University of Pennsylvania KENNETH PREWITT, School of International and Public Affairs, Columbia University LOUISE RYAN, Department of Biostatistics, Harvard University NORA CATE SCHAEFFER, Department of Sociology, University of Wisconsin at Madison ALAN ZASLAVSKY, Department of Health Care Policy, Harvard Medical School CONSTANCE F. CITRO, Director

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Contents     Acknowledgments   xi     Acronyms and Abbreviations   xv     Executive Summary   1      Data Quality,   2      Conclusions,   2      Recommendations,   4      Investment in the Future,   5 1   Introduction   7      Survey Description,   8      Charge to the Panel,   10      Issues in Survey Operations,   11      The Panel’s Approach,   12      Guide to the Report,   15 2   Contemporary Issues in American Agriculture   16      Priority Uses of ARMS Data,   17      Data Relevance,   25      Three Key Areas,   29      Maintaining Relevance by Developing New Data Sources,   45

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey 3   Survey Management   49      Collaborative Management,   49      Managing the Changing Focus of ARMS,   55 4   Sample and Questionnaire Design   62      Survey Design Goals,   63      Sampling Frame,   65      Sample Design,   69      Questionnaire Design and Development,   74      Understanding Record-Keeping Practices,   79      Appropriateness of the Survey Questions,   81      Supplemental Data,   84 5   Data Collection   86      Collection Arrangements,   87      Quality Assurance,   88      Collection Methodology,   96      Data Capture, Editing, and Processing,   103      Metadata and Paradata,   104 6   Nonresponse, Imputation, and Estimation   107      Unit Nonresponse,   107      Item Nonresponse,   115      Imputation,   116      Estimation,   122 7   Methods for Analysis of Complex Surveys   127      Data Analysis Issues,   128      Evaluation of Current Approach,   131      Estimation Strategies for Complex Surveys,   132      Guide for Researchers,   146 8   Dissemination   150      New Data Sources and Products,   151      Microdata Access for Data Users,   154      Training for Data Users,   157      User Forums and Feedback,   159      Data Consistency,   159 9   Conclusions and Recommendations   162      Conclusions,   162      Recommendations,   164

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey     References   170     Appendixes     A   Panel Data-Gathering Activities   177 B   Biographical Sketches of Panel Members and Staff   189

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey This page intentionally left blank.

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Acknowledgments This report is the result of the work by many individuals, each of whom has contributed their expertise. This section acknowledges some of the central contributions to this extensive undertaking. First, I acknowledge and thank my fellow panel members, who gave of their time and expertise so generously, participating in seven meetings and countless other activities critical to completing this report. They shared the administrative load in their service as session moderators during the open meetings and in their participation in the lively discussions of the issues at each of those events. Their assistance in drafting this report resulted in a more comprehensive and defensible product and is greatly appreciated. Major activities of the panel were the four public sessions, three of which were organized and conducted as formal workshops. In developing and conducting these productive workshops, the panel enjoyed outstanding support from the staff of the National Agricultural Statistics Service (NASS) and the Economic Research Service (ERS). Robert Bass, director of the Census and Survey Division of NASS, and Mary Bohman, director of the Resource Economics Division, served as the primary agency liaisons and facilitated our work with admirable patience and perseverance. Jim MacDonald of ERS provided much advice, counsel, and facilitation support and was instrumental in focusing the attention of the panel on the difficult issues related to tests of inference of the Agricultural Resource Management Survey (ARMS) data. He spoke before the panel on two occasions and maintained contact with the panel staff as the study emerged. Dania Ferguson of NASS assembled documents requested by the panel,

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey some quite old and requiring considerable research, and prepared several responses to questions submitted by the panel. In all, staff of NASS and ERS were of great assistance to us in understanding the complex survey design and implementation that characterizes ARMS. In addition to participating in the panel’s open sessions, both NASS and ERS responded to many questions formally transmitted to the agencies. A summary of the panel’s data-gathering activities appears in Appendix A. The committee gratefully acknowledges the excellent work of the staff of the Committee on National Statistics (CNSTAT) and the National Research Council for support in developing and organizing the workshops and this report. The experience, wise counsel, and untiring efforts of Tom Plewes, study director for the panel, were indispensable in bringing this report to fruition. He was ably assisted by Caryn Kuebler of the CNSTAT staff, and Michael Siri provided administrative support. Michael Cohen assisted in clarifying and resolving issues related to analysis of complex survey data. William Greene provided sound advice on issues of inference in data analysis in service as an expert consultant to the panel. We especially appreciate the participation of Constance F. Citro, director of CNSTAT, in the workshops and the preparation of this report. Her sage advice benefited the report in numerous ways. This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that assist the institution in making its report as sound as possible, and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. The panel wishes to thank the following individuals for their review of this report: Jason Abrevaya, Department of Economics, University of Texas at Austin; Richard N. Boisvert, Department of Agricultural Economics, Cornell University; D. Wade Brorsen, Department of Agricultural Economics, Oklahoma State University; Rachel Harter, National Opinion Research Center, Chicago, IL; Barrett Kirwan, Agricultural and Resource Economics, University of Maryland; Virginia Lesser, Survey Research Center, Department of Statistics, Oregon State University; Frank Potter, Statistical Services, Mathematica Policy Research, Inc., Princeton, NJ; and Alan M. Zaslavsky, Department of Health Care Policy, Harvard Medical School. Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Keith F. Rust, Westat,

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Inc., Rockville, MD. Appointed by the NRC, he was responsible for making certain that the independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of the report rests entirely with the authoring committee and the institution. Bruce Gardner, Chair Panel to Review USDA’s Agricultural Resource Management Survey

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey This page intentionally left blank.

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Acronyms and Abbreviations AAEA American Agricultural Economics Association ACS American Community Survey AEM Association of Equipment Manufacturers APHIS Animal and Plant Health Inspection Service ARDIS Agricultural and Rural Development Information System AREI Agricultural Resources and Environmental Indicators ARMS Agricultural Resource Management Survey BEA Bureau of Economic Analysis BLS Bureau of Labor Statistics CAI Computer-Assisted Interviewing CAPI Computer-Assisted Personal Interviewing CAR Commodity Cost and Returns CASIC Computer-Assisted Survey Information Collection CBO Congressional Budget Office CEA Council of Economic Advisers CEAP Conservation Effects Assessment Project CPS Current Population Survey CRP Conservation Reserve Program CRR Costs and Returns Report CSP Conservation Security Program CSV Comma Separated Value CV Coefficient of Variation

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey DAG Delete-a-Group EDI Electronic Data Interchange EDR Electronic Data Reporting EPA Environmental Protection Agency EQIP Environmental Quality Incentive Program ERS Economic Research Service, USDA FAO Food and Agriculture Organization of the United Nations FAS Foreign Agricultural Service, USDA FO U.S. Department of Agriculture State Field Office FSA Farm Service Agency, USDA FTE Full-Time Equivalent FVS Farm Value Sales GDP Gross Domestic Product GEE Generalized Estimating Equation GHG Greenhouse Gas GIS Geographic Information System GPS Global Positioning System HEL Highly Erodible Land IDAS Interactive Data Analysis System IID Independent and Identically Distributed I-O Input-Output IRS Internal Revenue Service JPSM Joint Program in Survey Methodology MILC Milk Income Loss Contract NASDA National Association of State Departments of Agriculture NASS National Agricultural Statistics Service, USDA NIPA National Income and Product Accounts NOL Non-Overlap NRC National Research Council NRCS National Resources Conservation Service, USDA NRI National Resource Inventory OECD Organisation for Economic Co-operation and Development OEPNU Office of Energy Policy and New Uses OMB Office of Management and Budget

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey P-B Perry-Burt Procedure PDA Personal Digital Assistant PPCR Production Practices and Costs Report PPI Producer Price Index PPR Production Practices Report PRISM Project to Reengineer and Integrate Statistical Methods PSM Policy and Standards Memoranda RDCs Research Data Centers RMA Risk Management Agency, USDA SAS Statistical Analysis System Software SCF Survey of Consumer Finances SIP Sequential Interval Poisson TCDO Training and Career Development Office TMDL Total Maximum Daily Loads USDA U.S. Department of Agriculture

OCR for page R1
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey This page intentionally left blank.