Appendix A
Panel Data-Gathering Activities

The panel conducted the bulk of its business in open forum, eliciting input from staff of the Economic Research Service (ERS) and the National Agricultural Statistics Service (NASS) as well as key data users, policy makers, and additional technical experts. The panel’s public sessions included a workshop on statistical methodology, June 8-9, 2006; a session for data users at the annual meeting of the American Agricultural Economics Association, July 24, 2006; a workshop on concepts and measurement, September 28-29, 2006; a workshop on inference, December 7-8, 2006; and an open discussion of cost-of-production issues, January 18, 2007.

At the panel’s introductory meeting in February 2006, Katherine Smith, then associate director of ERS, and Carol House, associate director of NASS, set the stage for the work of the panel. They discussed the purpose of the study from their perspective, and the charge to the panel. Their frank and open exposition on the need for a review of the survey were instrumental in focusing our attention on key issues to be addressed in the study and in enhancing our understanding of the partnership between these agencies that directs and sustains the survey. In Carol House’s presentation, the panel was challenged to consider not only the question of whether the agencies were doing the right things in terms of content, but also whether they are doing things correctly in terms of approach and methodology. Katherine Smith emphasized the critical importance of the survey as the main source of information on farm economics, production, and structure.

These presentations were supported and elaborated by a thorough discussion of the state of the survey by Robert Bass and Mary Bohman, who



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Appendix A Panel Data-Gathering Activities The panel conducted the bulk of its business in open forum, eliciting input from staff of the Economic Research Service (ERS) and the National Agricultural Statistics Service (NASS) as well as key data users, policy makers, and additional technical experts. The panel’s public sessions included a workshop on statistical methodology, June 8-9, 2006; a session for data users at the annual meeting of the American Agricultural Economics Association, July 24, 2006; a workshop on concepts and measurement, September 28-29, 2006; a workshop on inference, December 7-8, 2006; and an open discussion of cost-of-production issues, January 18, 2007. At the panel’s introductory meeting in February 2006, Katherine Smith, then associate director of ERS, and Carol House, associate director of NASS, set the stage for the work of the panel. They discussed the purpose of the study from their perspective, and the charge to the panel. Their frank and open exposition on the need for a review of the survey were instrumental in focusing our attention on key issues to be addressed in the study and in enhancing our understanding of the partnership between these agencies that directs and sustains the survey. In Carol House’s presentation, the panel was challenged to consider not only the question of whether the agencies were doing the right things in terms of content, but also whether they are doing things correctly in terms of approach and methodology. Katherine Smith emphasized the critical importance of the survey as the main source of information on farm economics, production, and structure. These presentations were supported and elaborated by a thorough discussion of the state of the survey by Robert Bass and Mary Bohman, who

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey addressed issues arising from the complexity of the Agricultural Resource Management Survey (ARMS): the complicated survey form, the problem of missing observations and nonresponse, the lack of comparability of measures across subsamples, and the difficulties in analysis imposed by the complex estimation and other procedures, among other topics. At our second meeting, the panel focused on issues of statistical methodology and heard mainly from staff of NASS and ERS with responsibility for various aspects of survey design and data collection, estimation, and processing. In the discussion of questionnaire content, the panel learned from Doug Kleweno, Jim Johnson, Carol Jones, and Bill McBride about the determinants of the content of ARMS from the perspective of resource use, farm business performance, farm household well-being, and commodities, respectively. Two presentations were made on the process of turning concepts into questions, as well as the methods of testing and implementing the questionnaires by Danna Moore of the University of Washington, and Kathy Ott of NASS. Collection methodology and other statistical aspects of the survey were discussed by Bob Bass, Bill Iwig, Alix Riley, Richard Barton, and Chadd Crouse. In the final session, Phillip Kott, Bob Dubman, and Nigel Key introduced the challenging topic of methods of imputation and analysis of the ARMS data. The panel revisited the difficult issues involved in imputation with a complex survey design in our fourth meeting. At the annual meeting of the American Agricultural Economics Association (AAEA) in August 2006, several panel members met informally with a large group of frequent ARMS data users to discuss accessibility of the data and data dissemination. The users also introduced issues regarding response rates, survey design, content, classification, editing and imputation, coding, cross-survey comparisons, and ongoing user forums. A summary of the discussion was reported to the entire panel at our third meeting. The third meeting focused on the needs of major public- and private-sector data users and on data dissemination. Useful insights on data needs by major federal users were provided by Jim Langley, Joseph Cooper, and Joseph Glauber, representing the perspective of the Congressional Budget Office, the Council of Economic Advisers, and the Office of the Chief Economist, U.S. Department of Agriculture (USDA), respectively. Bob Young discussed the data needs of the general interest U.S. American Farm Bureau Federation, and Gary Adams laid out the requirements of the statistics program of the National Cotton Council, one of the major commodity-oriented interest groups. Discussing concepts and measures for understanding environmental analysis, commodities, and farm household well-being were Timothy Kiely and Arthur Grube, U.S. Environmental Protection Administration; Jeff Goebel, National Resources Conservation Service, USDA; and Roger Claasen, Bill McBride, Carol Jones, and Jim Johnson, ERS.

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey The panel conducted a lively discussion of issues involved in dissemination of survey data, particularly to users of individual record data for research purposes, supported by presentations by Mitch Morehart, ERS, and Ron Jarmin, U.S. Census Bureau. Two substantive topics were revisited in the fourth and fifth panel meetings, which otherwise were largely focused on report preparation and discussion of potential recommendations. The panel delved more deeply into the problem of data analysis for complex surveys, with presentations by William Greene, New York University, who served as a consultant to the panel, Phillip Kott, and Jim MacDonald, ERS, who elaborated earlier presentations. Bill McBride rejoined the panel to add detail to prior discussions of the treatment of commodities in ARMS, with a focus on the implementation of recommendations by a special AAEA panel on cost of production. Agendas of the panel’s data-gathering meetings appear below. MEETING AGENDAS First Meeting February 2-3, 2006 Goals for the first meeting: Introduce panel members to each other and to the supporting staff. Conduct the bias and conflict of interest discussion. Hold an open session with representatives of the sponsoring agency and other experts to discuss the charge to the panel and to learn more about the Agricultural Resource Management Survey. Organize the approach to the task. Set dates for future meetings. Thursday, February 2 12:00 – 1:00 p.m. Welcome and Panel Introductions Bruce Gardner, Chair   Michael Feuer, Executive Director, Division of Behavioral and Social Sciences and Education   Constance Citro, Director, Committee on National Statistics

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey 1:00 – 1:45 p.m. Overview of the Agenda Bruce Gardner, Chair   Begin Discussion of the Statement of Work Panel Discussion 1:45 – 2:00 p.m. Break 2:00 – 3:00 p.m. Charge to the Panel   Kitty Smith, Associate Director, Economic Research Service   Carol House, Associate Director, National Agricultural Research Service 3:00 – 4:30 p.m. Status of the Agricultural Resource Management Survey/Issues to Be Addressed   Mary Bohman, Director, Resource Economics Division, ERS   Robert Bass, Director, Census and Survey Division, NASS 4:30 – 5:00 p.m. Bias and Conflict of Interest Discussion Kirsten Sampson-Snyder 5:00 – 5:45 p.m. Discussion of ARMS Program Issues Bruce Gardner, Chair 5:45 p.m. Adjourn for the Day Friday, February 3 8:30 – 10:00 a.m. Working Session   Review information sharing exercise from prior day; identify needed adjustments to approach Discuss issues to be examined; develop plans to address each; assign panel responsibilities Develop plans for follow-on activities

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey 10:00 – 10:15 a.m. Break 10:15 a.m. – 12:00 p.m. Working Session Continues 12:00 – 1:00 p.m. Lunch 1:00 – 3:00 p.m. Working Session   Revisit adequacy of panel expertise Determine if consultant assistance is needed Discuss audience for final report Develop outline of final report Develop plans for future meetings 3:00 p.m. Adjourn SECOND MEETING June 8-9, 2006 Goals for the statistical methodology workshop: To discuss statistical methodology issues in the ARMS program. To plan for the upcoming Concepts and Measurements workshop. To develop an outline for the final report. Thursday, June 8 8:30 – 9:00 a.m. Panel Business Bruce Gardner, Chair   Bias and Conflict of Interest Discussion Kirsten Sampson-Snyder 9:00 – 9:15 a.m. Welcome and Overview of the Agenda Bruce Gardner, Chair 9:15 – 10:00 a.m. Interagency Interaction on Issues of Statistical Methodology

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey   Robert Bass, Director, Census and Survey Division, NASS   Mary Bohman, Director, Resource Economics Division, ERS 10:00 – 10:15 a.m. Break 10:15 a.m. – 12:00 p.m. Survey Management and Data Collection Issues and Statistical Methodology Issues: Questionnaire Content   Cynthia Clark, Moderator   Information on Resource Use and   Allocation at the Field Level   Doug Kleweno, NASS   Information on Farm Business   Performance: Resources, Environment,   Other Enterprise Information   Jim Johnson, ERS   Information on the Farm Household:   Economic Well-Being   Carol Jones, ERS   Information on Commodities   Bill McBride, ERS 12:00 – 1:00 p.m. Lunch 1:00 – 5:30 p.m. Survey Management and Data Collection Issues and Statistical Methodology Issues   Fred Conrad, Moderator   Questionnaire Design   TBA   Questionnaire Development   Kathy Ott, NASS   Collection Methodology—Current and Options (CATI, CAPI, web-based);   Recruiting and Training of Interviewers   Bob Bass, NASS 5:30 p.m. Adjourn for the Day

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Friday, June 9 8:30 – 10:00 a.m. Survey Management and Data Collection Issues and Statistical Methodology Issues   Arthur Kennickell, Moderator   Sample Design and Frame (list and area)   Bill Iwig, NASS   Nonresponse (unit and item)   Alix Riley, NASS   Estimation and Imputation   Richard Barton, NASS   Nonresponse and Calibration   Chadd Crouse, NASS 10:00 – 10:15 a.m. Break 10:15 a.m. – 12:00 p.m. Survey Management and Data Collection Issues and Statistical Methodology Issues: Methods for Analysis   David Binder, Moderator   Complex Design; Jackknife   Phil Kott, NASS   ERS Procedures for Statistical Analysis   Bob Dubman, ERS   Scientific Uses in Research   Nigel Key, ERS   Review of Statistical Packages for Improving Analysis   TBA 12:00 – 1:00 p.m. Lunch 1:00 – 3:00 p.m. Working Session   Review information gathered during workshop Develop outline of final report Develop plans for upcoming concepts and methods workshop 3:00 p.m. Adjourn

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey THIRD MEETING September 28-29, 2006 Goals for the concepts and measurements workshop: To discuss conceptual and measurement issues in the ARMS program. To explore the needs of data users. To consider possible improvements in data dissemination, particularly for protected dissemination of sensitive individual records for research purposes. To explore issues regarding tests of inference for regression analysis using ARMS data. To further refine the outline for the final report and make plans for the upcoming meeting on December 7-8, 2006. Thursday, September 28 8:30 – 9:00 a.m. Panel Business Bruce Gardner, Chair 9:00 – 9:15 a.m. Welcome and Overview of the Agenda Bruce Gardner, Chair 9:15 – 10:00 a.m. Report on the AAEA Data Users Forum Bruce Gardner, Panel Members 10:00 – 10:15 a.m. Break 10:15 a.m. – 12:00 p.m. Data Uses and Users 12:00 – 1:00 p.m. Lunch 1:00 – 5:30 p.m. Concepts and Measures 5:30 p.m. Adjourn for the Day Friday, September 29 8:30 – 10:00 a.m. Tests of Inference for Regression Analysis 10:00 – 10:15 a.m. Break

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey 10:15 a.m. – 12:00 p.m. Discussion of Methods of Analysis Issues David Binder, Moderator 12:00 – 1:00 p.m. Lunch 1:00 – 3:00 p.m. Working Session   Review information gathered during workshop Develop outline of final report Develop plans for upcoming December 7-8 meeting 3:00 p.m. Adjourn FOURTH MEETING December 7-8, 2006 Goals for the inference workshop: To discuss issues regarding tests of inference for data analysis using ARMS data. To further refine the outline for the final report. To discuss preliminary findings and recommendations. To make assignments for tasks for completing the final report. To make plans for the upcoming meeting on January 18-19, 2007. Thursday, December 7 8:30 – 8:45 a.m. Introduction to Problem of Inference and Summary of Prior Discussion on the Topic   Bruce Gardner, Chair 8:45 – 9:30 a.m. Issues in Data Analysis for Complex Surveys   William Greene, New York University (Consultant) 9:30 – 10:00 a.m. Alternatives to Current Methods of Estimation and Variance Computation   David Binder, Discussion Leader

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey 10:00 – 10:15 a.m. Break 10:15 – 10:45 a.m. Practical Issues in Applying Methods of Inference to Complex Data Sets   Bill Greene, Discussion Leader 10:45 – 11:00 a.m. Summary of Issues in Complex Data Analysis   Bruce Gardner, Chair 11:00 a.m. – 12:00 p.m. General Discussion of ARMS Issues   Bruce Gardner, Chair 12:00 – 1:00 p.m. Lunch 1:00 – 5:30 p.m. Working Session   Panel discussion of issues 5:30 p.m. Adjourn for the Day Friday, December 8 8:30 – 10:00 a.m. Working Session   Review of information gathered Refine outline for final report 10:00 – 10:15 a.m. Break 10:15 a.m. – 12:00 p.m. Working Session   Assignments for preparation of final report 12:00 – 1:00 p.m. Lunch 1:00 – 3:00 p.m. Working Session   Plans for meeting on January 18-19, 2007 3:00 p.m. Adjourn

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey FIFTH MEETING January 18-19, 2007 Goals for the meeting: To discuss issues regarding cost of production estimates from the ARMS survey with ERS/NASS staff. To further refine the final report. To further develop sections written since the December meeting. To make additional assignments for tasks for completing the final report. To validate initial findings and recommendations. To make plans for the upcoming meeting in March 2007. Thursday, January 18 8:30 – 9:00 a.m. Discussion of Approach to this Meeting   Bruce Gardner, Chair 9:00 – 11:00 a.m. Discussion of AAEA Cost of Production Report and Status of Implementation of Recommendations in ARMS   Bill McBride, ERS 11:00 a.m. – 12:00 p.m. General Discussion of ARMS Issues with Department of Agriculture Staff   Bruce Gardner, Chair 12:00 – 1:00 p.m. Lunch 1:00 – 2:30 p.m. Break-out Working Sessions 2:30 – 5:30 p.m. Panel Discussion of Issues 5:30 p.m. Adjourn for the Day Friday, January 19 8:30 – 10:00 a.m. Working Session   Refine Final Report

OCR for page 177
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey 10:00 – 10:15 a.m. Break 10:15 a.m. – 12:00 p.m. Working Session:   Assignments for Final Report 12:00 – 1:00 p.m. Lunch 1:00 – 3:00 p.m. Working Session   Plans for Meeting in March 3:00 p.m. Adjourn