3
Survey Management

The complexity of the Agricultural Resources Management Survey (ARMS) extends to matters of direction, administration, and financing. Although this situation is by no means unique in the federal statistical system, understanding it does require a thorough understanding of the substantive, statistical, and institutional dynamics underlying the survey. A pattern of governance has evolved that includes a steering committee and numerous supporting committees and boards. This shared web of governance has enabled the survey to move forward gradually, but it may require reorganization in the future, depending on the directions for future growth chosen by the U.S. Department of Agriculture (USDA) or Congress.

COLLABORATIVE MANAGEMENT

Representing USDA in managing ARMS, the National Agricultural Statistics Service (NASS) and the Economic Research Service (ERS) share responsibility for the subject matter in a complementary manner. The approaches the agencies take to the various management tasks are quite different, because the roles and missions of the agencies are different.

NASS has a dual role: it is both the data collector and a major user of data derived from ARMS. The statistics it produces are usually seen as descriptive. They are for the most part, univariate counts, means, and totals for specific classifications of farms, farm production, or crops. Data on agricultural commodities, production costs and expenses, chemical and



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey 3 Survey Management The complexity of the Agricultural Resources Management Survey (ARMS) extends to matters of direction, administration, and financing. Although this situation is by no means unique in the federal statistical system, understanding it does require a thorough understanding of the substantive, statistical, and institutional dynamics underlying the survey. A pattern of governance has evolved that includes a steering committee and numerous supporting committees and boards. This shared web of governance has enabled the survey to move forward gradually, but it may require reorganization in the future, depending on the directions for future growth chosen by the U.S. Department of Agriculture (USDA) or Congress. COLLABORATIVE MANAGEMENT Representing USDA in managing ARMS, the National Agricultural Statistics Service (NASS) and the Economic Research Service (ERS) share responsibility for the subject matter in a complementary manner. The approaches the agencies take to the various management tasks are quite different, because the roles and missions of the agencies are different. NASS has a dual role: it is both the data collector and a major user of data derived from ARMS. The statistics it produces are usually seen as descriptive. They are for the most part, univariate counts, means, and totals for specific classifications of farms, farm production, or crops. Data on agricultural commodities, production costs and expenses, chemical and

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey pesticide use, and farm income and assets find their way in tabular form into NASS publications and other agency products. The role of ERS is less sharply drawn. At one level, ERS can be characterized as a consumer. Indeed, if a distinction were drawn between producer and consumer of the ARMS data within USDA, it would be appropriate, if not entirely accurate, to identify NASS as the producer and ERS as the consumer. But ERS is a very active and involved consumer. Because ERS considers ARMS to be vital to accomplishing its mission of research and analysis, the agency plays a significant role in driving ARMS content, particularly in areas that call for knowledge of the economics of agricultural production and the farm household. Like NASS, ERS produces descriptive statistics, although its primary focus is analytical. To support its policy work and longer term research, ERS makes extensive use of multivariate models. A fundamental difference between descriptive and analytical work is that the former can often be computed using independent sources of information, whereas the latter typically requires the full set of variables collected from each unit observed. Thus, all other things being equal, analytical users tend to press for increasing the scope of a survey, while descriptive users may be more sensitive to respondent burden issues that may lead to nonresponse or other aspects of survey operations that would contribute to error and variability. The influence of ERS is most strongly present in the development and analysis of the data from the Phase II and Phase III surveys. ERS has expertise in the development of information on environmental resource management and has worked collaboratively with NASS to frame the Phase II collection of data on chemical and pesticide use on cropland. The agency’s primary interest and ownership is over the Phase III survey operations, in which ARMS collects basic economic data on income, expenses, and debt annually. The ARMS economic questionnaire supports an ERS program of data analysis on farmers’ use of particular marketing channels and on management decisions and farm household well-being, including operator demographics. By combining data from the Phase II and Phase III questionnaires for the overlapping portion of the sample, ERS is able to add value by making ARMS a very powerful survey for analyzing the relationship of the environmental and economic components of agricultural production. Program Funding Most of the funding for ARMS has traditionally come from NASS, although ERS contributes substantially and increasingly to the financing of the survey. The reimbursement from ERS totaled $6.75 million for the base survey in fiscal year (FY) 2005 and FY 2006. In FY 2005 and 2006, NASS funding continued at the FY 2003 level of $9.9 million. The cost of ARMS

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey TABLE 3-1 ARMS Program Funding by Agency, Fiscal Years 1996-2006 (in millions of dollars) Fiscal Year National Agricultural Statistics Service Economic Research Service Total 1996 N/A $2.3   1997 N/A 4.0   1998 N/A 4.0   1999 N/A 3.9   2000 N/A 3.7   2001 $5.1 3.6 $ 8.7 2002 5.3 4.0 9.3 2003 9.9 6.4 16.3 2004 9.9 6.7 16.6 2005 9.9 6.7 16.6 2006 9.9 6.7 18.7 N/A = Not Available. was approximately $18.7 million in FY 2006, not including the time of about 36 ERS staff who are actively involved on at least a part-time basis with ARMS data development or who use ARMS data in their research. The funding for the survey jumped significantly in FY 2004, when an additional sample was added to provide data for a 15-state oversample, going from less than $10 million per year to over $16 million. Since that quantum jump in resources, the ERS contribution has been accelerating and the NASS contribution has remained fairly constant, although NASS continues to provide the bulk of the funding (Table 3-1).1 The FY 2005 NASS survey costs were distributed between data collection (44.7 percent), staff (48.2 percent), and direct and indirect (7 percent) costs. The ERS resources committed from its general appropriation to ARMS data development and research involve several units in the following program areas: agricultural structure and productivity, farm and rural household well-being, farm and rural business, and production economics and technology. Depending on the subject matter of interest, four other program areas have a role on the NASS/ERS steering committee: diet, safety, and health; animal products; grains and oilseeds, specialty and fiber crops; and resources, environmental, and science policy. About 7.5 full-time-equivalent employees are dedicated to ARMS, and 20 additional researchers in ERS use ARMS data. The full-time staff for ARMS is concentrated in the Resource and Rural Economics Division. 1 ERS funding for FY 2004 and FY 2005, includes $200,000 and $250,000, respectively, to fund collection of data on organic commodities—in 2005 for Phase III organic dairy and, in 2006, for Phases II and III organic soybeans.

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Division of Responsibility for ARMS Functions As it has evolved over time, the division of responsibility for ARMS follows the lines described above for each of the key functions of survey management: design, development, operations, and analysis/dissemination. Table 3-2 summarizes these functions. TABLE 3-2 Division of Responsibility for ARMS by Function Key ARMS Functions National Agricultural Statistics Service Economic Research Service Resources Obtains base resources for program of statistical methodology, survey operations, and preparation of NASS publications. Obtains supplemental resources to provide to NASS for conducting survey and funding for organics research and internal ERS funding for analysis and dissemination program. Survey approval process Prepares and defends OMB information collection request for survey. Assists in preparation of OMB information collection request. Identification of concepts and desired outputs Develops and implements statistical concepts. Aligns survey topics and questions with measurements and analytical goals. Survey specifications Obtains feedback from state offices, data collectors, and respondents regarding past surveys and develops questions based on needs of internal and external users. Identifies new or revised questions to be included in the survey based on interactions with users. Cognitive testing of questionnaires and modes of collection research Conducts program of cognitive development and testing of questionnaires and research into modes of collection.   Presurvey activities Identifies resources and develops operating plan for upcoming survey.   Sample design Primary responsibility for designing the surveys and integrating the phases. Reviews the sample allocation to offer analytical insight into state and commodity coverage. Edit design process Establishes criteria and designs the edit. Offers input into edit parameters. Preparation of manuals and training materials Prepares the survey operations manuals and training materials. Participates in writing the sections of enumerator manuals that pertain to analytical issues. Conducting supervisor and enumerator training Conducts the field supervisor (survey statistician) and the state-level enumerator training in coordination with NASDA. Participates in survey statistician training; assists in conducting state-level training.

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Key ARMS Functions National Agricultural Statistics Service Economic Research Service Predata analysis Develops data analysis tools that will be used by state and national state during the survey process. Contributes expertise in development of data analysis tools. Data analysis Produces a clean data set with as few data “errors” as possible. Uses computer interactive data analysis capabilities. Assists in use of computer-interactive data analysis; assists as needed and depending on the availability of staff. Imputation and final edit Conducts imputation and edit functions. Assists on a periodic basis. Summary Prepares summary data files.   Outlier identification and board process Takes lead in interacting with states to draw input on possible outlier records; convenes and manages outlier board; develops rules for outlier identification. Reviews information and participates as members of outlier board; assists in developing rules for outlier identification. Final summary Prepares final summary data files.   Transmittal to ERS Transmits data files to ERS.   Farm production expenditures board Conducts production expenditures board, regional review board, and state review board. Participates in the three board processes. Releases of farm production expenditures Publishes farm production expenditures release in paper format and electronic format via the Internet.   Releases of other Phase II and Phase III information   Publishes several products to release Phase II and Phase III data to public. Preparation of research databases   Uses ARMS file to produce estimates of farm business and household income and balance sheets for release through briefing rooms and ARMS data tool; prepares ARMS research databases. ARMS data dissemination tools Participates in developing ARMS data dissemination tools to ensure that appropriate rules are followed for maintaining data confidentiality. Develops and maintains ARMS data dissemination tools. OMB = U.S. Office of Management and Budget; NASDA = National Association of State Departments of Agriculture.

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Coordinating Mechanisms The NASS/ERS Steering Committee is the principal coordinating mechanism for ARMS. This committee is co-lead by the environmental and economic surveys section head from NASS and the ERS Deputy Director for Resource and Rural Economics Data, and is composed of senior management staff in each division of both organizations—six from ERS and four or five from NASS. There is a lot of interaction between the agencies, their various branches, and the steering committee. NASS and its branches focus on some of the following topics in the steering committee: The environmental, economics, and demographics branch in the statistics division serves as coordinator between NASS and ERS in matters of analysis design, and implementation, data analysis strategies, coordinates outlier treatment, and is responsible for the estimation process that culminates in NASS publications and delivery of the final data set to ERS. The economics section has responsibility to develop and implement specifications for all edit, summary, analysis, and estimation programs. The sampling branch designs the list and area frame samples for each phase of the survey, develops sample weighting procedures, and provides assessments of the sample designs. The data collection branch has responsibility for computer-assisted survey information collection (CASIC), a common system for editing survey results. NASS partners with ERS on the tasks of questionnaire design and defining data edits. Specifications for data entry using CASIC and editing (within sample unit) are located in the data collection branch. The survey administration branch (SAB) provides administrative instruction and coordinates all data collection activities conducted in the NASS field offices. The survey administration branch has responsibility for questionnaire content through collaboration with ERS; and scheduling, training, edit testing, completion of interviews (questionnaires), and monitoring response rates. SAB provides project management oversight for coordinating the development of survey specification documents, coordination of project resources, and for preparing the OMB Docket requesting approval to conduct the survey. The statistical methods branch in the statistical division develops the cross-record edits to identify outliers and produces the aggregate survey indicators and estimates. The research branch develops the procedures for calibration and variance estimation and evaluates the effectiveness of these procedures for providing a measure of reliability of the aggregate estimates for each variable.

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey The ERS representatives on the steering committee focus on the agency’s areas of responsibility: responding to congressional and USDA mandates, informing policy decisions, supporting national accounts, and providing information for USDA program management. In addition, ERS interest focuses on the impact of survey design and methods on the relationships between variables as assessed in econometric models. The steering committee is the decision-making body for survey requirements and adding or subtracting questions, although issues can go to NASS and ERS management if the steering committee cannot come to an agreement. This formal mechanism is designed to assist the agencies in coordination and information sharing. Larger decisions, such as designing a new sample for an extra survey or a new program, are made at the senior management level of NASS and ERS involving the administrators, since it may require extra funds to be allocated. However, there are limited opportunities for these major decisions to be made, since it is difficult to make changes once the survey is in the field. The steering committee has proven to be very protective of ARMS in managing its respondent burden and survey content. MANAGING THE CHANGING FOCUS OF ARMS These budgeting and survey management coordinating mechanisms have worked well by ensuring a common approach by the two agencies during the gestation and maturing periods of the survey. However, given the differences in perspective of NASS and ERS, the current management framework may not be sufficient to ensure that the survey will have the capacity to change to meet future data needs or to coordinate the increasingly complex technical and methodological environment that may define its future. In the future, the survey is expected to focus less on descriptive data and more on the type of high-level multivariate analysis represented by ERS and its constituency. Other models of survey management employed for other major U.S. government surveys may have potential for better focusing responsibility for directing and funding ARMS for the future. One mechanism is the notion of survey sponsorship, in which the agency that is the primary consumer of the information from the survey has responsibility for its overall direction and, critically, for securing funding. A number of Census Bureau surveys, most notably the Current Population Survey (CPS), are managed with this sponsorship model. The CPS is the monthly household labor force survey for the United States conducted by the U.S. Census Bureau for the Bureau of Labor Statistics (BLS). In the case of the CPS, BLS has primary responsibility for determining the labor force and other socioeconomic concepts and definitions, as

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey well as the questions to be posed, in collaboration with experts at the Census Bureau. In order to fulfill this responsibility, BLS has assembled a small, highly skilled staff of mathematical statisticians and cognitive scientists to support its oversight role, again in coordination with the experts in these disciplines at the Census Bureau. These resident skills permit BLS to communicate its requirements for information to support its program of research, analysis, and publication in a sophisticated manner. For example, data requirements are developed with specific coefficients of variation incorporated, and the trade-offs between cost (sample size) and reliability are discussed in those terms. Importantly, BLS has the responsibility for securing the bulk of the funding for the CPS, with funding for only the March income supplement and other specific collections used mainly by the Census Bureau remaining in the Census Bureau budget. Funds are transferred from BLS to the Census Bureau annually through the formal mechanism of a cooperative agreement, which establishes targets and standards for the survey. Sponsorship arrangements have the benefit of aligning use with cost. In the context of the federal government, they cause the ultimate customer to justify the information sought through a formal budget defense process and to be responsive to other governmental processes that seek to ensure that performance measurement is associated with resource inputs. In considering the current cooperative survey management structure for the ARMS program, the panel notes the potential opportunities of a sponsorship model that could clarify responsibility for the survey and more closely align the resources with its end uses. A potential sponsorship model for ARMS would include an increased role for ERS in developing the ARMS program, which may involve a shift in funding through the ERS budget (except perhaps for funding for NASS-specific products to meet NASS mandates). It would certainly require changes in staffing. For example, ERS may need to deepen its bench of mathematical statistics and cognitive science skills in order to assist in overall direction of the program. The panel lacks expertise in organizational design to assess the benefits and costs of recommending a switch to a sponsorship model or any other particular model, such as the model employed by the National Center for Health Statistics and the Agency for Healthcare Research and Quality with regard to joint management of the Medical Expenditure Panel Survey (MEPS) and the National Health Interview Survey (NHIS), which provides the sampling frame for the MEPS. However, the shifting priorities for use of the data require a management structure that will ensure that the survey continues to be responsive while maintaining an appropriate emphasis on data quality. The Department of Agriculture may wish to investigate the sponsorship model as one way these shifts may be managed.

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Stakeholder Feedback Although coordination and input mechanisms in USDA are formalized and fairly sophisticated, the same cannot be said for its arrangements for obtaining, filtering, and implementing input from stakeholders and users. In many agencies across the federal government, the mechanism for facilitating communication with outside interest communities in an open and structured manner is the advisory board. However, the advisory board mechanism has not proven to be very effective in the case of ARMS. The official NASS Advisory Committee on Agriculture Statistics is an organized and active body, but it is mainly focused on the Census of Agriculture and its related programs. This is understandable, since the NASS advisory committee was inherited from the Census Bureau when the Census of Agriculture was transferred from the U.S. Department of Commerce to USDA. The advisory committee does, from time to time, consider other issues. However, a review of all previous agendas and recommendations from the NASS advisory committee can find no reference to the advisory committee’s reviewing ARMS content per se. Nonetheless, ARMS is connected to the Census of Agriculture in the year of the census through coordination of the sample and questionnaire design, so the advisory committee has addressed issues of consistency between ARMS and the Census of Agriculture. The advisory committee did recommend in the 2002 annual meeting that NASS proceed with efforts to integrate concepts and processes of the agricultural census, ARMS, and related year-end surveys. This recommendation was implemented, with the result that operations selected for ARMS in 2002 were not mailed a census form. The census data were generated from ARMS. NASS has decided that a similar process will be used for the 2007 Census of Agriculture. Although this process reduces respondent burden, it does raise concern that if the questions or the context of questions differ in any respect between the two forms, then the estimates from the two questionnaires may well be different, resulting in bias in the census estimates. In the absence of a formal advisory mechanism, both NASS and ERS conduct periodic but largely ad hoc meetings with data users outside USDA in an effort to receive feedback on their statistics and research programs and to hear what policy issues need to be addressed. These user forums are sometimes cosponsored and may be specific to different types of data, such as crop or livestock production, economic, or environmental data. The feedback from these meetings is channeled to the ARMS steering committee if it relates to issues that the ARMS program might address. Over the years, USDA has solicited input from outside users in a variety of ways. One primary mechanism has been through informal cooperative

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey arrangements with the American Agricultural Economics Association (AAEA). As a professional association of scientists who use economic tools to analyze issues and solve problems in the area of agriculture, food, and environmental resources, the AAEA has long maintained an Economic Statistics and Information Resources Committee (ESIRC), which takes a close and continuous interest in ARMS as well as the other products of NASS and ERS. This committee has at times taken on studies and fostered research and analysis designed to address difficult conceptual and measurement issues in agriculture. A primary example of this informal collaboration is the Commodity Costs and Returns Handbook. This monograph was prepared by a task force organized by AAEA’s economic statistics and information resources committee, on the basis of recommendations stemming from an AAEA conference in 1991. The mission given to the task force by the committee was “to recommend standardized practices for generating costs and returns estimates for agricultural commodities after a careful examination of the relevant economic theory and the merits of alternative methods.” Most of the recommendations of this task force were subsequently adopted by ERS.2 The AAEA can provide a convenient base for collecting informal and current input on issues. For example, to obtain user input to this report, the panel sponsored a special user forum at the 2006 annual meetings of the AAEA. The feedback on the research uses of ARMS data, survey needs, and access requirements was invaluable in identifying issues to be addressed by the panel. ERS and NASS may want to jointly sponsor such forums on a regular basis in the future. Another way to gain outside input on program content and priorities is through the ARMS online briefing room, which is maintained by ERS. This innovative briefing room provides a mechanism for user feedback and often broadcasts requests for input on specific aspects of ARMS. Recommendation 3.1: The ARMS program should have structured mechanisms in place for stakeholder feedback and discussion on ARMS, beyond what is currently done, such as organized stakeholder forums, with some obligation to respond. Specifically, USDA should solicit input in developing the survey from stakeholders from within USDA and from other government agencies, universities, professional associations, and the private sector. Recommendation 3.2: The NASS Advisory Committee on Agriculture Statistics should expand its scope to include an annual review of ARMS. Research and Development Every official statistics survey program that operates on a continual basis should have associated with it a methodology research and develop- 2 Presentation by Bill McBride, Economic Research Service, January 18, 2007.

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey ment program. The jointly sponsored ARMS program suffers because such a program has not been formalized. This is not to say that there has not been some research and development during the decade of ARMS and its predecessor programs. There have, in fact, been some pieces of quite good research. However, these have been initiated primarily when a major change in the ARMS program has occurred or has been proposed. For example, when a mail economic questionnaire was introduced to increase the sample size, there was an effort to design a self-administered questionnaire to use alongside the interviewer-administered instrument. Similarly, a need arose for easier implementation of the complex sample design into calculations of variance to use in tests of significance, and so the delete-a-group jackknife procedure was developed. An official statistics program should have an established time frame during which changes may be made to the sample and survey design, the questionnaire, the resulting editing and processing system, and the weighting, estimation, and other adjustment systems (such as statistical disclosure or seasonal adjustment). The survey design would be a factor in determining what an appropriate period of time is for changes to the design. Once this is set in place, a research and accompanying development program would be put in place to operate to the time schedule. The ARMS program is in need of this formal structure to address both survey methodology and analysis of the results of the survey to inform policy-relevant issues. In view of the fact that ARMS is a jointly funded survey, the cycle for survey design changes and the methodology research and development program should be a jointly established and governed program. Joint goals should be set. The panel observes that many of the questions the agencies posed to us could have been answered by one or both of the agencies if such a program were in place. As noted in Chapter 2, there has been a marked increase in interest in doing econometric research with the ARMS data. This has resulted in new requests for statistical support directed to an already-overtaxed NASS methods unit to facilitate analysis of complex survey data. These needs should be factored into the methodology research and development program to ensure that the data files and paradata are in place to support the appropriate statistical analyses for the complex survey data. More importantly, resources need to be made available in both NASS and ERS to support the program of statistical analysis of survey data, which is increasingly based on highly sophisticated techniques and methodologies. Establishing joint governance of such a research and development program for ARMS would enhance the capabilities of both organizations to provide input and resources. As part of the methodology research and development program, both agencies might consider some options as to how they might work together in a more collaborative mode. It is conceivable that the two agencies might be able to fund one or more researchers

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey through the American Statistical Association/National Science Foundation program to contribute to this program. Such a joint program would ideally involve some skill transfer between the two agencies. Neither organization has many individuals with competence in the main skill set of the other agency. If NASS were to hire a few economic researchers familiar with survey research, and ERS were to hire a few mathematical statisticians with some background in economics, these individuals might help to provide a bridge between the survey focus of NASS and the econometric focus of ERS. An additional option (and possible interim solution) might be for the current senior mathematical statisticians at NASS and senior economic researchers at ERS to be sent to the sister agency to sit with agency staff for several days in each pay period. Many models for the kind of research and development program envisioned here exist in the federal statistical community. For example, with respect to research and development on questionnaire and data collection processes, there are ongoing programs that are regularly revised on the basis of studies by in-house methodologists: the CPS and the Consumer Expenditure Survey at BLS; the decennial census, the American Time Use Survey and the Survey of Income and Program Participation at the Census Bureau; and the National Health Interview Survey at the National Center for Health Statistics are some examples. The research typically involves a mix of laboratory and field activities (including the analysis of paradata, which we discuss in Chapter 5) conducted by methods researchers, usually with advanced degrees in the social sciences. Some agencies rely heavily for graduate training of existing employees on the Joint Program in Survey Methodology (JPSM). Over the years, this program has strengthened the methodological programs in all of these agencies and would be a useful resource for NASS and ERS as well. JPSM primarily produces master’s-level graduates and is expanding its Ph.D. program. The kind of research program we are advocating requires at least some Ph.D.-level staff, because such training leads to greater expertise in conducting methodological research. Finally, staffing this effort in-house is important because internal researchers understand the problems in a way that contractors rarely can. Recommendation 3.3: ERS and NASS should establish an ongoing, jointly sponsored, and appropriately funded methodology research and development program. The program should provide adequate resources to support current and future research, development, and statistical analysis needs throughout the implementation of ARMS and to assess and manage the quality of the data. If new funds cannot be obtained, funds from existing programs must be reallocated.

OCR for page 49
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Need for a Long-Term Strategy and Plan As with most government survey operations, as ARMS has matured, it has settled into a comfortable repetition of tried and true formats and collection rounds that have been periodically revised. Some significant strategic decisions have been put in place over the years, such as the closer alignment of ARMS with the 5-year Census of Agriculture. However, this has occurred largely without the discipline of a structured, long-term planning process. In the larger picture, the managers of ARMS have other venues for enhancing the use of the data for econometric policy-relevant analyses. One such venue occurs periodically with the reauthorization of the farm bill. Although ARMS data, as they are, are very useful for evaluating policies implemented by this bill, the cycle of reauthorization affords a once-each-five-year opportunity for ARMS to reach out and preemptively to develop a plan for collecting farm bill–related data in a policy evaluation framework, with an emphasis on specific policies of interest. For instance, adapting a pre- and postsurvey component for ARMS in conjunction with policy implementation or working with administrators to evaluate a randomized trial of some farm programs could dramatically enhance the value of ARMS as a policy analysis tool. For this and other reasons, it makes sense for ARMS to operate with a five-year plan in order to fit into the five-year cycle of the Census of Agriculture. Within each cycle, there is reason to hold the basic survey relatively constant, with changes permitted on an annual basis to add extra modules outside the core survey questionnaires or to enumerate follow-on surveys within the ARMS sampling frame. Thus, for example, inserting a question into the core set of questions should not be permitted within a given five-year cycle, except under extreme circumstances. More significant changes designed to maintain the relevance of ARMS in a changing farming environment could be made once each five years, with provision made for bridging the old and new times series in a manner that would enhance the value of the time series data to users. This would have a number of benefits, in addition to stabilizing the time series for data items of interest. For one thing, it would diminish the problems of recoding data items from year to year, which has had a confusing effect on users of ARMS microdata. Recommendation 3.4: NASS and ERS should commit resources to developing a five-year plan tied to the Census of Agriculture for ARMS content, coverage, and methodology. The agencies should develop measures to control changes during the five-year period to minimize disruptions to the time series of the core content in ARMS.