8
Dissemination

In reviewing the dissemination procedures in place for the Agricultural Resource Management Survey (ARMS), the panel drew on our own experience as users of the aggregated and microdata and reached out to data users in government and the research community. This chapter outlines the steps the Economic Research Service (ERS) and the National Agricultural Statistics Service (NASS) have taken to improve access. Internet capabilities have grown with the ARMS briefing room and the various levels of access and resources available.

Both ERS and NASS have taken some significant steps in the past few years to increase access to ARMS data, in both aggregated and individual record formats. Although hard copy publications are still prepared and offered, aggregated data are provided via the Internet for the most part, and individual record data are accessed mainly by computer terminals at ERS or NASS state offices.

Hard copy publications are fast becoming a thing of the past. NASS publishes two hard copy reports from ARMS. The first, called Agricultural Chemical Usage—Field Crops, is released in May following the Phase II data collection. The second report, Farm Production Expenditures, is compiled from the Phase III and released in July. ERS prepares several hard copy state, regional, and national reports using ARMS data, including Commodity Production Costs and Returns, Farm Operating and Financial Characteristics and the Annual Report to Congress on the Status of Family Farms (National Agricultural Statistics Service, 2007c).

As a general rule, however, hard copy sources are being supplanted by websites supported by sophisticated web tools, microdata files, and special



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey 8 Dissemination In reviewing the dissemination procedures in place for the Agricultural Resource Management Survey (ARMS), the panel drew on our own experience as users of the aggregated and microdata and reached out to data users in government and the research community. This chapter outlines the steps the Economic Research Service (ERS) and the National Agricultural Statistics Service (NASS) have taken to improve access. Internet capabilities have grown with the ARMS briefing room and the various levels of access and resources available. Both ERS and NASS have taken some significant steps in the past few years to increase access to ARMS data, in both aggregated and individual record formats. Although hard copy publications are still prepared and offered, aggregated data are provided via the Internet for the most part, and individual record data are accessed mainly by computer terminals at ERS or NASS state offices. Hard copy publications are fast becoming a thing of the past. NASS publishes two hard copy reports from ARMS. The first, called Agricultural Chemical Usage—Field Crops, is released in May following the Phase II data collection. The second report, Farm Production Expenditures, is compiled from the Phase III and released in July. ERS prepares several hard copy state, regional, and national reports using ARMS data, including Commodity Production Costs and Returns, Farm Operating and Financial Characteristics and the Annual Report to Congress on the Status of Family Farms (National Agricultural Statistics Service, 2007c). As a general rule, however, hard copy sources are being supplanted by websites supported by sophisticated web tools, microdata files, and special

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey centers for access to confidential data in a protected environment. These new forms of dissemination have increased the need for training, formal mechanisms for communication with users, and tightened procedures for control over the data. NEW DATA SOURCES AND PRODUCTS The most ambitious new product is the ERS-managed online ARMS briefing room.1 This website provides summary data as well as extensive ARMS documentation and access to questionnaires. The data available on the website are retrievable in the form of tailored reports and summary tables. The tables provide means and standard error estimates, they can be saved as comma separated value (CSV) or Excel files, and they have a capacity for graphical display of data. There are several means of accessing less aggregated data. The ERS produces special tabulations, typically for government agencies. ERS has provided data and research support using ARMS directly to the U.S. Department of Agriculture (USDA) Office of the Secretary, the Office of the Chief Economist, Research, Education and Economics, the National Resource Conservation Service (within NASS), the Farm Service Agency, the Foreign Agricultural Service, the Animal and Plant Health Inspection Service, and the Risk Management Agency. ERS has also used the data to respond to requests from Congress, the Office of Management and Budget, the Bureau of Labor Statistics, universities, and international organizations. The myriad uses of ARMS data require sophisticated access to data in order to depict increasingly complicated relationships. For example, recent analyses of the financial status of farming used ARMS data to prepare outlook information for farms and farm households by type of farm, size of farm, and U.S. region; balance sheets for farms by size of farming operation; a cumulative distribution of farms by economic cost-to-output ratios; and the distribution of key commodity production by production cost level. Other examples abound. ARMS data were used to provide a wide variety of information types: Distribution of government payments by farm size Farm household income by farm typology Impacts of energy price increases on farm businesses Impacts of seed price increases Characteristics of U.S. production of biotechnology-derived crops Crop insurance usage by typology, commodities, and regions Types of farm management practices 1 Available: http://www.ers.usda.gov/Briefing/ARMS/

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Agricultural land ownership by women Cost of production for corn and soybeans Limited resource farmers by state Changes in American farming from 1988 to 2001 In addition to supporting research that depends primarily on ARMS data, ARMS contributes to other research, analyses, and situation-and-outlook work in federal and state governments, academic institutions and other organizations because it provides the basic cost-of-production and supply response information on which other analyses depend. Examples of ARMS use in such research include USDA agricultural baseline projections to 2010. Baseline reports provide long-run (10-year) baseline projections for the agricultural sector. The reports contain cost-of-production data that are the baseline for projections covering supply and demand for agricultural commodities, agricultural trade, and aggregate indicators of the sector, such as farm income and food prices. Cotton. Background and issues for farm legislation. Recently issued commodity background reports, developed to inform the farm bill debate, provide charts and discussion on the distribution of costs across farms and other data derived from ARMS. Managing risk in farming. Concepts, research, and analysis. This report on the risks confronted by grain and cotton farmers and risk management tools and strategies used at the farm level uses ARMS-based data on farmers’ assessments of the risks they face and their use of alternative risk management strategies. Effects of federal tax policy on agriculture. Comparisons between financial and tax variables reported on farmers’ tax returns and the ARMS data highlight differences between accounting for tax purposes and the underlying economic realities. ARMS Web Tool ERS advertises that any user can get customized data summaries provided by the publicly available online data tool at http://www.ers.usda.gov/Data/ARMS/. Tailored reports enable custom queries, in which users can select from a set of variables and customize the estimates they receive, refine queries with specific samples or populations, group summary statistics for comparisons, and choose among output options for results (tables, charts, etc.). The basic customized data summaries available through the ARMS briefing room are broken into four major data topics: (1) farm structure and finance, (2) crop production practices, (3) commodity production costs

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey and returns, and (4) featured states. Within each topic area users can create tailored reports from the survey data. Farm structure and finance reports contain information on the structure and financial status and performance of U.S. farm operators, their households, and farm businesses. Crop production practices summaries include information about the status and trends in crop production practices for several field crops. Commodity production costs and returns summaries include statistics on the annual production costs and returns for major field crop and livestock commodities. Special reports for the 15-state oversample are available for the time period 2003-2005. The ARMS web tool for customized summaries is a convenient and effective way to disseminate findings from ARMS to the general population. It is user-friendly, and the summaries and findings are easy to interpret. A major strength is that the ARMS web tool can also provide summaries of variables by categories or for multiple years. Individuals interested in ARMS can conveniently begin their exploration using the web tool. In the panel’s view, ERS should continue to update and develop the web tool as their primary means of disseminating ARMS findings to the public and to data users. One suggestion for improvement is to continue to expand the list of variables that are summarized. ARMS Extranet Web Tool In addition to the publicly available ARMS web tool, ERS has developed an advanced statistical analysis web tool, which is available to approved researchers with proper authorization. Currently this web tool can perform regression analysis with variables and by categories that are predefined and provided by ERS. All output is screened to ensure data confidentiality. The ARMS extranet web tool has great potential to facilitate researchers in the conduct of preliminary or full analyses of their models. It allows researchers more flexibility to estimate advanced models and the convenience of using the ARMS data remotely, without compromising data confidentiality. This tool has great potential to increase the availability of the ARMS data to researchers for whom it is costly or inconvenient to access the data on-site. Although it is useful for preliminary analyses with the data, the current functionality of the tool is limited to ability to support ordinary least square regressions with predefined variables, and this has prevented data users from fully using the tool for research. It can be made more useful for research purposes. One improvement would be to allow researchers to use, redefine, and combine variables in the raw data set instead of only the aggregate, predefined variables that are available now. A second improvement would be to continue to expand the models that are available for estimation

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey (limited dependent variable models, etc.). Another improvement would be to allow for different years of data to be included in one model. Recommendation 8.1: ERS should continue to improve the ARMS web tool by providing summaries on more variables and more subsets from ARMS, and to improve the ARMS extranet web tool by adding the ability to link over years and to more sophisticated models. Special Tabulations On occasion, users may request special tabulations of ARMS data beyond what is available in published or downloadable reports and tabulations. Special tabulations require a commitment of time and expertise of agency staff. Under ERS policy, the agency may provide special tabulations, reimbursed on the basis of staff time required to prepare the tabulations, provided the staff are available. ERS policy is that special tabulations, once prepared, are available to the public, and the agency reserves the right to disseminate the results of special tabulations on the agency website or in agency publications. NASS has a similar program (National Agricultural Statistics Service, 2007d) and has produced special requests for users for a fee—an example that is cited is a report for John Deere. The reports produced in this manner are nonproprietary and are published on the USDA website. MICRODATA ACCESS FOR DATA USERS The individual records derived from ARMS have been collected and acquired exclusively for statistical purposes under a strict pledge of confidentiality. However, with the recognition that the individual records are valuable in understanding the status of farms and farm families, procedures have been established to provide access to microdata in an environment that ensures that the confidentiality of the records is maintained. Currently, approved data users can access the ARMS microdata either onsite at ERS or NASS offices. They can also access the data in a more limited form and run simple regressions online. As discussed in this section, alternative means of dissemination, particularly through the Census Bureau Data Centers, may increase access for some researchers or facilitate analysis that combines data sets. Procedure for Accessing the ARMS Microdata ARMS individual-record microdata can be made available to researchers and other government agencies that have collaborative projects with ERS or NASS that contribute to USDA’s public-sector mission. These projects must be formally administered through a cooperative research relation-

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey ship between ERS and a responsible research organization. Users apply by submitting a memorandum of understanding for research purposes between ERS and the research institution, an approved research project agreement, and a NASS confidentiality agreement. Users also must participate in a security briefing on the security and confidentiality requirements of using ARMS data. Once formal agreements have been signed, access to ARMS data is provided at ERS or NASS headquarters or at a NASS state statistical office, depending on availability of office and computer resources. ERS takes these steps to ensure that ARMS data are used for statistical purposes only, not for other purposes, such as regulatory or enforcement purposes or release under the Freedom of Information Act. The objective is that confidential data will not be disclosed. In addition, all reports, publications, and releases based on ARMS data must pass strict nondisclosure reviews. Entities and individuals outside USDA with access to confidential survey data are subject to the same federal statutes that apply to USDA and its employees. Under these statutes, individuals who unlawfully disclose confidential data are subject to fines and other penalties. The procedures for accessing the microdata are designed to establish widespread and uniform confidentiality protections that cooperators must adhere to. They take advantage of the Confidential Information Protection and Statistical Efficiency Act of 2002, which grants “licensed agents” use of confidential data for statistical purposes, providing criminal and civil penalties for misuse. Current Users and Projects Researchers can access the data through their institutions or directly with USDA. Over 25 institutions, primarily academic institutions and some government agencies, have agreements in place to gain access to the ARMS data on-site at ERS or NASS offices or online via the ARMS extranet tool. A number of ongoing initiatives are aimed at improving the dissemination of and access to ARMS data. They include cooperative agreements with the University of Illinois and the University of Minnesota, web design changes, and continuing research on alternative approaches to access. On-site Data Access at ERS and NASS State Offices To access the ARMS microdata at ERS or NASS state offices, researchers contact the NASS office in their state and arrange visits to estimate their models with the ARMS microdata. NASS offices provide data users with a secure computer on which the ARMS data and the statistical analysis system (SAS) software and Microsoft office programs are installed. Because time spent at NASS offices is limited by travel costs and work schedules, researchers need to arrive well prepared. They typically bring computer

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey programs to the NASS offices and spend the majority of their time running and modifying their programs and compiling the results. Before they leave, ERS or NASS staff inspect all output and results for compliance with the confidentiality of the records. For most university researchers, the availability of the ARMS data at the NASS state offices significantly reduces their travel costs in comparison to the alterative of traveling to Washington, DC. Nonetheless, many users find it inconvenient having to travel sometimes considerable distances to a state office. In addition, some have encountered difficulties in processing the data at the state centers. Users have called for making the data available in the state centers in a format that would facilitate the use of standard computer languages other than SAS, such as STATA, Limdeb, and Matlab. Policies on software vary from state office to state office. The software available may differ, and some state offices seem to be uncomfortable with allowing users to upload their own software to computers in the state office. At the panel’s session for ARMS data users at the 2006 meeting of the American Agricultural Economics Association (AAEA), the group consisted mainly of academic researchers but also included some private-sector users. They raised several concerns and made a number of useful observations about access to the ARMS data: NASS state offices vary with respect to computer and staff resources dedicated to ARMS research and the ability to host data users. Most are able to accommodate the specific needs of researchers with respect to their preferred time for visits or software needs, but some state offices require advance planning and scheduling of visits and are not very responsive during peak work activity periods. A suggestion for improving access was that ERS should create more comprehensive documentation of the content of the data files, so that researchers could better prepare before arriving at the state data center. Despite the relative convenience of the state centers, some researchers prefer to travel to Washington, DC, to access and work with the ARMS data at the ERS facility. This affords them special attention from knowledgeable ERS staff. Nonacademic and nongovernment researchers face particular difficulties in accessing ARMS data. Independent researchers cannot disaggregate the online data: they do not have access to the restricted data, and they often do not have the resources to request ERS staff to run a study for their purposes. Census Bureau Research Data Center Program An alternative means of allowing researchers to access federal government survey micro records is offered by the Census Bureau through its

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey research data centers (RDCs). Incorporating access to ARMS microdata in the RDCs would expand the number of points of access, and it could facilitate the linkage of ARMS data with other data sets, which could add value to the models that are built. The Census Bureau operates the RDCs under the authority of the secretary of commerce to grant special sworn status, which allows individuals outside the Census Bureau access for tasks that benefit the Census Bureau’s mandated programs. This “benefit” test means that users must go through a number of steps, including working with an RDC administrator, and submit an online proposal, which is reviewed on scientific merit, benefit to the Census Bureau, disclosure risk, policy sensitivities, feasibility, and proof of a clear need for nonpublic data. These proposals are reviewed by both Census Bureau and external (academic) reviewers. If another agency developed or sponsored the data, necessary proposals are also required to be forwarded to that agency for review. If the proposal is approved after these steps, all researchers on a project must undergo a background check, provide a Social Security number, sign an oath that they will preserve confidentiality, and take data and information technology security training. A recent decision by the Census Bureau leadership to consider allowing non–Census Bureau data at RDCs has made them a potentially attractive alternative to other agencies that now maintain their own data access programs. At the time this report was being prepared, the Census Bureau had begun to house confidential data at the RDCs for the National Center for Health Statistics and to work out arrangements for others to house their confidential data at the RDCs. An agency owning data housed in an RDC would be responsible for proposal review and setting disclosure standards but would use the RDC infrastructure, including online proposal capture, computer labs, and management capacity at the data centers. In return, the agencies would contribute to the cost of maintaining, upgrading, and expanding the RDC infrastructure. Recommendation 8.2: USDA should consider extending the availability of ARMS microdata through the Census Bureau research data centers to increase access opportunities for using additional data sets and enabling researchers to match ARMS files with other data sets. TRAINING FOR DATA USERS ERS provides data users with information, documentation, and training through the Internet and a help desk operation. In the online ARMS briefing room, interested users can learn about the survey and how to apply for access to the microdata. Users can use the ARMS web tool to generate custom data summaries and obtain downloadable copies of all survey questionnaires. Users seeking access to the microdata are directed

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey to contact the ERS Survey and Data Coordinator who provides help desk assistance, including relevant questionnaires, electronic copies of respondent booklets, NASS and ERS variable listings, format listings, summary programs, and artificially generated nonsurvey data for use in optimizing computer programs before visiting the state office or the ERS office in Washington, DC. ERS provides initial statistical guidance to microdata users. Researchers are assumed to have adequate statistical and economic modeling skills to accomplish their projects, including sufficient SAS abilities. ERS sends copies of reports on variance estimation under the complex sample design in ARMS and offers samples of SAS programs that illustrate jackknife techniques and regressions. ERS also provides on-site advice and support to microdata users who wish to come to ERS for a few days of hands-on learning and experience. ERS continues to explore ways to enhance access to ARMS data and training for users. A recent cooperative agreement between ERS and the National Opinion Research Center seeks to develop training guidelines for users on data security and on ways to use ARMS. ERS also organized a preconference workshop as part of the 2006 AAEA annual meeting entitled “The State-of-the Art in the Analysis of Survey Data with Complex Sample Designs.” The objective of the workshop was to illuminate the underlying statistical issues in analyzing data with complex sample designs. Many university researchers are interested in using the ARMS data for research but have not done so because of the steep learning curve and the travel costs associated with accessing the microdata. Training programs targeted at new data users and online tools for preliminary analyses are likely to be effective in attracting more data users. Because of the complexity of the survey design and the generally used estimation procedures, new data users will benefit from additional information and training to reduce startup costs of using the ARMS data. At the panel’s session for data users at the 2006 AAEA meeting, discussions among the existing data users revealed that the documentation related to ARMS is generally helpful and adequate for most purposes. However, it also became clear that many data users are not fully aware of all available documentation and resources, mostly because the materials are delivered on a case-by-case basis and because researchers visit NASS state offices without direct contact with other data users or ERS staff. Therefore, it would be helpful for the existing information to be packaged into a data user manual and delivered through training programs. The panel thinks that the complexity of ARMS access requirements and the data themselves, combined with the high costs and steep learning curve involved, especially for first-time users, have seriously impaired the value of ARMS as a research tool.

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey Recommendation 8.3: ERS should provide more training for new data users, including developing a data user manual, which also includes the recommended guide on statistical estimation (see Recommendation 7.5), and offering training workshops. USER FORUMS AND FEEDBACK In the ARMS briefing room, users can submit an open-ended feedback form to the ARMS team. The site also includes a few quick links through which users can discuss specific issues, such as ARMS content issues, and can request special tabulations or access to the online tools or the microdata. Recommendations for specific content to include in the survey and requests for special tabulations go directly to the data and survey coordinator in ERS. Users can also sign up to receive the ARMS update newsletter, which is also available through the briefing room. Since the feedback options available through the briefing room go directly to ERS staff, they do not allow users to actively discuss issues with one another. At the panel’s data user session at the AAEA’s 2006 annual meeting, users and potential users of ARMS data expressed interest in developing a process to facilitate communication within the user community as well as an ongoing feedback process to NASS and ERS. One idea is to schedule regular data-user meetings, in which researchers exchange ideas among themselves and offer feedback to ERS. Another idea is for ERS to help launch a “wiki” or some other type of interactive online documentation for ARMS, in which researchers themselves post ARMS-related information. This would also allow users to share programs, tips, and ideas about the ARMS data. DATA CONSISTENCY Although the panel has found no reason to doubt that ARMS data are equal in quality to those of other federal statistical agencies, occasionally clear errors have been found in the data by users, despite the review at NASS, ERS, and the state offices. A formal revision policy would encourage users to report outliers and data inconsistencies. As data files are updated, it will be useful to keep records of data corrections and updates. In order to access and process summary and microdata in electronic form, users must use codes that identify individual variables, and these codes change over time (see Table 8-1). Changes in questions over time are also another serious complication for users. A problem many of them face is that the questionnaires change over time and the questions change between versions—an issue discussed in this report. Table 8-1 provides examples of codes for the variables included in the debt section of the survey.

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey For these variables, there is a high level of consistency in the questions included in the survey every year, but the codes change and sometimes change back. NASS and ERS should develop a variable naming convention that does not permit code changes unless underlying concepts change. Although any arbitrary set of codes could be used by NASS and ERS in developing the data, but these should be mapped into a stable system of nomenclature. NASS and ERS have a very strong comparative advantage in judging the comparability of variables over time, particularly since they need such information to perform their own work. Like the variables themselves, the summary statistics are not consistent over the years. ERS should make the names and definitions of the summary statistics as consistent over the years as possible. This can be solved for future years by a policy of maintaining consistency in the names and definitions of summary statistics. In addition, ERS would be providing a TABLE 8-1 Examples of Codes for Debt Variables, 1996-2005 Year Lender Type Loan Balance Interest Rate Loan Term Year Obtained 1996 R611 R615 R619 R623 R627 1997 R938 R939 R940 R941 R942 1998 R931 R932 R933 R934 R935 1999 R1001 R1002 R1003 R1004 R1005 2000 R1001 R1002 R1003 R1004 R1005 2001 R1001 R1002 R1003 R1004 R1005 2002 R1161 R1162 R1163 R1164 R1165 2003 R1001 R1002 R1003   R1005 2004 R1001 R1002 R1003   R1005 2005 R1001 R1002 R1003 R1008 R1005

OCR for page 150
Understanding American Agriculture: Challenges for the Agricultural Resource Management Survey service to its users by revisiting the definition and naming of summary statistics from previous years to make them consistent with the names used in the most recent years. Only in this manner can a consistent time series for the summary series be ensured. In addition, if possible, the introduction of new aggregate or categorical variables (such as the new “combined farm” typology) should be accompanied by the revision of older data sets to include those variables. At a minimum, ERS should, when possible, provide computer code to users that provides for an intertemporally consistent definition of the key variables they publish. Recommendation 8.4: Database management practices should include a system for managing and reporting errors found by users for consistent labeling of the codes for raw variables, and for using the consistent names of the ERS-created summary variables over time. Percent for Farm Use Loan Guarantee Type of Loan Purpose of Loan Number of Other Loans Balance on Other Loans R631 R639 R635   R643 R644 R943 R944     R966 R967 R936 R937     R959 R960 R1006 R1007         R1006 R1007     R1036 R1037 R1006     R1007 R1036 R1037 R1166     R1167 R1196 R1197 R1006 R1008 R1009 R1004 R1007 R1046 R1047 R1006 R1008 R1009 R1004 R1007 R1046 R1047 R1006   R1004 R1007 R1046 R1047