The National Agricultural Statistics Service (NASS) and Economic Research Service (ERS) of the U.S. Department of Agriculture (USDA) publish statistics and reports that regularly and extensively detail the number of farms in the United States, the quantities and types of outputs they produce, the incomes of both farm businesses and the farm households that run them, and the status and conditions of the agricultural economy. The USDA’s data collection programs entail significant investments of the agency’s staff time, staff talent, and budget resources.1 Equally important, data collection is costly in terms of time and burden to survey respondents. To justify these investments, surveys and other data collection instruments must succeed at fulfilling a range of demands, from legislative and programmatic requirements to research, policy, and general user-community needs. In this chapter, we attempt to provide some appreciation of the complexity of the measurement tasks faced by NASS and ERS by describing in detail their current statistical programs and data infrastructure. A major component of this complexity involves conceptualizing in a consistent way the key productive units in farming activities that need to be measured.
As described in Chapter 1, agricultural production has been shifting to large farms in recent decades. And as this production has become concentrated among a relatively small number of large farms, the characteristics of farms and farming have become more complex in ownership structure and operational and management norms as well as the way farms are
1 NASS and ERS budgets for fiscal year 2017 were $171 million and $87 million, respectively. Of course, the agencies engage in many other activities beyond data collection.
integrated with other sectors in the economy involved in food production and delivery. An example of this complexity is the way farms have shifted toward employing firms to provide services—for land preparation, planting, spraying, and post-harvest transporting—as inputs in production. Agricultural production therefore embodies not just what farmers do, but also the activities that they may hire out to separate businesses.2 Farming activities are also contracted to nonfarm firms. In addition, large corporate nonfarm businesses may also be engaged in farming.
These trends have heightened the challenge of accurately characterizing the nation’s farms and farmers, and their productive economic activities, and carry implications for data collection and statistical reporting by NASS and ERS. The way farms are defined in the data collection apparatus shapes the way information in the sampling frame for USDA surveys is updated and maintained and the way key individuals involved in farms and households are designated, including the determination of who the appropriate survey respondents should be. The capacity of NASS and ERS to accurately account for complex operations engaged in large-scale and often diverse activities directly affects the reliability of agricultural statistics.
This chapter reviews the information currently collected and published by NASS and ERS and details how and why data are collected and reported. The ways in which statistical products are used by researchers, policy makers, and farm owners and operators are also discussed.
NASS and ERS are two of the 13 principal statistical agencies of the federal government. With the stated mission to “provide timely, accurate, and useful statistics in service to U.S. agriculture,” NASS
conducts hundreds of surveys every year and prepares reports covering virtually every aspect of U.S. agriculture [including] production and supplies of food and fiber, prices paid and received by farmers, farm labor and wages, farm finances, chemical use, and changes in the demographics of U.S. producers [. . . ].”3
2 Later in the chapter, we move beyond lay definitions and provide statistical definitions of terms such as “farm” and “agricultural production.” Technically, agricultural production includes a list of activities/sectors specified in the national income product accounts. See also the Glossary in Annex 2.1.
Among the publications produced by NASS are six of USDA’s seven leading economic indicator reports,4 which are broadly used in agribusiness and market analyses, including for decision making by buyers and sellers of agricultural commodities.
The mission of ERS is to “anticipate trends and emerging issues in agriculture, food, the environment, and rural America and to conduct high-quality, objective economic research to inform and enhance public and private decision making.”5 The top-level subject areas covered by ERS include the following:6
- Agricultural economy: farm-sector performance and farm households’ well-being; farm size and concentration; market analysis, data, and projections on commodity supply, demand, and prices; and federal farm policies
- Food and nutrition: food security; food and nutrition assistance programs; food choices and health outcomes; food access and store proximity; food retailing and marketing; and food prices
- Food safety: societal benefits associated with reducing food safety risks; global trade implications and economic impacts of food hazards; and potential results of regulation versus industry decisions
- Global markets and trade: domestic and international markets; trade; and the U.S. food and agriculture sector’s performance in increasingly globalized markets
- Resources and environment: economic impacts of alternative conservation programs; efficacy of policies designed to protect the environment; challenges of climate change and water scarcity; and enhancing agricultural competitiveness through technology
- Rural economy: investments in rural communities and the capacity of rural economies to prosper in a changing global marketplace; demographic change and its impact on rural communities; and drivers of rural economic performance
As described in Chapter 1, the key users of the information produced by NASS and ERS are diverse, ranging from Congress, the White House, and federal, state, and local government agencies to agribusiness and other businesses (e.g., secondary food-related businesses), participants in commodity market transactions, researchers, industry groups, and the farmers and ranchers themselves.
Legal Mandates to NASS7
NASS collects data to meet multiple demands: to fulfill legislative mandates, to generate key inputs for principal economic indicators, to support administrative programs and strategic goals and projects, and to inform research and policy making. Most NASS surveys are not mandated, strictly speaking; rather, they are conducted under a delegation of authority from the President to fulfill USDA’s mission to provide “leadership on food, agriculture, natural resources, rural development, nutrition, and related issues based on public policy, the best available science, and effective management.”8 Finally, although mandated information—such as data on income, finances, farm production, and households’ well-being—drives only a small portion of USDA’s data collection, obtaining this information is nonetheless a crucial aspect of the NASS (and ERS) missions.9
The centerpiece of NASS’s mandated responsibilities is to administer the Census of Agriculture, discussed in detail in the next section. This action is required by law under the Census of Agriculture Act of 1997,10 which directed the Secretary of Agriculture to conduct a Census of Agriculture in 1998 and in every fifth year thereafter, covering the prior year.11 In turn,
7 Much of the material in this subsection is distilled from a memo (information about NASS Mandatory Data Items, June 2017), prepared by NASS staff for this study.
8 The Code of Federal Regulations, Title 7, Subtitle A, Part 2, Subpart K, § 2.68 describes the delegation of authority made by the Under Secretary for Research, Education, and Economics to the Administrator of NASS. Delegations of authority are in place to (i) prepare crop and livestock estimates and administer reporting programs, including estimates of production, supply, price, and other aspects of the U.S. agricultural economy; (ii) collect statistics, conduct enumerative and objective measurement surveys, and construct and maintain sampling frames and related activities; and (iii) prepare reports of the Agricultural Statistics Board covering official state and national estimates. Additional provisions are in place to ensure data security precautions to prevent disclosure of crop or livestock report information prior to the scheduled issuance time and to avoid disclosure of confidential data or information supplied by any person, firm, partnership, corporation, or association. Language in the delegation of authority provides further guidance for improving statistics, maintaining coordination with OMB and other federal agencies on statistical methods and techniques; cooperating and working with national and international institutions and other persons throughout the world in the performance of agricultural research; and carrying out a number of administrative functions. For details of the NASS delegation of authority, see https://www.law.cornell.edu/cfr/text/7/2.68.
9 “Non-mandated” but still crucial aspects of data collection and statistical production, in terms of the agency fulfilling its mission, are detailed later in this chapter.
10 For a description of Public Law 105-113, see https://www.gpo.gov/fdsys/pkg/PLAW105publ113/pdf/PLAW-105publ113.pdf.
11 Prior to 1997, the Census of Agriculture was conducted by the U.S. Census Bureau, often in conjunction with the Decennial Population Census—as was the case with the first Census of Agriculture in 1840. Later, the timing was adjusted so that the reference year would coincide with the economic censuses covering other sectors of the nation’s economy. NASS publishes a detailed procedural history of the Census of Agriculture, see https://www.agcensus.usda.gov/Publications/2012/Online_Resources/History/2012%20History%20Final%203.14.17.pdf.
anyone who receives the Census questionnaire is mandated by Title 7 of the U.S. Code to respond to the Census of Agriculture, even if they did not operate a farm in that year. In connection with the Census of Agriculture, the secretary may conduct any survey or other information collection and employ any sampling or other statistical method that he or she deems appropriate. The intent of the Census of Agriculture is to provide a complete count of U.S. farms and ranches and of the people who operate them at various levels of aggregation. The content of this census, however, is not specified in the law.
In addition to the Census of Agriculture, NASS is mandated to produce a series of reports for the following:12
- Cold Storage, which includes mandatory data collection items such as stocks of butter and stocks of cheddar cheese (Public Laws No. 106-532 and 107-171)13
- Dairy Products, which includes mandatory data on stocks of dry whey and stocks of nonfat dry milk (Public Laws No. 106-532 and 107-171)14
- Cotton Ginnings, which includes mandatory statistics and estimates of grades and staple length of cotton (13 U.S. Code § 41)15
- Cotton Supply and Price Data, which includes mandated information on market supply, demand, condition, and prices (Title 7 U.S. Code § 473b)16
- Cotton Acreage, with mandated information on the total estimated acreage of cotton planted and on estimated harvested acreage (Title 7 U.S. Code § 476)17
12 While NASS must produce these estimates, it is not mandatory for a selected operator to respond to most of these surveys.
13 For monthly and annual Cold Storage reports, see http://usda.mannlib.cornell.edu/MannUsda/viewDocumentInfo.do?documentID=1034 and http://usda.mannlib.cornell.edu/usda/nass/ColdStor//2010s/2016/ColdStor-02-23-2016.pdf.
14 For monthly and annual Dairy Products reports, see http://usda.mannlib.cornell.edu/usda/nass/DairProd//2010s/2016/DairProd-12-05-2016.pdf and http://usda.mannlib.cornell.edu/usda/nass/DairProd//2010s/2016/DairProd-04-05-2016.pdf.
15 For monthly and annual Cotton Ginnings reports, see http://usda.mannlib.cornell.edu/usda/current/CottGinn/CottGinn-01-12-2017.pdf and http://usda.mannlib.cornell.edu/usda/nass/CottGinnSu//2010s/2017/CottGinnSu-05-10-2017.pdf.
16 For Cotton Supply data, see http://usda.mannlib.cornell.edu/usda/nass/CropProdSu//2010s/2017/CropProdSu-01-12-2017.pdf and for Price Data, see http://usda.mannlib.cornell.edu/usda/nass/AgriPric//2010s/2016/AgriPric-12-29-2016.pdf.
17 For Prospective Plantings, see http://usda.mannlib.cornell.edu/usda/current/ProsPlan/ProsPlan-03-31-2016.pdf; for Acreage, see http://usda.mannlib.cornell.edu/usda/current/Acre/Acre-06-30-2016.pdf; and for the Crop Production Report, see http://usda.mannlib.cornell.edu/usda/current/CropProdSu/CropProdSu-01-12-2017.pdf.
- Peanut Processing, with mandated information on a wide range of statistics pertaining to peanuts and peanut-based products (Title 7 U.S. Code § 951)18
- Prices Received, with mandatory information on corn, wheat, and cotton prices (Title 7, Chapter 35A, Subchapter II, § 1441)19
- Cash Rents, with a mandated survey—conducted “not less frequently than once every other year” (Agricultural Act of 2014, Title II, § 2005)—of per-acre estimates of county average market dryland and cash rental rates for irrigated cropland and pastureland in all counties or equivalent subdivisions within each state that have 20,000 acres or more of cropland and pastureland (Food Security Act of 1985, Public Law 99-198, 99 Stat. 1504, amended through Public Law 113-75; § 1234(C)5b).20
Additionally, the Food, Agriculture, Conservation, and Trade Act of 1990 and the Food Quality Protection Act of 1996 require NASS to collect and publish annual data on field crop chemical use.21
Legal Mandates to ERS and for Its Agricultural Resource Management Survey
USDA is required by Congress, through both authorizing and appropriations legislation, to produce statistics on a range of topics, many of which are estimated using data collected through the Agricultural Resource Management Survey (ARMS), which is jointly conducted by NASS and ERS. ARMS is the “primary source of information on the financial condition, production practices, and resource use of America’s farm businesses and the economic well-being of America’s farm households.”22
The Food and Agriculture Act of 1977 mandated that ERS, through the Secretary of Agriculture, report annually on trends in family farms—which ERS met primarily by collecting data from principal farm opera
18 For Peanut Processing, see: http://usda.mannlib.cornell.edu/usda/current/PeanStocPr/PeanStocPr-12-30-2016.pdf.
19 For Prices Received, see http://usda.mannlib.cornell.edu/usda/current/AgriPric/AgriPric05-31-2017.pdf.
20 For Cash Rents, see https://www.nass.usda.gov/Surveys/Guide_to_NASS_Surveys/Cash_Rents_by_County.
21 For the Agricultural Chemical Usage Field Crops Summary, see http://usda.mannlib.cornell.edu/MannUsda/viewDocumentInfo.do?documentID=1560 and the Agricultural Chemical Usage Restricted Use Pesticide Summary, see http://usda.mannlib.cornell.edu/MannUsda/viewDocumentInfo.do?documentID=1572.
tor households in ARMS—and produce comprehensive national and state-by-state data on nonfamily farm operations. Although that mandate expired, recent Family Farm Reports are still produced on a periodic basis (they have been published annually since 2014).23 The reports draw on ARMS data to illuminate a host of relationships, including: (i) farm participation in agricultural programs and the distribution of farm program payments; (ii) the structure and organization of farms, including family and nonfamily ownership; (iii) the use of new production technologies and other management practices; (iv) farm use of credit; (v) farmers’ participation in off-farm employment; and (vi) the characteristics of producers purchasing crop insurance (National Research Council, 2008, p. 17). Income estimates are designed to be consistent with the household income definitions used in the reporting of all U.S. households for most Census Bureau data series.
USDA’s annual sector estimates and forecasts of net cash farm income and net farm income are in turn used by the Bureau of Economic Analysis (BEA) in the construction of its national, regional, and industry economic accounts. The USDA income statistics include detailed data on value added, cash receipts and value of production, government payments, and farm production expenses. The statistics are based on responses to ARMS as well as sector-level information provided by NASS, the Farm Service Agency, the Risk Management Agency, and other administrative data sources. Data for more than 200 components of farm income are provided to BEA and used in deriving farm sector GDP and personal income both for the United States and by state. In addition, the primary source for BEA’s estimates of farm output—as used in their input-output accounts—is “cash receipts from farm marketings” by commodity as produced by USDA.
ERS is also mandated to publish cost-of-production information for a number of commodities. U.S. Code states that the
Secretary of Agriculture, in cooperation with the land grant colleges, commodity organizations, general farm organizations, and individual farmers, shall conduct a cost of production study of the wheat, feed grain, cotton, and dairy commodities under the various production practices and establish a current national weighted average cost of production. This study shall be updated annually and shall include all typical variable costs, including interest costs, a return on fixed costs, and a return for management (U.S. Code, Title 7).
ARMS data are the key input into the annual cost-of-production estimates and also provide baseline estimates for the years in which specific commodities are targeted.
Production input data collected through ARMS are also used to generate annual weights for the Prices Paid by Farmers Index, computed by NASS. This index, which indicates the average costs of inputs purchased by farmers and ranchers to produce agricultural commodities, is mandated by the 1933 Agricultural Adjustment Act. “Parity prices,” generally calculated as national averages, are used in administering federal marketing orders for 45 categories of fruits, vegetables, and nuts. The 1978 Public Range Improvement Act stipulates that these price indexes are also to be used by the U.S. Bureau of Land Management and the U.S. Forest Service in the calculation of annual federal grazing fees on the nation’s western public lands (National Research Council, 2008, p. 18).
How Legislative Mandates Shape Data Collection Strategies at NASS and ERS
As described above, most data collections are conducted by NASS and ERS under a delegation of authority from the President—which stipulates broadly the kinds of information needed for USDA missions—or to meet important research, policy, or general stakeholder demands for information. Relatively little data collection is driven directly by and with specific instructions from legislative mandates.
The legislative mandates that NASS and ERS are required to fulfill typically specify the types of information required but not how the information should be generated.24 This means the agencies have considerable latitude in how they fulfill these mandates. In many cases, USDA is required to collect a particular datum, such as on pesticide use, and to publish statistics about it, but there is no stipulation that it be collected through a particular kind of survey or by using a particular source of administrative information. For example, the mandate to “report on the financial health of the farm sector” could be handled in a number of different ways. Currently, ARMS data, including the production expense and the “farm-related income” line items, are used. However, administrative data are also used—from programs involving government payments (Farm Service Agency), commodity insurance indemnity payments (Risk Management Agency), commodity loans (Commodity Credit Corporation), and cash receipts (NASS production and price data sourced from Quick Stats).
Likewise, Congress mandates that data be produced for national income accounts for the sector, such as on household incomes and commodity costs, but the mandates do not specify that this information must be obtained using surveys. USDA’s interpretation is that ARMS is the best mechanism for fulfilling the core of this mandate. Although ERS relies on
24 This is the case for most of the principal statistical agencies in the federal government.
a single year of ARMS data to set the baseline cost and returns measure, subsequent annual updates are made using non-ARMS data on input prices by commodity. And, because ERS is mandated to report income received by the entire sector, data must be collected from all types of operations, including complex operations.
In other cases, Congressional mandates directed to USDA do not specify the “type of information” to be produced, but only that the collection take place. Most notably, the content of the Census of Agriculture is not specified in the law; the mandate is only that this census be conducted.
CONCLUSION 2.1: The mandated responsibilities faced by the National Agricultural Statistics Service (NASS) and the Economic Research Service (ERS) allow for considerable latitude in how data are collected from farms and how statistics on their activities and finances are produced. For this reason, the panel’s recommendations for improving or streamlining the Census of Agriculture and Agricultural Resource Management Survey (see Chapter 5)—in terms of content, questionnaire structure, burden reduction, and design elements to facilitate easier and more accurate linkages with nonsurvey sources—would not hinder NASS and ERS from fulfilling their mandates. Meanwhile, the recommended actions are intended to improve the agencies’ capacities to fulfill their missions to provide the data and statistics needed for policy, research, and other stakeholder requirements.
USDA also has considerable flexibility to explore nonsurvey sources of data, an approach that has improved a number of its data products. Mandates to NASS and ERS generally do not constrain the use of multiple kinds of data (administrative, commercial, Web-based, etc.) that could complement or, possibly, substitute for elements of the current survey-centric approach. Indeed, given the types of information about the sector that have value, expanding the breadth and diversity of the data sources drawn from represents a natural evolution for statistical agencies. Across these agencies, the trend in fulfilling information demands is toward greater use of administrative data. Canadian and European statistical offices are leaders in this regard (Prewitt, 2010, pp. 11–12; National Academies of Sciences, Engineering, and Medicine, 2017a). At the same time, agencies within USDA have also demonstrated the value of using administrative data for statistical purposes, while also documenting the difficulties of gaining access to such data (Prell et al., 2009), as is discussed in detail in Chapter 6.
Finally, although the legislative mandates to NASS and ERS are not especially constraining, ideally their content would be revisited and evaluated for relevance to the contemporary agriculture sector. In some cases, the underlying rationale for certain mandates may no longer apply, which
means it is possible that resources are being unnecessarily diverted away from more pressing information needs. For example, some mandates have their origins in the Great Depression, when most rural households earned their livelihoods through agriculture and when there was a large gap between rural and urban household incomes. Neither of those conditions holds true today for the majority of farm households.
Any consideration of changes to data collection should take stock of the reporting requirements faced by the USDA. Articulating data needs can be challenging: almost any piece of data could be described by a user or group of users as “needed” or “essential,” and many kinds of data will have value to some individual or organization. However, data collection carries with it a public cost, and it should therefore create public benefits. Here, and throughout this report, we highlight the types of USDA data for which there is a strong justification that they be publicly provided. Key data uses include the following:
- Program administration: Data are needed to enable agencies to run programs as effectively and efficiently as possible. Example: Understanding how crop insurance premiums affect crop insurance enrollment.25
- Policy analysis: Data are needed to evaluate whether policies and programs are affecting the right people and having the desired effects. Data are needed to evaluate how policies and programs operate in practice, including how the distribution of impacts across different groups is affected. Example: Identifying what types of farms receive the most farm payments.26
- Research on agriculture, health, food, and environmental concerns: Data can be used to simulate how the promotion of particular farm practices can affect environmental quality in a region. Example: Managing the costs of reducing agriculture’s footprint in the Chesapeake Bay.27
- Informing markets: Information is essential for improving the workings of markets. Example: Reports on planting intentions for particular crops.28
Some kinds of data—such as price data—may be best collected from markets themselves rather than from producers. Even in these cases, USDA can play an important role as an aggregator or clearing house for such data.
Reports and data products from NASS and ERS often serve one or more of the above purposes, and this report does not attempt to classify them. Instead, here, we simply provide a brief description of what and how the two agencies report data or analysis.29
NASS reports typically present totals (such as total chemical expenditures) and ratios of totals (such as total chemical expenditures / total number of farms) for the United States and for regions, states, and counties, as well as across types of farms. Its reports generally do not include distributional statistics, such as percentiles, or relationships between variables.
Once every 5 years, NASS publishes tables of statistics based on the Census of Agriculture. Annual surveys provide the basis for annual reports on topics such as farm production expenses (from the ARMS), agricultural chemical usage (from the Chemical Use Survey/ARMS), and cash rents and agricultural land values (from the June Area Survey). Other reports occur more frequently. The quarterly Farm Labor Survey provides estimates of employment and wages for workers employed directly by farms and ranches. It serves as the basis for the semiannual Farm Labor report.
Monthly or quarterly series provide timely information on production and prices, and some of these are considered principal economic indicators for various programs within the agency. A principal federal economic indicator is defined as a major statistical series that describes the current condition of the economy.30 Examples include the monthly Cattle on Feed report and the monthly Crop Production report. Two of the principal economic indicators reports only occur annually but provide prospective information on acreage planted or harvested, namely the Prospective Plantings report and the Acreage report.
ERS provides information and analysis that often goes beyond aggregate statistics such as production or expense totals. These include statistics to understand central tendencies, such as the typical number of hours worked off-farm, and diversity across groups, such as commodity program participation by demographic group. Analysis often involves farm-level
29 Detailed documentation of data products and reports is available from both NASS (see https://www.nass.usda.gov/Publications/catalog.pdf) and ERS (see https://www.ers.usda.gov/data-products).
30 Economic indicators are compiled, released, and periodically evaluated in accordance with procedures established in the Office of Management and Budget’s (OMB) Statistical Policy Directive No. 3. NASS provides OMB with its schedule of Principal Economic Indicator releases for the upcoming calendar year. If unforeseen circumstances make it necessary to change any scheduled release date after OMB issues the schedule, the agency must announce and explain the change as soon as it is known.
relationships between variables, such as how participation in crop insurance relates to access to credit. New questions are added to surveys in the effort to anticipate new trends, technologies, and policies.
ERS does much of its reporting by publishing data products, often as a time series. Examples include statistics on farm household characteristics and finances, on farm business finances, on commodity costs and returns estimates, and on the adoption of genetically engineered crops. Context for these statistics is provided through topic Web-pages associated with particular data products.
Another form of reporting is through agency reports that provide analyses of data, including tables and statistical results. Reports such as the Family Farm Report are repeated annually or every few years, but most reports are unique and emerge from a combination of stakeholder and researcher interest, data availability, and analysis of new policies, technology, or trends. Recent reports include the following:
- Changing Farm Structure and the Distribution of Farm Payments and Federal Crop Insurance (2012)
- USDA Microloans for Farmers: Participation Patterns and Effects of Outreach (2016)
- Farm Household Income Volatility: An Analysis Using Panel Data from a National Survey (2017)
- Federal Crop Insurance Options for Upland Cotton Farmers and Their Revenue Effects (2016)
- Changing Structure, Financial Risks, and Government Policy for the U.S. Dairy Industry (2016)
In addition to providing statistics through reports, both NASS and ERS allow Web users to query data for particular geographies or subject types. The NASS Quick Stats tool provides totals or ratios of totals from the Census of Agriculture or its various surveys.31 To give an example, users can use the online tool to access the corn acreage for a particular county and year or download corn acreage for all counties and years. ERS has a similar query tool for the ARMS data on farm structure and finances or crop production practices.32 Through these Web tools, users can find results for particular states or types of farms and view trends over time.
ERS and NASS provide an additional service to the research community by granting access (to individuals who clear an approval process) to ARMS microdata through a data enclave system, and to the Census of
Agriculture and other surveys through NASS Data Labs. Through data-use agreements, researchers outside the USDA can access farm-level data (that clears disclosure and confidentiality requirements) to perform analyses for reports or academic articles. This access has made possible hundreds of peer-reviewed academic publications on diverse topics ranging from agroenvironmental issues to farm finances to issues for beginning farmers.33
The primary data collection from farms is administered by NASS on behalf of USDA. NASS collaborates with ERS on the content and other design features for some surveys, including ARMS. As discussed above, the collection of farm data occurs both through the Census of Agriculture, which is conducted every 5 years, and the annual ARMS. The two instruments provide information widely used in reports by both NASS and ERS.
Census of Agriculture
The Census of Agriculture is a complete count of U.S. farms and ranches and the people who operate them. It is conducted once every 5 years by NASS to collect information on land use and ownership, operator characteristics, production practices, income, and expenditures. It provides the only source of uniform, comprehensive agricultural data for every county in the nation through time, showing historical changes in U.S. agriculture and long-term trends. The first agricultural Census was taken in 1840 as part of the Sixth Decennial Census of Population. For 156 years (1840–1996), the Census Bureau (and its predecessor, the Census Office) was responsible for collecting data for the Census of Agriculture. In 1997, responsibility for conducting this census was transferred to USDA.
The Census of Agriculture remained part of the Decennial Census through 1950, with separate mid-decade Censuses of Agriculture taken in 1925, 1935, and 1945. As time passed, the Census of Agriculture years were adjusted until the reference year coincided with the Economic Censuses covering other sectors of the nation’s economy. Currently, the Census of Agriculture is conducted for years ending in “2” and “7.”
The Census of Agriculture also collects information on the agricultural industry that may not be gathered elsewhere in the annual survey programs conducted by NASS, covering topics such as agritourism, organic production, farmer demographics, specialized agricultural production, Internet
33Moss, Featherstone, and Wilson (2012) found that a Google Scholar search (May 17, 2012) produced 1,290 documents for the period 2011 to 2012 using the terms “USDA ARMS data.” Using “anytime” as the period of time for the search, 18,200 documents appeared.
access, and more. However, the Census of Agriculture does not provide balance sheet information or information on household well-being. Several surveys that NASS has conducted following the Census of Agriculture cover on-farm energy production, farm and ranch irrigation, organic production, horticultural specialties, and local foods.
The Census of Agriculture defines a farm as a place from which $1,000 or more of agricultural products were produced and sold, or normally would have been sold, during the census year. Discovering new farms and properly accounting for existing and continuing farms and operators is an ongoing challenge. NASS accomplishes this through use of a list frame that covers the population of all farms and farm operators known to NASS and through an area frame based on land segments. It collects data from units in both sampling frames, targeting the full population, for the Census of Agriculture. For surveys, the sample depends on the target population, the commodity of interest, and so on.
In the list sampling frame that NASS uses, the sampling units are operations. Operators, defined as those who run farms—that is, make day-to-day management decisions—may receive a questionnaire for each operation they are involved in. Since the Census of Agriculture produces county estimates, such as for livestock or crop production, it also needs to attribute the agricultural production and the land of an operation to each county.
ARMS and Other Key Surveys34
ARMS is an annual cross-sectional survey that collects information on farms and farm households. The national survey is unique in that it collects, in a representative sample, (i) observations of field-level farm practices, (ii) information on the farm business, and (iii) characteristics of the household operating the farm.35 Responses to the survey are meant to provide estimates that are representative and reliable both at the national level and, for key states with the highest value of agricultural production, at the state level. Every year, farms producing a commodity (or commodities) of interest are oversampled and targeted with a commodity-specific version of the questionnaire.
ARMS occurs in phases, with initial screening of sampled farms occurring in Phase I. In Phase II, which samples roughly 4,000 to 10,000 farms, the survey collects information about a particular field, only from farms producing the crop being targeted in the survey year. The phase is dedicated to a detailed look at the production practices associated with the targeted commodity.
34 The discussion here is based on the ARMS documentation page, see https://www.ers.usda.gov/data-products/arms-farm-financial-and-crop-production-practices/documentation.
35 Alaska and Hawaii are not covered in this national survey.
In Phase III, for which roughly 35,000 farms are sampled each year, the survey collects farm financial and household information, as well as additional information on production practices, from all sampled farms. Sampled farms that are producing the targeted commodity receive a commodity-specific Phase III questionnaire, which asks questions unique to the commodity and permits a detailed assessment of the costs and returns associated with it. Sampled farms not producing the targeted commodity receive a more general version of the questionnaire.
The sampling unit for ARMS is the operation-operator pair from the sampling frame. Two operators of the same farm will not each receive an ARMS survey in the same year. Rather, only one operation-operator pair will be selected for a given operation. In addition, NASS employs techniques intended to help ensure that each operator only receives one ARMS questionnaire, even if she or he operates multiple farms. This contrasts with the design of the Census of Agriculture, in which a single operator may receive an additional questionnaire for each operation with which she or he is associated.
In years when the Census of Agriculture is carried out, ERS uses an integrated ARMS/Census questionnaire so that operators selected for ARMS fulfill their Census obligation by completing ARMS alone. In comparison to the Census of Agriculture, ARMS has questions that are more comprehensive, including balance-sheet information and farm-household income information.
NASS uses a stratified sample design for ARMS, in which the strata are defined by various farm characteristics such as commodity, farm sales class, and state. Larger farms are generally oversampled and small farms are undersampled. The oversampling of large farms can result in the same farm being sampled for ARMS two, three, or more times over a decade. Between 2000 and 2013, 16 percent of all sampled farms received the ARMS questionnaire more than once (Weber, Key, and O’Donoghue, 2016).
Other major annual surveys conducted by NASS include the June Area Survey and the Chemical Use Survey (there are many others as well). The June Area Survey collects information about agricultural land use, value, and rental rates in sampled land units, with the sampling unit typically being one square mile of land. Information is collected for all the land area in the sampled unit, with questionnaires filled out for each farm operating land in the sampled unit. For field crops, the Chemical Use Survey is embedded in the commodity-specific ARMS Phase II survey (fruit and vegetable chemical use are surveyed outside of ARMS II); questions appear as part of “Production Practices, Costs and Returns” (PPCR) field crop questionnaires administered by ERS and NASS. Information is collected regarding on-farm chemical use and pest management, including the area treated and rates of application of fertilizers and pesticides.
Key Concepts and Definitions Guiding Data Collection36
To produce meaningful information about the farming sector, there must be a common conceptualization of the basic measurement units among survey respondents and within statistical agencies. To this end, USDA has developed a set of related definitions for a farm, family farm, farm operator, and farm household. In addition, the sampling unit can be the operation, operator, field, or a combination of them, depending on the survey. Finally, the farm operation may also be involved with a variety of on-the-farm, off-the-farm, and value-added activities.
As mentioned earlier, the USDA defines a farm as “any place from which $1,000 or more of agricultural products were produced and sold, or normally would have been sold, during the year.” According to O’Donoghue et al. (2009), this definition was established by the Office of Management and Budget (OMB) and the Department of Commerce in 1975, suggesting that it is an administrative decision, not a definition set in statute. Note that the $1,000 value has not been adjusted for inflation over the subsequent period. Places with less than $1,000 in sales may also in some cases be considered farms based on their potential for sales, estimated using a points system that accounts for cropland or livestock assets. The idea is to continue to count these operations even if actual sales temporarily failed to meet the sales threshold due to bad weather, death of a household member, change in marketing strategies, and so on. Payments from government programs like the Conservation Reserve Program are considered sales when determining whether a place constitutes a farm.
There are practical (and political) reasons for maintaining the current definition of a farm in USDA publications. One of many examples is that 1890 universities (universities designated with land-grant status under the Morrill Act of 1890) receive funding partly based on the number of farms in a particular state. However, for collecting data with the purpose of measuring the activity and output of the nation’s farms, there are compelling reasons to identify farm businesses in alternative ways as well, based on how those businesses are organized. These alternatives are discussed in detail in Chapters 4 and 5. Questionnaires employed by the USDA use the word “operation” instead of “farm”—NASS generally treats the two terms as synonymous. The number of operations enumerated in the Census
36 Definitions in this section reflect current usage by NASS and ERS. Officially used definitions for many of the terms referenced here can be found in the USDA glossary, see https://www.ers.usda.gov/topics/farm-economy/farm-household-well-being/glossary.aspx. Additional discussion of key terms continues in Chapters 4 and 5 where, when noted, usage may differ.
of Agriculture, for example, determines the USDA’s estimate of the total number of farms in the United States and is highly dependent on the $1,000 sales (or potential sales) definition. ARMS and the Census of Agriculture collect information from the same universe of operations. However, a key difference is that ARMS focuses on an operation-operator pair, because it seeks information about a specific household associated with the operation. In practice, this means that a person managing three farms would receive three Census of Agriculture questionnaires (in the Census year) but only one ARMS questionnaire (if selected for ARMS in a given year), which would ask about the person’s household and about one of the farms that he or she operates.
How one operation is delineated from another is largely determined by the respondent. If a respondent’s notion of an operation meets the definition of a farm, it is considered a farm and enumerated as such. For example, a respondent could own and operate several poultry houses and also plant corn and soybeans. That respondent might choose to report the crop and livestock activities as one operation, or else as two. If he or she considers them two operations, the respondent would only be required to report one of them for ARMS. The Census of Agriculture is stronger in its guidance. The Report Form Guide for the 2012 Census of Agriculture (as cited by MacDonald, Hoppe, and Newton, 2018) instructs respondents to complete a separate report form for each distinct agricultural operation (farm, ranch, feedlot, greenhouse, etc.) for which separate records of operating expenses and sales, livestock, and crop acreage and production are normally maintained.
Certain farm-related activities and their economic quantities (that is, their assets or income) may be reported as part of the operation if they are not part of a separate business. In some cases—such as when activities underlying the income generation use farm assets and create farm costs, such that they are joint products—this is at least partly justifiable. In practice, activities are treated as separate businesses if they are separate from an accounting perspective—that is, if they maintain separate financial records.
A family farm is defined by the USDA as a farm in which “the majority of the business is owned by the operator and individuals related to the operator by blood, marriage, or adoption, including relatives that do not live in the operator household.”37 The business referenced is the same as the “operation” as defined in the questionnaire; the operator is the principal operator of the farm (as defined below). Thus, a farm is a family farm if
more than 50 percent of the farm’s assets are owned by the principal operator and any people related to him or her by blood, marriage, or adoption. This definition is similar to the definition of a family-owned business used by the Census Bureau in its Economic Census.38
“Farm Operator(s)” and “Principal Farm Operator”
The farm operator is defined by the USDA as the “person who runs the farm, making the day-to-day management decisions.”39 The respondent determines who is an operator, and a farm may have multiple operators, including a hired manager or partner(s). The only restriction on who is considered an operator is that the person must make day-to-day decisions for the operation. As such, anyone who considers themselves to be consistently involved in management of the farm could report themselves as operators. Operators are not required, by definition, to have an ownership stake in the farm or to spend a certain number of hours working on the farm.
If the operation has multiple operators, the respondent is asked to identify a principal operator; in other words, it is left to the respondent to define the principal operator. For surveys such as ARMS, ERS uses information on the principal operator to identify the household about which it will collect household demographic and financial information. In contrast, the 2017 Census of Agriculture has moved away from the principal operator concept, instead identifying up to four operators, now called “persons” involved in decisions. The 2017 ARMS also identifies up to four persons involved in specific decisions in the operation, but continues to identify a person who is most responsible for decisions as the principal operator and collects data for the associated household. A recent report recommended folding farm owners and decision makers into one group under the term producer—a recommendation that was adopted for the 2017 Census of Agriculture—to indicate any person involved in the business’s governance structure.40
38 The question used in the Economic Census to define a family business: “In 2012, did two or more members of the one family own the majority of this business? (Family refers to spouses, parents/guardians, children, siblings, or close relatives.)”
40 See Publication of Agriculture Census Data on Farm Operator Demographics (a report by the National Institute of Statistical Sciences Technical Expert Panel, October 12, 2017). There it was recommended to replace the label “Operator” with “Producer” in all publications. The 2017 Census of Agriculture and future censuses use these terms: “All producers,” “Principal Producers,” “Non-Principal Producers.” These terms span the breadth of agriculture and are seen as consistent with current terminology used by producers and by professional agricultural organizations.
USDA collects and reports information on the household of the principal operator of a family farm. It defines farm operator households as those who share dwelling units with the principal farm operators of family farms. According to ERS, multiple operators who do not share the same household operate less than 10 percent of family farms. Using this definition, the farm operator household population would include the households of the principal farm operators, but not the households of the other operators.
Household income for the principal operator refers to all income earned by members of the principal operator’s household, including both on-farm and off-farm income. Farm income includes net income from the farm operation that accrues to the principal operator household as well as farm-related income, such as income from renting out land to other farms. Nonfarm income is all other income, include that from wages, nonfarm businesses, and interest and dividends received by any member of the principal operator’s household.
ERS defines farm businesses as farms with annual gross cash farm income greater than $350,000, along with smaller operations where the principal operator has farming as his or her primary occupation. This categorization is mostly used in reports and statistics as opposed to survey instruments—that is, it is not a cut-off line used to guide data collection in surveys. This distinction is descriptively important given the skewed size distribution of farms, as discussed in Chapter 1. In its reporting of farm business income, ERS estimates that “farm businesses” represent less than one-half of U.S. farms but “contribute over 90 percent of the farm sector’s value of production and hold the majority of its assets and debt.”41 The farm business concept provides an example of how ERS has latitude in what it reports to fulfill mandates.
NASS defines a field as a continuous area of land devoted to one crop or a single land use, such as farmstead, pastureland, woods, or wasteland. For some data that are collected, the sampling unit within the operation can be a field. In ARMS Phase II, the respondent is asked to list all fields with the target commodity, and then the enumerator selects one field randomly
for the respondent to complete the questionnaire. Because of the design of ARMS, information collected about the field is linked to an operation and operator.
“Value-Added Activities and Products”
This phrase refers to the manufacturing processes that change the physical state or form of the product(s), increasing the value of the primary agricultural commodities produced on the farm, and to the final product(s) of those processes. As examples, USDA defines value-added products to include beef jerky; fruit jams, jelly, and preserves; and floral arrangements. In recent years, both the Census of Agriculture and ARMS have identified farms participating in value-added activities and collected information on the associated income or value of sales.
If a value-added activity, such as cheese making or grape processing, is not a distinct business, meaning that it is inseparable from the farm from an accounting perspective, then its economic values (assets, debt, costs, and revenues) are typically included in those of the farm being enumerated.42 If the value-added activity is a separate business, then only the income from it would be recorded in ARMS, where it would be recorded in the household section as income that the household earned from a nonfarm business. USDA reports how the operator has decided to run the value-added activities and whether the operator reports them as part of the farm or not. Typically, when these manufacturing or transportation value-added activities become sufficiently large they would be reported as a separate business, allowing for proper classification of value-added products.
From the National Income Product Accounts (NIPA) perspective, where the goal is to appropriately measure gross domestic product as the value of final products consumed, it may not matter whether value-added activities are classified as part of the farm or not, as long as there is no double- (or triple-, etc.) counting in the intermediate stages. For example, when a farmer produces wheat that is used by a miller to produce flour that is used by a baker to produce bread that is sold to a consumer, it is important to not double-count the value of the wheat and the value of the flour if milling is not part of the farm. It is worth noting that ERS also uses the Value Added Component Series, where the concept of value added has a different use, to denote the contribution of farm production (as opposed to food
42 In some cases, an effort is made to distinguish between the farm output and the value added from nonfarm production. For example, NASS’s 2015 Certified Organic Survey asks respondents reporting sales of roasted soy nuts to also estimate the value if they had instead sold the product as raw, unprocessed soybeans. In Section 11 (Item 4) of the survey, respondents are asked to record this value along with the Gross Certified Organic Value-Added Sales.
processing, packaging, food service, etc.) in the overall value added of all establishments that contribute to total food dollar purchases.
However, proper classification of raw farm product versus value-added product is still important in the North American Industry Classification System (NAICS). For example, grape vineyards (NAICS code 111332) are classified as agriculture (NAICS code 11), whereas wineries (NAICS code 31213) are classified as manufacturing (NAICS codes 31–33). To the extent that a farm does not report its wine-making activities and income as a separate winery business, those activities will be comingled with the farm, which may then be classified as a grape vineyard. Arbitrary reporting of value-added products as farm production is a central measurement problem, whether the goal is to measure the size of the agricultural sector, broadly defined, or to specifically measure “farm activities.” Note that this boundary issue is dealt with throughout the remainder of this report.
Challenges Raised by Current Data Collection Practices
The concepts and definitions that USDA uses, described in the previous section, are helpful in providing guidance to survey respondents. Yet some definitions give considerable flexibility to the respondent on how to report data, and this has implications for data accuracy, interpretability, and respondent burden.
Several key concepts on which data collection are based are vague and left to the interpretation of respondents, including “the operation,” “the principal operator,” and a “separate business.” Whether respondents consider (and report) their agricultural activities to belong to one farm or more than one is currently at their discretion. Respondents are also given little guidance on how to identify the principal operator, particularly in cases where multiple people each have primary responsibility over distinct aspects of the farm, such as management of marketing and management of crop production, as well as in cases of farms operated with spouses or in partnerships. Although guidance is provided to the portion of the sample visited by enumerators, this vagueness could create confusion for respondents, particularly in cases where there are multiple operators, managers, and owners, as is increasingly true of modern farming businesses. Interpreting the data that respondents provide also depends on assumptions about how respondents have understood the terms used in a survey.
Vagueness in the definition of a separate business is particularly important, as it influences whether various economic values are properly reported as part of the farm sector or not. As mentioned above, once value-added activities are identified as comprising a separate business they are excluded from outputs reported for the farm (while still included for the household).
Among the measurement goals of NASS and ERS is to obtain an accu-
rate measure of gross farm income, including income from value-added activities. It is challenging to get information on how farm and nonfarm activities are linked into a single business (this is discussed in greater detail in later chapters). In other cases, a firm may be coordinating economic activity on farms and realizing a share of the value added from their agriculture, but without operating any of the farms. In these latter cases, the question is, Should the aim be to survey and measure data from all farm firms?
USDA only reports on households containing the principal operator of a family farm. ARMS asks about the number of households that share in the net farm income from the farm but does not collect data on these additional households involved with the farm. All reported statistics, therefore, refer to the population of households containing a principal operator, not the full population of households containing any farm operator. Because a growing number of farms involve ownership, investment, and management by people from multiple households, the method of including only households with a principal operator will increasingly omit many households involved with a farm.
Moreover, it is unclear whether current practice best fulfills the mandate to report on the well-being of households of family farms in general. For example, as detailed in Chapter 5, redefining the farm household population as all households that include a person who is an operator of a family farm would permit capturing information about the financial health of households in a position to succeed a principal operator in the coming years.
CONCLUSION 2.2: When respondents are given a choice to decide the unit of measurement, such as which activities are included as part of the farm, who are the operators, or who is the principal operator, statistics on the number of farms, the size and scope of the farm sector, and the farm population are affected. Improving the clarity of definitions and requiring respondents to follow them would produce more accurate and interpretable estimates of the farm sector.
2.4. THE ESSENTIAL PERSPECTIVE OF DATA PROVIDERS: RESPONDENT BURDEN, RESPONSE RATES, AND DATA ACCURACY
An important aspect of assessing current practices is to consider how alterations would impact burden to respondents and, in turn, the accuracy of data and statistics produced by USDA. In this subsection, we consider each of these questions.
What Is the Level and Distribution of the Respondent Burden on Farm Operators?
Respondent burden can best be defined by the length of a survey questionnaire, the amount of effort required by the respondent, the amount of stress on the respondent, and the frequency with which the respondent is interviewed. Length is usually measured by the total time it takes to complete the questionnaire. Effort refers to the ease with which questions can be answered, particularly the need to consult records and the degree to which records are kept in categories that match those asked about in the questionnaire. Stress refers to the sensitivity of the questions and the degree to which they may evoke emotional reactions in the respondent. Frequency of interviewing is determined by the design of the survey (Bradburn, 1978).
The gatekeeper for the federal statistical system, the agency determining if and when data collection can proceed, is OMB. OMB measures the total burden of a given survey as the average time required for a respondent to answer the survey questions multiplied by the total number of respondents. The amount of time required to answer a survey is affected not only by the length of the interview or questionnaire but also by the difficulty for the respondent in reporting the requested data (McCarthy, Beckler, and Qualey, 2006, p. 97).
Annex 2.2., prepared for the panel by Hancock and Ott (2017), summarizes the annual total burden and the burden per contact for various surveys administered by NASS. The 2017 Census of Agriculture was estimated to have a total annual burden of 2,763,085 hours, which was distributed across 2.1 million farms (and, due to multiple operators in some cases, 3.2 million farmers). The numbers in Annex Table A2.2.2 indicate that NASS data collection accounts for only a small portion of the total public burden generated by USDA agencies.43
Relative to the Census of Agriculture, the total respondent burden from fielding ARMS is much smaller because it is imposed on a far smaller portion of the farm operator population. Its survey sample consists of approximately 30,000 farms and ranches selected from NASS’s list frame and area frame. However, the questionnaire contains more than 800 items for the respondent to potentially complete and for NASS to process after data collection; it is a very detailed survey.
In 2010, there were two primary versions of ARMS, known as the Core survey and the Cost and Returns Report. The Core survey was 16 pages long and took the average principal operator one hour and seven minutes
43 The USDA agency that imposes by far the most burden hours is the Food and Nutrition Service (FNS), which must collect information on a number of major programs, such as WIC and SNAP, and produce statistics on food distribution and child nutrition. Annex 2.2. to this chapter itemizes the burden figures for FNS.
to complete. The even more detailed version of the survey, the Cost and Returns Report, was 32 pages long that year and required an average of 1 hour and 36 minutes to complete (Weber and Clay, 2013, p. 757). These surveys create high levels of burden on selected individuals, not only due to their length but also as a result of the effort (including the process of checking records to identify a response) and the stress involved, such as when sensitive questions are asked.
Average burden figures, such as those emphasized in OMB evaluations, mask substantial variation across farm operations in the amount of effort required to comply with data requests. As described in Chapter 1, a relatively small number of large farms now account for the overwhelming majority of production for some commodities, including eggs, fed cattle, and some vegetable crops. Being involved in multiple activities, as many of these large farms are, leads to their experiencing an increased respondent burden. Also, certain categories of farmers may be contacted on numerous occasions, even on a yearly basis. For some surveys, the probability of a large operation being sampled is 1.0, or very close to it, for recurring surveys, especially with establishment surveys (McCarthy, Beckler, and Qualey, 2006).
A complex farm includes many working entities, information on each of which has to be reported. Additionally, large, complex operations may face an increased burden if the definition of the responding unit—that is, who should answer the questions—is unclear. With added mandatory reporting requirements for such things as pesticide use, fertilizers, and water quality, the total respondent burden of compliance requires significant resources to manage, especially for large operations. This burden does not originate solely from the USDA but stems from the requirements of other federal, state, and county departments and agencies as well. Indeed, relative to many state and county requirements, the mandatory Census of Agriculture represents only a small fraction, as a percentage of total reporting burden, in part because it is only required once every 5 years.44
44 For illustrative purposes, a farm in California may be asked to fill out the Agricultural Commissioners’ crop, production, and income reports for several counties; the California Water Resources Control Board’s Irrigated Lands Report for crops, nitrogen planned to apply, nitrogen actually applied, and erosion control plan; the Air Quality Management District report for tractor hours used, truck mileage used, and stationary and mobile engine hours used; the State of California’s pesticide-use report for every field; the State of California Environmental Reporting System’s hazardous materials inventory for each facility; the commodity purchaser’s report for pesticide use, water use, and production; the California Certified Organic Farmers’ Production report on inputs and income; the State of California Organic Program report on income; the Federal Food Safety Modernization Report; and numerous federal and state market news surveys.
How Does Burden Affect Respondent Cooperation and Data Accuracy?
Minimizing the burden placed on survey respondents is a matter of deep concern at statistical agencies, for several reasons. The most obvious reason is that people’s time is an economic good in its own right. As with all productive members of society, farmers’ time has high value. Thus, reducing respondent burden brings down the full cost of data collection. Another important motivation for reducing burden is provided by survey research45 which, while not definitive, suggests that increased burden can lessen survey cooperation. Reduced cooperation, in turn, can affect the robustness of findings and conclusions based on analyses of the resulting data. This is an especially pressing issue in today’s climate of declining survey response rates and increasing survey costs.
Declining response rates are a problem for surveys throughout the U.S. statistical system. Recently, even the mandatory Census of Agriculture has achieved only around an 80 percent rate.46 And roughly one-third of sampled farm operators ignore the ARMS survey entirely, an occurrence known as “unit nonresponse.” This leaves a unit response rate for ARMS that is well below the 80 percent level, which triggers the OMB, which monitors all federal information collection, to require the administering agency to conduct a nonresponse bias analysis (U.S. Office of Management and Budget, 2006, p. 8). Although low unit response rates do not always create significant nonresponse bias in the resulting statistics, the lower the rate the greater the effect on any derived estimates. That is, the lower the unit response rate, the greater the effect that any differences in answers between respondents and nonrespondents will have on estimates based on respondent-only data (Groves, 2006). Besides unit response, among responding units nonresponse to certain questions can be greater than 50 percent on the ARMS questionnaire (Miller and O’Connor, 2012).47 Such high item nonresponse may indicate that the questions are too difficult to answer, either because they require information to which respondents do not have access (for example, property taxes on rented land or contractor expenses), or because they are too complex or require detailed record checking, or because they are sensitive. Again, whether it is measured as length, effort, or stress, burden may be leading to missing data.
46 The response rates for the Census of Agriculture in 2012, 2007, and 2002 were 80.1 percent, 85.2 percent, and 88.0 percent, respectively. See https://www.agcensus.usda.gov/Publications/2012/Full_Report/Volume_1,_Chapter_1_US/usappxa.pdf.
Reasons Underlying Nonresponse
If greater cooperation with data requests is to be achieved, it is essential to understand why farm operators sometimes choose to not respond to agriculture surveys. Here, we identify several of the key factors.
Based on a limited research literature, one reason for nonresponse is a resistance to committing the time necessary to complete questionnaires.48 A study by O’Connor (1992) of the 1991 Farm Cost and Returns Survey found that the most common reason for noncooperation—accounting for one-quarter of all refusals—was that respondents “would not take the time/were too busy.” A study by Gerling, Tran, and Earp (2008) of the 2006 ARMS in the state of Washington reported the same top-level finding. Changes in questionnaire design and data collection modes have occurred after some of these studies, so evidence on the relationship between survey length and nonresponse cannot be assumed to apply uniformly over time or from survey to survey.
One factor affecting survey length is the ability to use information residing in other sources effectively. Often, respondents are called upon to provide the same information on multiple questionnaires. This problem sometimes surfaces because enumerators are not always able to tap into information from previous interviews. In part because data are confidential, situations arise in which information previously collected must be confirmed (re-collected) by enumerators. Information on the number of acres, for example, does not always prepopulate in NASS data sets, but possibly it could be configured to do so.49 Of course, confidentiality protections are needed, but statistical agencies are increasingly finding ways to automate surveys that prepopulate previously collected data. The Census Bureau’s management of the annual American Community Survey, which includes such capacity, serves as something of a model.
The reasons underlying respondent nonresponse, particularly item nonresponse, go beyond time burden. Some questions are problematic to answer for conceptual reasons, such as when it is not clear which entities, processes, or activities to include in responses; others may be ignored for practical, logistical, or cost reasons.
49 NASS is piloting the idea of doing just this for its Acreage and Production Survey.
As part of its information gathering, the panel met with a number of large farm producers to understand their farms and practices and the challenges they may have reporting on them. One conceptual hurdle highlighted by producers is how to match the structure of their farms with the categories and concepts used by the survey. Despite NASS’s well established and professional approach to collecting data, producers reported cases where answering questions was hard because of ambiguity over which entities, processes, or activities to include. More clarity about the definition of production units to be used for reporting would alleviate some of these difficulties. And, as outlined in Chapter 5, a Farm Register that followed the accounting structure used by the farm business could also help to demarcate entities that an individual respondent could better understand.
Farm operators also reported to the panel that surveys sometimes ask for data formatted in a way that does not align with the way information is recorded in their accounting or other record keeping systems. For example, one operator reported that its accounting software recorded combined fuel expenditures. The ARMS, however, asked respondents to differentiate gasoline from diesel fuel, so obtaining that level of detail would require a review of all of the fuel receipts. As another example, a question on the 2017 integrated Census of Agriculture/ARMS asked for data on tractors based on horsepower. One operator interviewed by the panel has more than 800 tractors, so the question would require him to go through the farm’s entire inventory to provide accurate numbers.
Flows of income for a given field may be another example requiring a difficult effort because, among other reasons, income from a crop cycle is not constrained within a single year; payment for a seed crop, for example, may not come in until 18 months after planting and therefore involves guesses about value. Another example is when there are two different products from a field. In such cases, the dividing lines for sorting inputs and linking them to outputs make reporting difficult and do not capture the economic processes, which are integrated.50 In contrast, questions about acreage are examples of information that is relatively straightforward to report.
In the above situations, operators may have little choice but to skip a question, provide a guess, or spend a large amount of effort preparing an answer—that is, to convert the records into the format requested by the survey. Ideally, the format of information requested by the survey would match the format of records as they are typically kept by farmers.
The trend of production moving to large operations composed of multiple enterprises, which require legal and contractual complexity, exacerbates
50 There are also issues in the timing of questions (i.e., how to allocate income when the crop overlaps the reporting year). In some instances, such as with crop futures, the actual income related to crops may not occur until some future year.
these challenges and, by extension, increases respondent burden. When a farm with multiple owners or managers overseeing a range of product lines is surveyed, this may necessitate the involvement of several people in information reporting tasks, which may be difficult to coordinate (Weber and Clay, 2013, p. 758).
In general, the cost accounting required appears to generate information that, for complex entities, is at best difficult to interpret and at worst simply inaccurate. Indeed, an earlier expert panel convened by the Committee on National Statistics (CNSTAT) (National Research Council, 2008, p. 80) recommended improving the understanding of respondents’ record-keeping practices in order to assess their effect on the quality of survey-collected data.
There is anecdotal evidence, echoed in the panel’s meetings with farmers, that certain types of questions are perceived as sensitive. Previous studies cited by McCarthy, Beckler, and Qualey (2006) of operators in North and South Dakota found that privacy concerns were a reported reason for refusals to participate, even though they were not the reason that was mentioned most frequently. Financial information can be seen as sensitive or proprietary and releasing it may be seen as risky.
Understanding which questions might be perceived as sensitive or difficult is an important task for statistical agencies. Identifying these questions during pretesting is an approach that allows agencies to consider whether they should be modified or asked at all. Such identifications are generally made through cognitive interviews, focus groups, and other pretesting methods. A previous CNSTAT panel (National Research Council, 2008, p. 80) recommended that NASS and ERS “test revised instruments before they are put into production, and use experimental control groups to evaluate the differences between the old and new questionnaires.” NASS responded, instituting a variety of testing methods for both the ARMS and the Census of Agriculture, including cognitive testing, focus groups, and split-sample field tests.51 It is important that testing continue and cover diverse types of farms, especially complex ones.
Among the reasons researchers found for nonresponse among farmers is that farmers are willing to complete other kinds of surveys “but not financial surveys,” and that “information [requested is] too personal / none of your business” (Gerling, Tran, and Earp, 2008). One large-operation producer, who described to the panel his own complex farm operation, cited several reasons why a respondent may be reluctant to participant in NASS surveys: (i) not wanting to reveal information about net worth; (ii)
the enormous amount of time commitment required; (iii) the need to create and maintain a separate database; (iv) perceived political agendas showing up in the surveys; and (v) failure to see what is in it for the respondent.
When combined with other reporting requirements—such as county pesticide reports, regional water quality reports, and all the other county, state, and federal forms—the Census of Agriculture and ARMS are perceived by respondents to generate high levels of burden to farm businesses. As reported by one of the operators visited by the panel, compliance with data requests requires not hours but many days of labor each year.
Concerning household surveys, there is speculation that one reason for survey nonresponse is survey fatigue, that is, persons receiving too many requests to complete surveys. This hypothesis is difficult to assess for household surveys, given the relatively low sampling rates for virtually all such surveys. Therefore, there is scant evidence to confirm or reject this explanation. One study of students found that the number of prior survey requests did predict nonresponse to a new survey (Porter and Whitcomb, 2005).
McCarthy, Beckler, and Qualey (2006) assess the impact of frequency on the success of subsequent requests to complete a new survey and note that 73 percent of farm operations in the NASS frame were never sampled for the surveys during the four-year period examined. Further, among the 27 percent of operations that were sampled, 72 percent were sampled for four or fewer surveys. The authors evaluate five measures of burden: the number of contacts; the number of surveys completed; the total length of all completed surveys; the number of days since previous contact; and previous participation in ARMS. They look at previous participants in ARMS separately from other surveys, since it is a particularly long survey. What they find is uneven evidence: for some survey requests, but not all, each measure was associated with nonresponse.
The Link Between Farm Size/Complexity, Burden, and Nonresponse
Large farm operations exhibit lower response rates to ARMS and Census questionnaires than do small farms. Whether the higher burden of reporting on large, complex operations is a factor in response rates is a key question of interest to statistical agencies. Weber and Clay (2013) address the size variable by examining the motivations and characteristics associated with unit nonresponse for the ARMS. As represented in Figure 2.1, they find that
response rates decrease monotonically with farm size . . . [and] even after controlling for other farm and household characteristics, farm operators
who do not respond have substantially greater sales than respondent operators, part of which reflects that completing the survey takes longer for operators of larger farms. (p. 756)
Descriptive statistics calculated for the study indicate that “refusal” farms have higher average sales ($902,327) than responding farms ($518,934) and own more land (906 acres compared to 627 acres). Because of the skewed distribution of farm output, with a small percentage of farms producing a high percentage of the total output, it is also useful to examine median figures. The median refusal operator harvested more than twice as many acres as the median respondent operator (307 acres compared to 132 acres), and the difference in sales was almost as large ($229,130 compared to $120,454). In “probability proportional to size” sampling, size would be associated with the number of times sampled, which makes it difficult to assess precisely why these associations occur. For example, it could be driven by the frequency of sampling, or it could be driven by the survey in question being more onerous to complete for large operations, due to the reasons noted above.
The results from Weber and Clay (2013) corroborate earlier findings from Earp et al. (2008a, 2008b). By comparing means of variables for ARMS respondents and nonrespondents who also responded to the 2002 Census of Agriculture, which is mandatory and captures higher response rates, Weber and Clay (2013) find that “the matched sample means for
sales, production expenses, and acres operated all exceeded those from the respondent-only sample, implying that nonrespondents have larger farms than respondents” (p. 757).
Even after conditioning on-farm and operator characteristics using a multivariate analysis, Weber and Clay (2013) find that “response propensities decrease in a perfectly monotonic fashion when going further out in the distribution of farm size” (p. 763). In their model, which accounts for the relationship between farm size (sales class) and complexity, respondents who received the least burdensome version of the survey, the Core survey, were indeed most likely to respond:
Conditional on many other variables, receiving the shorter Core version increases the propensity to respond by 0.035 [3.5 percent] relative to receiving the Version 1 Cost and Returns Report, which over the last three ARMS took an average of 30 minutes longer to complete than the Core. (p. 763)
If the differences in response rates are attributable to the length of the survey, the estimates “imply that decreasing the time required to complete the survey by one hour would increase the propensity to respond by 7 percent (or 3.5 percent per half hour)” (Weber and Clay, 2013, p. 763). Under this scenario, response burden accounts for 21 percent of the different response propensities between the smallest and largest farms; however, this would be an underestimate if the nonrespondent group consists disproportionately of large farm operators who anticipated that completing the questionnaire would take a long time (Weber and Clay, 2013, p. 763). As noted above, however, the time spent completing the questionnaire is not the only variable affecting response burden, so shortening the questionnaire without dealing with other issues of questionnaire clarity would not necessarily increase the response rate. Weber and Clay (2013) argue that
The time and disutility of responding will tend to increase with the size and complexity of the farm. Looking at ARMS response times over the last three years reveals that farm operators in the largest sales decile took about 55 percent longer (36 minutes) to complete the survey than operators of farms with no sales. (p. 758)
At the same time, it is unclear what aspects of complexity matter most for burden: Weber and Clay (2013) find that sole proprietorships were slightly less likely to respond to the ARMS, while farms using production contracts were more likely to respond. It is also unclear if all types of burden reduce respondents’ willingness to participate. Completing two shorter surveys six months apart may have a different effect from completing one long survey.
For example, McCarthy, Beckler, and Qualey (2006) produce results indicating that “burden (for example, the number of other NASS surveys operations were contacted for, the length of time since they were last contacted, and the type of information they were contacted for in the past) does not uniformly have a negative effect on survey response” (p. 97).
Finally, the form of the contact may play a role in response. NASS has already identified highly tailored strategies to recruit large operations to surveys like the ARMS. These strategies often include personal contact from NASS field staff who build relationships over time with the large operations. NASS might consider further tailoring contact strategies for small- and medium-sized operations. Thompson and Kaputa (2017) present the results of experiments with small- and medium-sized manufacturing establishments. They find that different contact strategies are more effective for operations of different sizes.
Impact of Burden on Data Quality
Greater respondent burden can reduce data quality by reducing the willingness of farmers to respond to any questions (resulting in unit nonresponse), or to particular questions (resulting in item nonresponse), or to give careful and accurate responses. Aside from reducing the effective sample size, an increase in the percentage of farmers refusing to answer any questions has several potential implications for data quality, including the following:
- Introducing bias into estimates of totals and ratios of totals (mean values). Such a bias is introduced if respondents differ from nonrespondents and there is no recalibration of sample weights. NASS researchers have used Census of Agriculture data on ARMS respondents and nonrespondents to assess bias in unconditional means of variables of interest. They find that NASS recalibration methods generally correct for nonresponse bias in the estimates of totals for many variables (Earp et al., 2008a; McCarthy et al., 2017).
- Introducing bias into estimates of conditional means. This bias appears if, for example, the relationship between operator education and crop yields is different for respondents and nonrespondents. In two examples that they examined, Weber and Clay (2013) did not find evidence of differing conditional means: “Despite the observed differences between respondents and nonrespondents, we find minimal nonresponse bias in the two econometric models estimated. The coefficients estimated from the respondent sample always fall inside the confidence intervals generated by repeatedly drawing from the full sample of respondents [provided by
- the Census of Agriculture] and nonrespondents” (p. 756). They conclude that nonresponse bias in ARMS is unlikely to undermine conclusions based on analysis of conditional means, but they also note that bias can vary from application to application.
- Introducing bias into estimates of percentiles. This too is a bias that can appear if respondents differ from nonrespondents in ways that are not addressed by re-weighting methods. NASS only reports totals and ratios of totals, but ERS regularly reports percentiles such as median farm household income. Neither agency has explored how current re-weighting methods affect estimates of percentiles. However, Robbins and White (2011) find that NASS imputation methods for direct and counter-cyclical payments dramatically understated payments to farms at the lower end (10th and 25th percentile) and upper end (90th percentile) of the payment distribution.
Farmers refusing to respond to particular questions can introduce similar issues, but responses to other questions can allow for imputation of missing values. At the same time, to the extent that an imputed value differs from the real value, imputation introduces a source of nonsampling error (National Research Council, 2008, p. 107). The error could increase dramatically as more respondents refuse to answer questions: greater nonresponse increases the need for imputation and, at the same time, reduces the accuracy of imputed values because less data is available to establish the statistical relationships on which imputation is based. Moreover, care must be taken to ensure that imputation methods do not lead to an understatement of variance or bias in distributional statistics (such as the percentage of farms with off-farm income) (Ahearn et al., 2011; Robbins and White, 2011).
Burden can also affect the quality of the data that farmers themselves provide. A farmer suffering from survey fatigue may begin to provide very rough numbers just to finish a survey sooner. This aspect of data quality is perhaps even more pernicious than explicit refusals of questions or of the survey itself, because it can be difficult, if not impossible, to distinguish between careful responses and rough guesses. Operators of complex farms in particular may find many questions inappropriate or simply unclear and become frustrated. For example, should the hired manager or the farm owner be listed as “the principal operator?” One panel member enumerated the ARMS with a farmer friend and found that mounting confusion over how to apply questions to his operation led to less concern for providing specific and accurate information.
This annex lists some of the key terms used in this report and in the measurement of the farm sector and broader agricultural sectors. Some of these terms are used in multiple ways, which can create confusion for individuals providing information or interpreting it. In the following table, the middle column reproduces definitions supplied by USDA—such as in the USDA/ERS glossary52—and as generally described in Chapter 2 of this report. The right-hand column includes some alternative definitions used in later chapters of this report to help describe more precisely some of the measurement approaches proposed by the panel.
|Terms||Official USDA Definition (used in this chapter)||Alternative Definitions (used in Chapters 3–6 of this report)|
|Farm||A place from which $1,000 or more of agricultural products were produced and sold, or normally would have been sold, during the census year.||An establishment (single unit with a legal or informal management structure) that (1) has its principal or secondary activity in farming, with the production of agricultural products and biological assets as seeds and animals; and (2) for which full economic data on key business variables, such as costs and revenues, can be collected and made available.|
|Farm business||A farm with an annual gross cash farm income of more than $350,000, or a smaller operation where the principal operator has farming as his primary occupation.||A collection of business establishments with at least one farm establishment linked by common ownership or control; this includes cases in which one business owns and operates one establishment (a simple farm business) or in which one business owns and operates a group of establishments (a complex farm business).|
|Farm operation||USDA treats terms “farm” and “farm operation” as synonymous.||Same as for “farm.”|
|Terms||Official USDA Definition (used in this chapter)||Alternative Definitions (used in Chapters 3–6 of this report)|
|Farm establishment||A business establishment engaged in farming.|
|Family farm||A farm in which “the majority of the business is owned by the operator and individuals related to the operator by blood, marriage, or adoption, including relatives that do not live in the operator household.”|
|Farming||Not defined.||The management of biological processes in crops or livestock.|
|Agriculture||The sector of the economy that includes both farming and agricultural support activities, as defined in NAICS.|
|Farm operator||The person who runs a farm, making the day-today management decisions.||The owner(s) of the business entity who are responsible for decisions made on the farm (by appointing managers if there are others), and who bear(s) all the financial risks.|
|Principal operator||Determined (defined) by the respondent in the ARMS; not used in the Census of Agriculture.|
|Producer||Replaces the term “operator.”|
|Farm household||Those who share dwelling units with principal farm operators of family farms; determined by survey respondents.|
|Place||USDA uses “place” in a nonstandard way. A “place” does not imply contiguous land parcels where ownership and management overlap.|
|Terms||Official USDA Definition (used in this chapter)||Alternative Definitions (used in Chapters 3–6 of this report)|
|Field||A continuous area of land devoted to one crop or land use, such as a farmstead, pastureland, woods, or wasteland.|
|TERMS FOR OUTPUTS|
|Farm Outputs/Products||Goods and services produced that fall under NAICS 111 (Crop production) and 112 (Animal production and aquaculture).|
|Agricultural Outputs/Production||A broader sector than farming that also includes agricultural services, many of which are found in NAICS 115 codes (Support Activities for Agriculture).|
Prepared for the panel by David Hancock and Kathleen Ott, NASS
For each data collection NASS conducts, a docket is submitted to OMB for approval to conduct the survey. The docket contains information on the survey sampling plan, sample size, data collection plans, questions asked, analysis plan, estimated number of burden hours that will be placed on the public, estimated number of contacts that will be made, and other relevant information about the survey. Table A2.2.1 shows the current OMB approved survey dockets for 2017 (as of December 2016) and an estimate of the number of burden hours and number of contacts that would be made.
Notes for reading Table A2.2.1. Each docket listed may contain multiple surveys. For example, the Agricultural Surveys Program docket contains the quarterly Crops Agricultural Production Surveys, the Quarterly Hogs Survey, the biannual Cattle Survey, the biannual Sheep and Goats Survey, and the June Area Survey. Surveys with no hours or contacts listed are currently inactive.
Neither column is the same as the number of operations sampled. Some of the surveys are conducted multiple times during the same year, so the number of contacts is much higher than the actual sample size for that survey (such as for the weekly crop weather and weekly broilers).
The number of operator contacts is an estimate of the number of times an operation will be contacted, given multiple contacts for a survey and estimates of nonresponse during each contact. For example, if a questionnaire is mailed out twice and nonrespondents are called by telephone, a respondent could be contacted one, two, or three times. Some of the contacts would not occur until 2017 (such as many of the Census of Agriculture contacts).
The number of burden hours is the estimated number of contacts multiplied by the average estimated time for each contact. The total number of estimated burden hours for all USDA surveys in 2017 was 211,851,887. The total number of estimated operator contacts for all USDA surveys in 2017 was 1,070,506,570.
The USDA agency with by far the most burden hours is the Food and Nutrition Service (FNS). FNS and two other larger-burden agencies are shown in Table A2.2.2 (not all USDA agencies are shown).
TABLE A2.2.1 Estimated Total Annual Burden Hours and Operator Contacts for Selected NASS Surveys, 2017
|OMB #||Docket Title||Annual Burden Hours||Annual Number of Operator Contacts|
|0535-0002||Field Crops Production||200,919||817,100|
|0535-0004||Egg, Chicken and Turkey Surveys||2,493||15,904|
|0535-0020||Milk and Milk Products||10,035||60,100|
|0535-0039||Fruits, Nuts and Specialty Crops||36,821||105,250|
|0535-0088||Field Crops Objective Yield||2,820||8,000|
|0535-0109||Agricultural Labor Survey||12,634||53,000|
|0535-0140||List Sampling Frame Survey||40,219||354,400|
|0535-0209||Supplemental Qualifications Statement||-||-|
|0535-0213||Agricultural Surveys Program||204,764||1,257,250|
|0535-0218||Agricultural Resource Management, Chemical Use, and Post Harvest Surveys||91,208||148,306|
|0535-0226||2017 Census of Agriculture||2,763,085||13,468,839|
|0535-0234||Farm and Ranch Irrigation Survey||-||-|
|0535-0235||Childhood Injury and Adult Occupational Injury Survey (NIOSH)||-||-|
|0535-0236||Census of Horticultural Specialties||-||-|
|0535-0237||Census of Aquaculture||-||-|
|0535-0243||Census of Ag - Content Testing||42,552||196,550|
|0535-0244||Nursery Production Survey and Nursery and Floriculture Chemical Use Survey||659||3,115|
|0535-0245||Conservation Effects Assessment Project (CEAP)||13,080||27,420|
|0535-0247||Distillers Grains Survey||-||-|
|0535-0248||Generic Clearance of Survey Improvement Projects||15,000||25,000|
|0535-0249||Organic Production Survey||13,004||44,032|
|0535-0251||Residue and Biomass Field Survey||64||220|
|0535-0252||Wheat and Barley Scab Survey||-||-|
|0535-0253||Pesticide Protection Equipment||-||-|
|0535-0254||Current Agricultural Industrial Reports||4,746||15,130|
|0535-0255||Colony Loss Survey||7,899||53,120|
|0535-0256||Feral Swine Survey||6,192||19,440|
|0535-0257||Organic Certifier Survey||885||55|
|0535-0258||Cost of Pollination Survey||14,987||78,000|
|0535-0259||Local Food Marketing Practices Survey||28,905||131,600|
TABLE A2.2.2 Total Annual Responses and Burden Hours for Surveys by Three USDA Agencies, 2017
|Number of Dockets||Total Number of Annual Responses||Annual Burden Hours|
|Food and Nutrition Service||79||847,066,971||106,736,781|
|Animal and Plant Health Inspection Service||135||147,424,507||7,530,873|
|Food Safety and Inspection Service||30||52,660,413||11,469,151|
SOURCE: National Agricultural Statistics Service.
This page intentionally left blank.