Economic policy making is an inescapable activity of government in a representative democracy with an economy grounded in free market principles. In a world of constantly evolving technology and information, most social institutions, including markets and property rights, are also changing. Many markets function well, but only government can provide the legal framework in which these markets exist. Changing technology leads to new markets, for which changes in this framework are required. Some markets do not function well because of the inherent characteristics of the goods and services traded in these markets. For these markets, too much or too little of the commodity will be produced or consumed unless government intervenes. In this environment, new research and information must constantly be brought to bear if economic policy is to be made wisely. The quality of economic policy decisions affects the welfare of the nation's individuals, and is an important factor in the competitive position of our nation with respect to others.
Government agencies charged with policy support responsibilities are some of the most important conduits from new research and information to public economic policy. Within the U.S. Department of Agriculture (USDA), the Economic Research Service (ERS) as did its predecessor, the Bureau of Agricultural Economics (BAE), from 1922 to 1953, provides much of the research and information in support of the department's economic policy mandate. In early 1997, ERS requested that the Committee on National Statistics of the National Research Council convene a panel to assess the management and structure of the ERS research program and produce a report of general principles for improving the quality and effectiveness of research in an intramural social science program that must serve agency program needs. The panel was also asked to examine
attendant issues, including relevance, timeliness, quality standards, and employing research by others, and to recommend changes in the management and structure of the research program.
In meeting its charge, the panel undertook several initiatives to understand the function of ERS and similar agencies in the policy making environment; to evaluate the research and information services of ERS with respect to quality, relevance, timeliness, and credibility; to study alternative organizations for providing research and information in the United States and other developed countries; and to evaluate the management of ERS. The panel studied the history of ERS since the establishment of the BAE in 1922, examined the economics of the supply of research, information, and analysis, and considered the scope for potential change in ERS and similar agencies. The panel carried on extensive discussions with current and former administrators of agencies; conferred with current administrators and senior staff at ERS; consulted with clients of ERS; read ERS research reports, staff analyses, and publications; reviewed the relevant theoretical and practical literature; and carried on extended internal discussions. The panel met five times between June 1997 and June 1998. This report reflects the program and organization of ERS as of early 1998.
This report presents the panel's findings, conclusions, and recommendations. Consistent with the charge to the panel, the recommendations address ERS, the Office of the Secretary of Agriculture, and the committees of Congress that set policy and appropriate funds for ERS. They also bear on other agencies providing the information and organizing the research and analysis that is necessary to inform public policy. The report sets forth a goal for informed public economic policy that the panel believes is attainable and recommends essential changes to realize this goal. This summary contains the most urgent recommendations; the complete set of recommendations is found in the body of the report.
The ERS and Informed Public Economic Policy
Since 1922 the BEA and ERS have provided research and information to support the policy mandate of USDA, which has evolved extensively in the intervening 77 years. The most immediate and visible category of service is staff analysis in response to questions from the Office of the Secretary, and often from the Office of the Chief Economist. Other requests come from USDA agencies, the Congress, the Office of Management and Budget, and the Council of Economic Advisers. In 1997, ERS utilized about 20 percent of the time of its staff of professional employees, most of whom are economists, responding to about 350 such requests.
The second category of service is the development of secondary data and analysis, often presented in the form of indicators and accounts, such as those found in USDA Situation and Outlook reports. This activity occupies about 40 percent of professional staff time. Indicators and accounts provide quantitative
summaries of economic activity in the food, fiber, and natural resource sectors of the economy, which account for about 16 percent of gross domestic product. Examples include agricultural trade trends and forecasts, farm financial status, food consumption and waste, marketing margins, and natural resource use in agriculture and associated indicators of environmental quality.
The final category of service is intermediate and long-term research related to the economic policy mandate of USDA. This research reflects the diversity of the economic policy mandate of USDA, including, for example, economic incentives for potential participants in the Conservation Reserve Program, evaluation of commodity procurement for food assistance programs, analysis of the economic impacts of proposed changes in tariffs on agricultural products, and cost-benefit analysis of conservation tillage.
In the past 20 years, the breadth of research, information, and analysis required in ERS has grown along with the policy mandate of USDA. Environmental and food safety issues, global warming, the consequences of international financial instability, and other issues have augmented the traditional concerns with production agriculture. Yet inflation-adjusted funding has decreased by one-sixth and staff by over 30 percent since 1992. The training of ERS professional staff reflects its traditional agenda more than it does the current policy mandate of USDA. Whereas once ERS was the dominant employer of agricultural economists with limited alternative career prospects, today ERS most compete to recruit and retain a much broader array of professionals, most of whom have many attractive alternatives. All of these factors, combined in a fractious political environment, have led to a widespread perception that the quantity and quality of ERS products are not what they should be, the reduced staff and budget of ERS perhaps notwithstanding. These reservations about ERS are becoming more acute, as ERS is asked to address an ever-widening range of serious economic issues. The recommendations in this report address these problems specifically, as well as general principles for improving the quality and effectiveness of agencies charged with producing research, information, and analysis in support of economic policy.
Evaluation as a Framework for Management
The principle of competitive supply applies to the provision of research and information in support of public economic policy, just as it does to other economic activities. The ultimate consumers of these services compare alternative sources and make choices. A successful provider, including ERS, must clearly identify the services it provides and the clients for its services. The successful provider must also understand who its competitors are or could be, and the attributes of the service that underlie the comparisons and choices of its clients. Like any successful enterprise in a competitive market, an agency providing research and information in support of public policy must continually evaluate it-
self retrospectively on these four dimensions: the services it provides, the set of current and potential providers of these services, existing and potential clients, and the attributes of services that underlie clients' comparisons and choices. Employing the evaluation process prospectively, such an agency should manage itself so as to achieve the most favorable ultimate evaluation.
On the basis of its examination, the panel concludes that there are four attributes of research, information, and analysis that matter to public policy makers and other public- and private-sector clients. First, research, information, and analyses should be of high quality, meeting relevant disciplinary and professional standards. Second, research and analyses should be relevant, addressing the essential policy question and with consideration of the policy context in which decisions are made. Third, these services should be timely. Intermediate and long-term research conducted in anticipation of policy questions and concluded before political lines are drawn is a treasured resource, not only for clearly being independent of specific interests, but also for its availability at critical junctures, when decisions must be based on what is known rather than what might be learned. Fourth, all of the services provided by agencies in support of public economic policy must be credible. The credibility of research, information, and analysis in support of public economic policy derives from its quality, relevance, and timeliness, and its established independence from the political decision making process.
Until recently, agencies that support public economic policy have not been evaluated formally with respect to these dimensions. The panel finds that there is little infrastructure for effective evaluation in ERS, including its responses to the requirements of the Government Performance and Results Act of 1996. For example, ERS administrators are unable to account for staff time and other resources on a project-by-project basis, even on a large scale.
The panel recommends that ERS systematically evaluate the services it provides. Formal program evaluation instruments should elicit from clients and potential clients their choices among alternative providers and potential providers of the services provided by ERS, and the attributes of the services critical to their choices, including prices. The instrument should solicit the identities of additional potential clients and alternative providers of these services. ERS should participate in the design of evaluation instruments, but their administration should be delegated to an independent party. The panel further recommends that ERS should allocate its costs and staff time across the same services used in its system of evaluation, according to generally accepted accounting principles.
In the long-term, ERS or any agency providing research and information in support of public economic policy must have the widest possible scope for the way it produces these services. In its review of these agencies and discussions
with their current and former administrators, the panel found a rich array of organizations, including long-term relationships with university-based research institutes and federally funded research and development centers; the use of grants, contracts, and cooperative agreements with individual and small groups of investigators; quasi-public agencies; mixtures of intramural and extramural research; and various forms of public-private partnerships; as well as intramural research and information programs carried out exclusively by permanent agency employees. These organizations reflect different methods for procuring the research and information that supports public policy. A successful agency most be able to choose among these methods of procurement over the long-term, and it must be free to make changes as its policy mandate and operating environment evolve. Specific choices should be reflected in agencies' strategic plans, but not their more durable mission statements.
The panel recommends that the mission of ERS should be to provide timely, relevant, and credible information and research of high quality to inform economic policy decision making in USDA, the executive and legislative branches of the federal government, and the private and public sectors generally. It should identify information and frame research questions that will enhance and improve economic policy decisions within the authority of the secretary of agriculture, organize the subsequent collection of information and conduct of research, and evaluate alternative approaches to policy problems. The work of ERS should address anticipated as well as current and continuing policy questions.
Administration of Research, Information, and Analysis
The principle that services should be supplied competitively is a fundamental premise of our economic system, including government procurement. This principle is recognized in the Competition in Contracting Act of 1984. A competitive supplier of research services, in particular, must constantly integrate new ideas in order to continue supplying those services. There is pressure for both individuals and organizations to reach beyond their immediate area of expertise to gain a competitive edge by bringing to bear new results from related fields. Suppliers of research services with permanent, sole-source awards have no such incentives to reach out and often become isolated within narrow fields using methods that are increasingly outdated.
The panel concludes that no organization should ever be given a permanent sole-source award for the provision of a service. Decisions to provide sole-source awards must be defended on a recurring basis, beginning from the presumption that services should be procured competitively.
The panel recommends that research and information in support of public economic policy should be procured competitively. All potential suppliers, including ERS, should be on the same competitive footing. If an outside supplier is selected as an awardee, in many cases ERS should have a secondary role as a partner in the provision of the service. No supplier, including ERS, should have a permanent, sole-source award for the provision of any service. Any decision to grant a sole-source award must be defended periodically.
The partnership role of ERS envisioned in this recommendation is important to the effective competitive procurement of research and information in support of public economic policy. It is essential that USDA ensure that the supplier has an understanding of the policy context for research and information and apprise contractors of developments in policy over the lifetime of the award. Intellectual command of the areas in which research is contracted, as well as an understanding of the universe of potential suppliers and their capabilities, are essential to making appropriate decisions about the procurement of research and information. In many cases, a partnership role for ERS may be the most effective way for USDA to ensure that it effectively addresses the critical choice of the best vendor for the research and information needed to support its policy decisions.
The panel recommends that choices among alternative vendors of research and information in support of public economic policy should be based on prospects for favorable evaluation of the services provided, as well as on the costs of the services. The critical attributes established in program evaluation provide the framework for choice among vendors. No single model of choice among vendors is appropriate for all programs. In particular, the methods used by the National Science Foundation, the National Institutes of Health, the Agricultural Research Service, and the National Research Initiative will not be suited to many ERS programs and should not be presumed to be appropriate to any.
Expanding the universe of potential suppliers of research and information is essential to the success of ERS given the breadth, growth, and changing character of the USDA economic policy mandate. It is unlikely that any permanent staff of professionals could, under foreseeable budgetary conditions, come close to meeting the needs for information and research in support of economic policy in USDA. The flexibility in choosing suppliers demanded by the principle of competitive procurement will also enable USDA to meet expectations of quality and quantity in the research, information, and analysis provided by ERS.
Intermediate and Long-Term Research
The credibility of its research has been troublesome throughout the history of the ERS and its predecessor agency, the BAE, as well as in other agencies providing research and information in support of public policy. Separating findings of fact from political considerations in a credible fashion is one of the most difficult tasks in the administration of research and information production. Identifying and studying relevant policy problems before they emerge as political issues enhances credibility, but achieving such success on a regular basis is not realistic given practical limitations on resources.
The most important practical consideration is to distance those who provide counsel to decision makers, like the secretary of agriculture, from those who carry out the research in support of their decisions. This fact has been demonstrated repeatedly in the history of the BAE and ERS, and it has been identified previously by others who have analyzed the problem.
The separation of the conduct of research from the making of policy has both administrative and organizational implications. With respect to administration, the most difficult problem is the need for research and information findings to be cleared by political appointees in government agencies. The panel finds that there is no effective substitute for the independence of research from the making of policy.
The panel recommends that USDA should support the integrity of its intermediate and long-term research programs in support of economic policy, while retaining the prerogative to disagree with research findings. These programs should be conducted with the clear objective that peer-reviewed research findings may be published by the investigators independently and without prior approval by USDA, and with the clear understanding that USDA does not necessarily endorse the findings of any research program.
If the independence of intermediate and long-term research conducted by ERS employees from the political process cannot be guaranteed in this way, then this research should be carried out by external vendors to whom these guarantees can be extended, as they are, for example, when agency-sponsored research is published in an academic journal with agency disclaimer. Peer reviews of research are always appropriate. The model of peer review, not prior clearance, will be more effective in obtaining the service of the best professionals (whether employees or external vendors), a step that is essential to a reputation for quality. The maintenance of the integrity of government-sponsored research is the responsibility of the research agency, its cabinet secretary, and the relevant committees of Congress.
Reports and Indicators
Both historically and currently, there has been consistent demand for the provision of secondary data With accompanying analyses by ERS in the public and private sector, and there has been more consistent political support for this function of ERS than for its intermediate and long-term research. These services are vital to USDA in providing detailed projections of supply, demand, and prices in order to estimate the budgetary and farm income implications of extensive price support programs. But these needs are changing. Many of these programs are being reduced under farm legislation enacted in 1996 and may or may not be further reduced even eliminated in 2002, the next farm legislation renewal date. At the same time, environmental and other regulations in agriculture are increasing, providing a renewed justification for some existing indicators and secondary data as well as creating new needs for USDA analysis.
The provision of price and production reports and indicators by USDA extends well back into the nineteenth century, predating commodity support programs, which began in the 1930s. The modern program of market and farm income outlook reports began in the 1920s. USDA reports and analysis were developed to provide farmers with market information similar to that available to purchasers of their commodities. Political support for the provision of secondary data, information, and accompanying analyses by ERS has been grounded in these considerations of equity. In the wake of the information revolution, valid arguments for the public collection and provision of primary data—including increasing returns to scale and the pure public good nature of these data—may or may not apply in the same way to secondary data and information. New public policy demands in areas like nutrition and food safety may also change government needs regarding indicators and secondary data. In light of all of these changes, the reports and indicators programs must be reexamined from first principles.
The panel recommends that the secondary data preparation and analysis programs of ERS should be evaluated within the framework outlined by the panel, including consultations with clients. On the basis of this evaluation, a long-term plan should be drawn up, including new and discontinued services. The plan should indicate which of the services provided will be produced in ERS, which will be procured from other vendors, and which will be left to the private sector. The plan should include anticipated impacts on clients and the projected impact on the USDA budget.
Staff analysis is the point of contact between the ERS research, information, and analysis programs and the policy decisions that these programs support. Close
contact between staff analysis leaders and policy makers is required to ensure that the entire ERS program remains relevant to the substantive economic policy mandate of USDA. Information that must be provided on a very short-term basis—often a few days or less—requires that those providing the information be available immediately. Highly political requests should go to the Office of the Secretary or the Office of the Chief Economist.
Staff analysts must also be closely involved in guiding the ERS research and information program, including the assessment of future policy questions, the framing of questions for investigation, and the organization and supervision of research, because staff analysts are the first line of contact with policy decisions. Thus, leadership in staff analysis requires a sophisticated combination of analytical and management skills. The important attributes of credibility and relevance in staff analysis and the need for leadership in staff analysis to oversee research and information programs indicate that this function must be provided by a permanent, skilled group of staff analysis within USDA.
The panel recommends that USDA should maintain a permanent core of staff analysts to provide immediate support for its economic policy decisions. The size and composition of this group should reflect the level of detail and timeliness required in support of the economic policy mandate of USDA, and it should be reviewed from time to time as the mandate evolves. The leadership of this group must provide a combination of management and analytical skills essential to the administration of the research and information programs of ERS. ERS should regularly invigorate this group by means of visiting scholars, sabbaticals, internships, or similar programs, to maintain the contact of staff analysts with the wider research community.
Organization and Placement
The mission of ERS and evaluation of the services it delivers drive the administration of research, information, and policy that we recommend. This administration of services will be effective in delivering research and information to policy makers only if it is embedded in organizations that support it, extending from USDA to the president and the Congress. As the history of this function in the USDA shows (see Chapters 3 and 4), this has not always been the case.
ERS within USDA Today
In the current organization of USDA, the Office of Chief Economist, situated within the Office of the Secretary, has primary responsibility for economic policy advice to the secretary. The chief economist is appointed to serve the secretary, has direct contact with the secretary in policy meetings, and has a small policy advisory staff of about eight professionals. The administrator of ERS, in contrast,
reports to the Under Secretary for Research, Education, and Economics (REE), along with the administrators of three much larger agencies—the Agricultural Research Service, which oversees largely biological research, the National Agricultural Statistical Service (NASS), which collects primary data, and the Cooperative State Research, Education, and Extension Service. NASS collects much of the primary data used by ERS, and ERS is one of the principal clients of NASS. It is a statistical agency, and most of the data it collects are economic. The REE undersecretary has no responsibility for economic policy and is not likely to be a social scientist. Many requests to ERS for staff analysis come from the chief economist.
Research and information in support of economic policy within USDA are not well served by these lines of authority. The administrator of ERS, with responsibility for over 300 professional employees, is several steps removed from the policy process to which the work of ERS must be relevant. The chief economist, charged with representing economic information in the decision making process, has no direct line of authority to the greatest concentration of talent in USDA for marshaling this information.
These lines of authority would not serve research and information in support of economic policy well under the model of competitive procurement of services by ERS advanced in this report, either. In the current organization, there is no position suited to deciding whether particular information and research services in support of economic policy should be procured from outside vendors, or, in the event that both ERS and outside vendors could supply services, whether or not ERS should be chosen. Reorganization of the economic policy support function within USDA should therefore be considered simultaneously with the question of how these research and information services are procured.
The principles for procuring information and research, the history of the BAE and ERS, and the experience of other cabinet-level agencies suggest a reorganization that copes with all of these problems, First, both economic policy decision making and research and information in support of economic policy should be brought into a single line of authority. This was the case for many years in the BAE and ERS, and it is true in many cabinet-level departments today. Second, consistent with the lessons learned from the history of the BAE and ERS and with the model for procurement of information and research services developed in this report, the functional separation between policy decisions, on one hand, and credible research of high quality in support of these decisions, on the other, should be clear and transparent.
The panel recommends that ERS should never be involved in recommending or deciding on specific policy actions, which are the prerogative of the secretary.
The panel recommends that a small, highly capable policy analysis and advisory group should be led by an appointee, such as a chief economist or an assistant secretary for economics, who manages day-to-day economic policy staff support for the Office of the Secretary. Such a unit would be appointed to serve the secretary and would provide any advice on political and policy action, keeping prescriptive advice and highly political matters from being directed to ERS. The administrators of the Economic Research Service and the National Agricultural Statistical Service should report to the chief economist or the assistant secretary for economics.
The Office of the Chief Economist or the Assistant Secretary for Economics, in this recommended organization, requires individuals with a thorough understanding of current and emerging policy issues and strong abilities in framing research questions. The Office of the Chief Economist or Assistant Secretary for Economics must be able to pose well-framed research request, that address their policy needs, while balancing timeliness, qualifications to do the work, and a sense of what is possible.
The professional staff in the Office of the Chief Economist or the Assistant Secretary for Economics, and not ERS, would be responsible for bringing the research and information services of ERS to bear in policy councils. They must therefore have a thorough command of the economics of policy questions, whether provided internally by ERS, through sponsored extramural research, or through syntheses of existing research and information. The same staff of the Office of the Chief Economist or the Assistant Secretary for Economies would be responsible for evaluating the program of research and information conducted externally and through ERS, for directing research and information projects, and for choosing vendors for research and information.
ERS, in this recommended organization, would have primary responsibility for the policy relevance of research programs in its role as primary or secondary provider, would be responsible for the administration of internal research and information projects, and would have a direct interest in maintaining programs that are competitive with alternatives in the public, private, and academic sectors. The administrator of ERS should be a professional, career economist, not subject to political appointment. He or she would be available to explain the research and information findings of ERS, as would external contractors, but should never be called on to represent the policy position of the secretary, the assistant secretary for economics, or the chief economist.
Adoption of the recommendations in this report will be effective only if there is agreement among senior policy makers on the principal points underlying them. These points include the nature of public economic policy and the desirability of informed rather than uninformed policy. In the production of information, research, and analysis to inform public economic policy, they include the principle of competition and the necessary attributes of quality, relevance, timeliness, and credibility.
The history of ERS amply demonstrates the vulnerability of an agency that informs policy decisions with credible and relevant information yet is not itself a political decision maker. Yet the same history indicates that this role is essential to success in informing policy decisions. The concept of such an agency is too fragile to sustain disparate expectations by the executive and legislative branches. It requires cooperation and agreement between the secretary and the relevant congressional leadership on a common set of expectations and rules for shared access to ERS services and the role and expected behavior of ERS in dealing with both branches of government. Only in such an environment will informed public economic policy survive.