Elements of Science Advice
To evaluate the quality and effectiveness of the science advice prepared and utilized by the UN and its component organizations, it is necessary to understand the meaning of science advice and how leading scientific organizations practice it. This material was collected by means described in the Preface and is here generalized so that it can be adapted and adjusted to the needs of specific users. It may be kept in mind as an aid in evaluating the science advice procedures and practices of the UN system, as described in Chapters 3 and 4.
Elements of science advice
There is usually a powerful consensus among working scientists regarding the current state of knowledge in any field, and there are established procedures that have the confidence of the scientific community to determine and express that consensus. The consensus knowledge in science evolves with time through experiment, discovery, and new theoretical ideas, and what is considered good science advice mirrors this evolutionary process, taking account of change and uncertainty. Like science, science advice is not so much a body of information as a procedure, which utilizes the processes of the scientific culture to decide questions of science. These processes lean heavily on peer review and the establishment of consensus, with the explicit exposure of areas of uncertainty, disagreement, and dissent.
Role of a Science Advisor
A science advisor cannot be one individual who “tells truth to power,” in part because no imaginable science advisor can possess more than a small part of the knowledge that is relevant to most complex scientific questions. The role of the advisor is to serve as a link between decision makers and the scientific community. The science advisor’s key value is the ability to know how science works, and to be known and trusted in the scientific community to ensure that the process of science advice involves a broad perspective and produces the best balanced advice possible, with explicit explanation of its uncertainties and remaining unknowns.
A science advisor1 to a senior decision maker cannot be expected to carry in his or her head all of the knowledge necessary to advise on every problem, or even all knowledge of the sources of knowledge. The rapid rate at which science and technology are evolving makes it more difficult to rely on the heroic expertise of one individual. This is particularly true in the United Nations system, given the diversity of countries, conditions, and scientific and technological priorities. What is required, rather, is a sound knowledge of the scientific process through practical, productive experience in a scientific subject and broad familiarity with the national and international organization of science. The advisor should be known among scientists and should be well regarded, if not necessarily eminent, although eminence can be a significant advantage.
It is a further advantage if the career of an advisor has involved contact with a variety of scientific disciplines, some experience in management and administration, and some contact with public policy. The ability to maneuver at the boundaries of science and general policy is important, as is the ability to deal with a variety of people and professions.
The real task of the science advisor is to serve as an intermediary to engage the broad scientific community in the service of the organization or the decision maker. The science advisor, or the office of the science advisor, must be able to set in motion the science advisory process described below. This may include assisting the policy maker(s) to see the relationship of policy problems to science issues, and assisting them to decide what science information and advice may be helpful. The science advisor must help to interpret the policy-making world to the scientific community and help interpret the science advice to make it most useful in the policy world.
It is important to distinguish the advisory group or study committee that is convened to provide advice on one single question from the more general science advisory board or standing committee that may advise policy makers on issues related to science and technology. In this chapter we will deal with the first kind of group, which we shall call a study committee for clarity and distinction.
Knowing when science advice is needed
Many policy problems that are apparently unrelated to science have scientific aspects, for which decision makers might profit from science advice. This may not always be clear unless someone familiar with the science is involved early in problem formulation. Much of the problem formulation in the United Nations is provided by government representatives and by the UN monitoring, assessment, and reporting systems. Several flagship reports of the United Nations, such as UNEP’s Global Environment Outlook, UNDP’s Human Development Report and UNESCO’s World Science Report, highlight emerging issues whose solutions may require science advice. Science advice may also be required to deal with controversial issues upon which governments are planning common actions or are being requested to allocate resources. Other areas that benefit from science
advice involve conflict that could threaten peace and security. For example, advice regarding the use of satellite imagery and global positioning systems is being used to settle border disputes between countries (NRC, 2002). The use of science advice early in the policy process may make the policy problem easier to attack.
It is the experience of the committee that narrowing the scope of a problem (smaller “intellectual bandwidth”) often makes it more easily solved. To the extent that science and science advice can help to narrow the scope of policy problems, or to divide them up into more manageable pieces, the probability of successful problem solution may be increased.
Furthermore, resources often can be saved by using information available publicly or from private sources that can shorten the process and reduce the need for a formal science advice process. For example, the need for technical training or scientific education is universal, and the needs of one organization or country may be met by information already prepared by and for others. Not every problem requires a unique solution.
Stating the Science Advisory Task
The problems faced by UN policy makers are generally not inherently scientific problems. To be amenable to science advice, the problems must be stated in a form that identifies their scientific or technological elements. For example, a farmer must decide which crops to plant; in order for scientific experts to be of any help, the question must be expressed in terms of soil characteristics, climate, water, markets, and costs. The farmer receiving the advice will evaluate it in the context of his culture and economic situation and those of his neighbors before making a final decision. Similarly, a policy maker may want to know whether a certain organization regulating fisheries is performing effectively; for science advice, this must be translated into terms of target setting, data gathering, methodologies, monitoring, and results. The policy maker may then factor this assessment in with other legitimate political and economic considerations in making a budgetary decision.
Translating a policy question into one amenable to the advisory process requires consultation between those who need the advice and someone who understands how to do the initial formulation of the task so that it is clear on which issues the advice and advisors might help. It is important to identify at the beginning of the process the principal user or users of the advice, those who will be its primary recipients. These may be senior decision makers or their representatives—those who will take action based upon, or influenced by, the advice. The product of the consultation between the users and the science adviser is the statement of task, the instructions to the study committee containing the concrete questions to be answered.
Identification and recruitment of a study committee
The first step after the users have approved the statement of task is to recruit a study committee. This is an ad hoc group, generally convened solely to answer one question, and the members will be recruited precisely on the basis of their ability to contribute to the answer to that question. This step requires knowledge of the landscape of science and
scientists and knowledge of possible sources of reliable and useful advice. The entire study might be contracted out to an independent organization like the InterAcademy Council or ICSU. If the advice is to be prepared in-house, outside experts will still play a key role. Persons selected may be employed at scholarly institutions, in industry, in private practice, or in national or international organizations. These must be selected on the basis of their expertise, as individual members, and should not “represent” their institutions, although their institutional knowledge may be valuable to the process. In the United Nations context, this requirement would demand separation between expert advice and diplomatic representation.
Issues of credibility frequently determine the eventual usefulness of the advice, and the selection process used will be important to the study committee’s credibility. Policy makers and the public are frequently unable to assess the credentials of individual scientists, but they are likely to be influenced by the reputation for scientific integrity of the appointing institution and by the transparency of the process. In this key matter of credibility, there is special value in selection of members of advisory bodies by the scientific community itself, or by clearly scientific merit-based processes. Examples of institutions with such processes include various science academies and national and international science funding organizations. For example, the British Royal Academy of Engineering posts the proposed members of its science advisory committees on its public web site for comments by its members and others who are interested.
Balance of regions, disciplines, and views
In recruiting committee members it is important to include different kinds of experts who might usefully contribute to the advice. They may be representatives of different geographical regions, as often required in the United Nations, or scientists whose specialties are on the margins of the specific scientific question. The capability of scientists with differing views is usually evaluated on the basis of their previous work published in peer-reviewed journals.
It is often useful to include members in the study committee who have general competence in science but may not be experts in the specific areas to be discussed. These seemingly “inappropriate members” of the group, if well chosen, are able to ask questions and make useful suggestions that are unlikely to emerge from a specialized discussion. It is possible to be inclusive of a range of relevant and responsible scientific opinion without including opinion that does not have any basis in data, observation, or logically developed theory. There may be legitimate reasons for the policy makers to take such views into account, but they should not be confounded with science advice.
One of the key issues facing science advice in the United Nations is the uneven distribution of scientific and technical capacity among countries and the limited resources available to developing countries to support the participation of their nationals in international science advisory activities. Strengthening science advice in the United Nations will require simultaneous strengthening of science, and even science advisory institutions, in developing countries and countries with economies in transition.
Management of bias and conflict
It is frequently difficult to find study committee members who are sufficiently knowledgeable about the subjects involved and are at the same time unbiased (i.e., have no previously published or publicly expressed opinions or conclusions on the subject) and have no conflicts of interest (i.e., have no likelihood of personal gain or loss depending on the outcome).2 One option is to use necessary experts who have conflicts of interest, not as members of the study committee itself, but as consultants to the committee, or invite them to speak to the committee in open meetings. This may allow the committee to receive the benefit of their knowledge without the appearance of bias in the advice itself. In other circumstances, a large fraction of the expert population may have bias or personal interest, and it may be necessary to include them as committee members, but in a balanced way. In the burgeoning field of genetic engineering, it is often found that nearly all experts competent to serve have already declared their views or participation in profit-making entities. In such cases, the specific biases and conflicts should be known to all who participate, to those who have a stake in the results, and to those who receive the advice. Where biases and conflicts are unavoidable, a balanced group and a policy of transparency is the best approach to getting credible, objective advice.
In any case, the deliberations of a study committee always should be shielded from external political influence. The members should be operating as scientists in their personal capacity, and not subject to external instructions, threats, or rewards that might depend on the conclusions they draw.
Management of data; role of staff
Most scientific questions cannot be answered entirely with the knowledge and data in the possession of the committee members alone, however well they may be chosen. The consideration of externally supplied data, theory, analyses, interpretations, and opinions is necessary. It is generally advisable to obtain this “outside” material in an open and public way, for example, by publishing lists of consultants and speakers heard by the committee and of references cited.
Management of data and information requires a well-trained staff to serve the study committee. Staff can carry out extensive review of the literature, commission analytical papers on particular aspects of the topic, arrange expert testimony to the committee, and convene workshops where a variety of experts may present data and express their views.
In addition to the competence of the staff, care should also be given to its independence. A large number of United Nations employees are either seconded by governments or
maintain close contacts with their home countries, and candidates for senior positions in the United Nations require the endorsement of their home countries. This political environment could introduce bias or perception of bias in staff performance. This has in some cases led to government distrust of documents prepared by secretariat staff, and it is not uncommon under such circumstances for government delegates to generate their own documents. This is particularly common in treaty negotiations where decisions are binding on governments. The principles of the management of bias, and of establishing clear and transparent procedures, must therefore also be extended to staff.
Generally, it is desirable to have the entire information gathering process as open as possible. Openness allows those interested to know the inputs made by others, and permits all interested parties to provide input, in person or in writing. However, there must always be opportunity for the committee to meet for discussion and debate without the presence of outside observers, sponsors, or interested parties. A distinction must be made between the information-gathering process, which is open to outside input, and the deliberative process that must be kept closed and free of outside influence. Aside from the technical and methodological issues involved, the issue of public confidence in the results must be paramount.
The product of the science advice process should be in the form of a report. A formal report is the mechanism by which it can be demonstrated that all members of the committee had an opportunity to present their views, that the recommendations are based on data and the best scientific knowledge at the time, and that outside views were taken into account. A member or members of the committee, each writing sections of particular interest to them, the chair, or staff may draft the report. In any case, the study committee as a whole must approve the resulting draft.
Consensus and dissent
It may happen that, after reasonable efforts to produce conclusions and recommendations that achieve a consensus of the committee, one or more members are unable to agree to the draft. Sometimes the problem may be solved by including the details of the issue and the reasons for the dissent in the draft itself, thus achieving a balance to which all may agree. If this is not possible, signed written dissents may be attached to the draft. If advice is to be useful it must be clear, and clarity may require statements of uncertainty and incompleteness where that is the state of knowledge at the time. Dissent may be part of providing this clarity.
The draft report should be reviewed by outside reviewers (scientific journals call them “referees” and research managers call them “peer reviewers”), some of whom are chosen for their expertise in the subject, and some because they have a good general view of the relevant technical subjects and are experienced in the science advice process. The
purpose of review is not to “certify” the result but, rather, to look for errors, lacunae, lack of clarity, missing information, etc. The reviewers represent the universe of potential critical readers, and the review serves as “quality control” for the advisory process. It serves and protects the institution carrying out the advisory process.
Recruitment of reviewers is carried out in much the same manner as recruitment of the study committee, and bias and conflict of interest must be taken into account for the same reasons. However, the role of reviewers is such that a wider spectrum of views and biases may be appropriate for the reviewers than for the committee itself. Reviewers’ comments often reflect national and other biases, and efforts should be made to ensure that the review process is not a substitute for introducing political negotiations into advice documents.
The committee should consider the comments of the reviewers and make appropriate changes in the documentary advice in response to them. Some institutions require that all comments be answered, whether changes are made to the work product or not. To avoid endless debates with reviewers, this can require some individual who has not been part of the process to be responsible for vouching for the satisfactory nature of the responses.
Delivery of advice
The final report is delivered to the principal users designated at the beginning of the process. The delivery is frequently accompanied by personal briefings to the users by the chair or other members of the committee. It is helpful if the chair or the designated members can continue to be available from time to time to answer questions and for further discussions and interactions with the users.
It is generally appropriate to publish or otherwise disseminate the report to legislative bodies, executive bodies, and the public. This will have clear benefits in credibility with parties interested in the matter.
Implications of the process
Generally, the most serious problems arise at the “front end” and at the “back end” of the process. The formulation of the task must be clear, relevant, and doable. The science advice must be delivered in an understandable form, with its implications for action (or inaction) clear. Those having the best and most detailed scientific knowledge are not necessarily best able to perceive its implications for policy, decision making, and action.
For this reason, it is useful for senior decision makers to have their own science advisor who has their trust and the resources to carry out the scientific advisory process as described. Because of the multidisciplinary nature of the process, a single isolated advisor without the resources to call upon experts in a variety of disciplines cannot provide science advice that will be accepted by the scientific community, whatever his or her credentials, or serve well the decision maker or the institution.
The participation of the science advisor in bringing science into the complexities of policy problem definition and into the interpretation of the science advice as it may impact upon policy can be very helpful in the implementation of the recommendations.
This can make the difference between successful use of science and the failure of an action because of scientific constraints. It is important that science be part of the senior policy process, where understanding its implications may be crucial.
Follow-up and Impact
Of course, even very good science advice is often not translated into policy, for a very wide range of reasons. As noted above, the scientific dimension, although very important, must compete with many other considerations. However, there are an impressive number of important cases in which solid scientific inputs have made critical contributions to policy. Well-known examples are scientific understanding of ozone depletion and an early study by the US Institute of Medicine which first called attention to the urgent need for awareness and prevention programs related to AIDS. The case of the IPCC is also noted elsewhere in this report. The potential for high impact underscores the importance of publication, dissemination, and discussion of science advisory products with policy makers, groups that are affected by the policies in question, and the public.
The elements outlined above are synthesized from efforts around the world to find ways of ensuring scientific credibility on the one hand and interactions between science and policy on the other. They may therefore provide a foundation against which to assess the role of science advice in the United Nations system.