Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 255
D
Decision Conferencing and Multicriteria
Decision Analysis
Benefit–risk assessment characterizes information regarding estimates of
benefits, estimates of risks, and the severity and comparability among health end-
points associated with benefits and risks. Much has been published about methods
for quantitative benefit–risk assessment (Coplan et al., 2011; Guo et al., 2010).
Such assessments are widely used in decision-making contexts, particularly with
regard to environmental regulation, and the mode of their particular application
varies (NRC, 1994, 2009). The committee does not wish to prescribe a particular
method for conducting benefit–risk assessment, particularly inasmuch as formal
quantitative approaches are likely to be used only in select circumstances when
disagreements about potential regulatory actions arise. Instead, the committee
highlights in this appendix two decision tools that incorporate key consider-
ations of benefit–risk assessment relevant to regulatory decision-making: deci-
sion conferencing, which is a social process intended to engage all the relevant
stakeholders to provide scientific judgments at key points in the decision-making
process, and multicriteria decision analysis (MCDA), which is a technical model
for making decisions that have multiple objectives (Walker et al., 2005). Decision
conferencing and MCDA have been used in other contexts and are offered here as
examples of potentially useful approaches that integrate analytic and deliberative
processes for in-depth evaluation and informed assessments of the benefit–risk
balance of approved drugs (Phillips, 2006; Walker et al., 2005). Both processes,
when used in a transparent way that documents the inputs into the process, can
help to identify the underlying sources of scientific disagreements that are dis -
cussed in Chapter 3.
255
OCR for page 256
256 STUDYING THE SAFETY OF APPROVED DRUGS
DECISION CONFERENCING
Decision conferencing is a tool that brings key experts and stakeholders
together to generate a shared understanding of a challenge, create a sense of
common purpose, and gain commitment to a way forward (Phillips, 2006) to
aid evaluation and assessment of the benefits and risks associated with a drug
(Walker et al., 2005). Decision conferencing has four basic elements: attendance
by key stakeholders (for example, regulators; methodologists who have expertise
in design, conduct, and analysis of observational studies and clinical trials; deci -
sion scientists; physicians who have relevant clinical expertise; patients; and the
public); impartial facilitation to guide discussions of estimates of benefit and risk,
degree of uncertainty, and values and preferences for health endpoints; on-the-
spot modeling with continuous display of the developing model; and an interac -
tive and iterative group process (Phillips, 2006). When quantitative benefit–risk
assessments are needed in situations in which disagreements about appropriate
regulatory action arise, a neutral facilitator can guide the group through the
stages of discussing the issues, developing models for evaluating the issues, and
eliciting assumptions about the quality of evidence regarding benefits and risks
and about underlying ethical values and preferences for health endpoints without
contributing to the content of discussions. Although quantitative estimates result -
ing from benefit–risk assessment may appear to provide objective information
about optimal regulatory decisions, assumptions used in the model are often
based on individual judgments about the quality of evidence related to benefits
and risks, as discussed in Chapter 3, or based on individual values or preferences
for different health endpoints. Using a process like decision conferencing helps
to frame the issues and identify the relevant data and evidentiary gaps to guide
later data-gathering and thereby improves the efficiency and transparency of the
benefit–risk assessment process (Phillips, 2006).
The decision-conferencing process could be integrated into FDA’s current
processes and requirements under the Federal Advisory Committee Act,1 and it is
similar to initiatives that FDA currently has planned. For example, FDA is working
with stakeholders, using faculty of the George Washington University as facilita-
tors, to develop guidance material for approving obesity drugs (McCaughan, 2011).
Decision conferencing in benefit–risk assessment has four advantages:
• It facilitates and focuses thinking about a complex challenge.
• It helps to establish common purpose among participants and commitment
to move forward.
• It promotes transparency and a shared understanding of how stakehold-
ers define benefits, risks, degree of uncertainty, ethical values regarding
outcomes, and potential regulatory actions for managing risk.
1 Federal Advisory Committee Act, PL 920-463, 86 Stat. 770 (1972).
OCR for page 257
257
APPENDIX D
• Because decision conferencing can generate both the outcome and a
description of the process that leads to the outcome, it can improve stake -
holders’ and the public’s understanding of a regulatory decision.
MULTICRITERIA DECISION ANALYSIS
MCDA is a set of methods designed to bring together evaluations of options
on different criteria into one overall decision (CHMP, 2008). There are many vari-
ants of MCDA, some of which may be adapted to consider the uncertainty of the
decision-maker (Linkov and Seager, 2011), and MCDA can be used to provide
either a qualitative or a quantitative assessment. The basic methods of MCDA are
scoring and weighting (CHMP, 2008). Scoring involves the process of assigning
numerical values to options according to particular criteria. Weighting ensures the
comparability of the numerical values assigned to all criteria, which allows com -
parison of different health states with a single metric (Linkov and Seager, 2011).
The weights assigned to scores reflect the relative importance of the underlying
criteria for the benefit–risk assessment outcome (Walker et al., 2005).
MCDA uses four steps (Linkov and Seager, 2011):
1. Defining the problem and the decision context.
2. Identifying stakeholders, decision-makers, assessment criteria (for exam -
ple, health outcomes of interest), and the relative importance of different
health outcomes.
3. Defining and assessing management alternatives whereby the effects of
different regulatory decisions on each criterion, or health outcome, are
assessed.
4. Allowing for variability in weighting of different criteria and accounting
for the stochastic nature of data through the use of probabilistic sensitiv -
ity analysis to provide a rank order of different alternatives for distinct
stakeholder groups.
Those four steps help to ensure that all participants understand and are in
agreement about the need for a regulatory decision, the criteria by which benefit–
risk balance is judged, the evidence and its uncertainties, the values and prefer-
ences of different stakeholders, and the consequences of different regulatory
decisions. The outputs or information synthesis of the benefit–risk assessment
stage of the framework should include model inputs and model outputs (Linkov
and Seager, 2011). The model inputs would include estimates of benefits, esti-
mates of risks, the degree of uncertainty, and preferences for health outcomes
based on ethical values. The model outputs may be characterized quantitatively,
for example, the benefits of a drug outweigh the risks 85 percent of the time; or
qualitatively, for example, there is clear and convincing evidence that the benefits
of a drug outweigh its risks. The synthesis should also discuss any uncertainty
OCR for page 258
258 STUDYING THE SAFETY OF APPROVED DRUGS
analyses that were conducted and the process by which the benefit–risk assess -
ment was performed—that is, a description of the “decision conferencing” or
other process that FDA used to seek and include stakeholder input as necessary.
Useful information includes statements of who provided the inputs and who
moderated the process. Both the process and the outcome should be documented
as a way to set the stage for understanding the regulatory decision. The outcome
of the assessment process, whether quantitative or qualitative, becomes the evi -
dentiary basis of the regulatory decision-making that is at the heart of the next
stage—benefit–risk management.
LIMITATIONS OF USING MODELING APPROACHES
FOR BENEFIT AND RISK ASSESSMENTS
MCDA is a useful tool for benefit–risk assessment, but it has its limita-
tions, both in its own right and as an aid to regulatory decision-making. Using
a quantitative MCDA approach to benefit–risk assessment forces participants to
be explicit about how they evaluate existing evidence regarding the effectiveness
of a drug and its associated harms and about the degree of uncertainty regarding
benefits and risks. Such a quantitative assessment, however, can obscure underly-
ing interpretations of scientific findings. The interpretations should be explicitly
described, and decision conferencing can mitigate some concerns by describing
differences in underlying assumptions. In addition, quantification of intangible
factors, such as a patient’s preferences that might be based on various degrees of
dread for different diseases, may be difficult in MCDA models although relevant
for the decision-making process.
Preferences regarding the relative importance and severity of health states
associated with the disease or the treatment may also vary widely among stake-
holders and could potentially be obscured by using modeling approaches. How -
ever, decision conferencing can serve as a useful tool for conducting MCDA by
explicitly describing stakeholders’ values and preferences and how they may
affect regulatory decisions.
Even with the aid of social and technical decision tools, benefit–risk assess -
ment is unavoidably limited by the quality and quantity of available evidence.
Uncertainty analysis may identify the key information needed to support a
particular regulatory action, but such data may not be available in the time
needed to address the public health issue at hand. The available data may lead to
multiple interpretations and contribute to decision-making gridlock. That delay
could compound limitations of the resources and expertise available in FDA to
conduct benefit–risk assessments and result in a backlog. Such risk-assessment
backlogs have occurred in the Environmental Protection Agency (NRC, 2009).
If tools like decision conferencing and MCDA are used to enhance transparency,
the rationale for regulatory decisions can at least be understood by stakeholders
even if disagreements about optimal regulatory action remain. The transparency
OCR for page 259
259
APPENDIX D
allows stakeholders to have a shared understanding of differences in scien-
tific assessment of preferences regarding health outcomes, which can help to
determine whether additional data are needed and whether those data will meet
thresholds for influencing future regulatory action. Decision conferencing and
MCDA, however, may not be appropriate for every situation, and benefit–risk
assessments should be scalable to the severity and scope of the particular public
health concern at issue, the level of controversy surrounding it, and FDA resource
constraints.
REFERENCES
CHMP (Committee for Medical Products for Human Use). 2008. Reflection paper on benefit-risk
assessment methods in the context of the evaluation of marketing authorisation applications of
medicinal products for human use. London, UK: European Medicines Agency.
Coplan, P. M., R. A. Noel, B. S. Levitan, J. Ferguson, and F. Mussen. 2011. Development of a frame -
work for enhancing the transparency, reproducibility and communication of the benefit-risk
balance of medicines. Clinical Pharmacology & Therapeutics 89(2):312-315.
Guo, J. J., S. Pandey, J. Doyle, B. Bian, Y. Lis, and D. W. Raisch. 2010. A review of quantitative risk-
benefit methodologies for assessing drug safety and efficacy-report of the ISPOR risk-benefit
management working group. Value Health 13(5):657-666.
Linkov, I., and T. P. Seager. 2011. Coupling multi-criteria decision analysis, life-cycle assessment, and
risk assessment for emerging threats. Environmental Science and Technology 45(12):5068-5074.
McCaughan, M. 2011. FDA rethinking weight loss standards: A test case for patient-focused drug
development. “The Pink Sheet” Daily, June 6, 2011.
NRC (National Research Council). 1994. Science and judgment in risk assessment. Washington, DC:
National Academy Press.
NRC. 2009. Science and decisions: Advancing risk assessment. Washington, DC: The National
Academies Press.
Phillips, L. D. 2006. Decision conferencing. Operational research working papers, LSEOR 06.85.
London, UK: Operational Research Group, Department of Management, London School of
Economics and Political Science.
Walker, S., L. Phillips, and M. Cone. 2005 (unpublished). Benefit-risk assessment model for medicines:
Developing a structured approach to decision making.
OCR for page 260
