National Academies Press: OpenBook

A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap (2006)

Chapter: Appendix E Integration of Data-Based Evidence and Expert Opinion in Decision Making

« Previous: Appendix D Bone Fracture Risk Associated with Prolonged Exposure to Microgravity
Suggested Citation:"Appendix E Integration of Data-Based Evidence and Expert Opinion in Decision Making." Institute of Medicine and National Research Council. 2006. A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap. Washington, DC: The National Academies Press. doi: 10.17226/11467.
×

E
Integration of Data-Based Evidence and Expert Opinion in Decision Making

BAYESIAN UPDATING

In certain cases, it may be necessary to update research data with new findings and with stakeholder opinions (where stakeholders are defined to be mission specialists, National Aeronautics and Space Administration [NASA] directors, managers, and flight surgeons). Bayesian updating may be one strategy for integrating stakeholder opinions with data from research studies. Such a strategy can accommodate contrasting points of view from stakeholders expressed in a subjective manner. It is necessary that the data from different research studies measure the same underlying factor (e.g., diastolic blood pressure, depression) on the same scale (e.g., millimeters of mercury for blood pressure, Hamilton scale for depression).

This updating strategy consists of the following steps: (1) selection of a sample of stakeholders; (2) elicitation of probability information from these stakeholders; (3) translation of this information to statistical distributions, called “prior” distributions, for each contrasting view of the stakeholders; (4) assignment of an “importance” weight to each of these prior distributions for each of the contrasting views; (5) with these importance weights, derivation of a “summary prior” distribution by taking a weighted combination of the contrasting prior distributions; (6) derivation of a “summary likelihood” pooling all study datasets while accounting for the varying variability and sample sizes across the study datasets; (7) derivation of a “sum-

Suggested Citation:"Appendix E Integration of Data-Based Evidence and Expert Opinion in Decision Making." Institute of Medicine and National Research Council. 2006. A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap. Washington, DC: The National Academies Press. doi: 10.17226/11467.
×

mary posterior” distribution from the summary prior distribution and summary likelihood; (8) choosing a utility function to incorporate costs and stakeholders’ sensitivity to such costs; and (9) decisions based on regrets or opportunity costs in cost–benefit or risk–benefit models by weighing outcome information from summary posterior distribution (e.g., mean differences, risk differences, risk ratios, odds ratios, and interactions involving these effects) against utility functions. Each of these steps is described in more detail below.

  • Steps 1–2. Selection of a sample of stakeholders and elicitation of probability information from these stakeholders: The selection process should at least be comprehensive, maximizing the number of contrasting points of view, if the process is not random. Stakeholders’ prior opinions will be elicited with questionnaires. In these questionnaires, stakeholders will be asked to provide ranges of probabilities of confidence in positive and negative results. The design of the questionnaire will be selected from several different designs available in the research literature. Chaloner and Rhame (2001) presented an interactive approach based on iterative elicitation from physicians enhanced by real-time iterative and graphical feedback to the physicians of their quantified opinions. Parmar et al. (1994, 2001) and Spiegelhalter et al. (1994) presented a questionnaire for eliciting prior distributions in a pair of large randomized trials conducted as part of the British Medical Research Council Cancer Trials. In a hepatocellular carcinoma clinical trial, Tan et al. (2003) also used such a questionnaire to elicit prior information on the equivalence between surgery with adjuvant therapy versus surgery alone on recurrence-free survival. Alternatively, a series of individual description formats have been developed by Vennix et al. (1994). These individual questionnaire formats focus on the different phases proposed by for elicitation: (1) the positioning phase, which defines the context of the information; (2) the description phase, which guides stakeholders through four aspects of description (visual, verbal, textual, and graphic); and (3) the discussion phase, in which the individual descriptions from phase 2 are examined and compared.

  • Step 3. Translation of this information to prior distributions: Individual histograms representing the prior beliefs of each investigator can be constructed from the relative probability values that stakeholders may be asked to provide in Step 2. Following Spiegelhalter et al. (1994), these probability values may be summarized across stakeholders with similar opinions to then construct “overall histograms”and “skeptical histograms.”

Suggested Citation:"Appendix E Integration of Data-Based Evidence and Expert Opinion in Decision Making." Institute of Medicine and National Research Council. 2006. A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap. Washington, DC: The National Academies Press. doi: 10.17226/11467.
×

These histograms will represent the overall (or clinical) and skeptical (or cautious) prior distributions associated with the stakeholders’ views of the problem. A skeptical prior distribution corresponds to the beliefs of individuals who are reluctant to accept alternative hypotheses of interest to the investigators. The resulting histograms can be transformed to the scale on which research data have been collected (Tan et al., 2003).

  • Steps 4–7. Determination of multivariate prior distributions from multiple stakeholders, estimation of summary likelihoods from multiple datasets, resulting in a derivation of a summary posterior distribution: Such a procedure is based on Bayes’ rule and entails intractable integration resulting in simulation-based integration (e.g., Markov Chain Monte Carlo), which many commercially available software packages now offer (Spiegelhalter et al., 1994).

  • Steps 8. Choosing a utility function to incorporate costs and stakeholders’ sensitivity to such costs: Such functions involve determining costs from implementing mitigation strategies and reduced costs from preventing problem outcomes (Pliskin et al., 1980; Berger, 1985; Lindley, 1985; Gold et al., 1996). Sensitivity analysis of the overall procedure outlined here includes varying such cost estimates (Matchar and Samsa, 1999; Matchar et al., 1997).

  • Step 9. Decisions based on regrets or opportunity costs from weighing information on outcomes under mitigation strategies against outcomes under the absence of mitigation strategies: Regrets are based on loss functions as contrasts between decisions that lead to optimal utility benefits and the utility benefits based on observed or predicted data. The expected loss functions allow the incorporation of research data and previous opinions of stakeholders by integrating utility functions for the optimal and observed decisions with respect to the summary posterior distributions (Berger, 1985: Lindley, 1985). For complex sequences of branching decisions based on outcomes of previous decisions, backward induction algorithms may be used (Bellman, 1957).

Overall, such decision processes present a complex web of different statistical procedures, research datasets, and opinions by stakeholders. This complex web is sensitive to selected procedures and corresponding assumptions; thus, this sensitivity is assessed by varying assumptions and operational procedures (Matchar and Samsa, 1999). Varying assumptions, procedures, and information used for forming utility functions and prior distribution may be done formally with quantified ranges of possible values

Suggested Citation:"Appendix E Integration of Data-Based Evidence and Expert Opinion in Decision Making." Institute of Medicine and National Research Council. 2006. A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap. Washington, DC: The National Academies Press. doi: 10.17226/11467.
×

and model-averaging techniques or informally by choosing plausible values of model and prior distribution parameters (Spiegelhalter et al., 1994).

REFERENCES

Bellman RE. 1957. Dynamic Programming. Princeton, NJ: Princeton University Press.

Berger JO. 1985. Statistical Decision Theory and Bayesian Analysis, 2nd Ed. New York: Springer-Verlag.


Chaloner K, Rhame FS. 2001. Quantifying and documenting prior beliefs in clinical trials. Statistics in Medicine 20: 581–600.


Gold MR, Siegel JE, Russell LB, Weinstein MC (eds). 1996. Cost-Effectiveness in Health and Medicine. Oxford, UK: Oxford University Press.


Lindley DV. 1985. Making Decisions, 2nd Ed. New York: Wiley.


Matchar DB, Samsa GP. 1999. Using outcome data to identify best medical practice: the role of policy models. Hepatology 29 (6 Suppl): 36S–39S.

Matchar DB, Samsa GP, Mathews JR, Aneukiewicz M, Parmigiani G, Hasselblad V, Wolf PA, D’Agostino RB, Lipscomb J. 1997. The Stroke Prevention Policy Model (SPPM): linking evidence and clinical decisions. Ann. Int. Med. 127: 704–711.


Parmar MKB, Griffiths GO, Spiegelhalter DJ, Souhami RL, Altman DG, van der Scheuren E, CHART steering committee. 2001. Monitoring of large randomised clinical trials: a new approach with Bayesian methods. Lancet 358(9279): 375–381.

Parmar MKB, Spiegelhalter DJ, Freedman LS. 1994. The CHART trials: Bayesian design and monitoring in practice. Statistics in Medicine 13: 1297–1312.

Pliskin JS, Shepard D, Weinstein MC. 1980. Utility functions for life years and health status: theory, assessment, and application. Operations Research 28: 206–224.


Spiegelhalter DJ, Freedman LS, Parmar MKB. 1994. Bayesian approaches to randomised trials. J. R. Stat. Soc. A 157: 357–387.


Tan SB, Chung YF, Tai BC, Cheung YB, Machin D. 2003. Elicitation of prior distributions for a phase III randomized controlled trial of adjuvant therapy with surgery for hepatocellular carcinoma. Control. Clin.Trials 24(2): 110–121.


Vennix JAM, Anderson DF, Richardson GP, Rohrbaugh J. 1994. Model building for group decision support: issues and alternatives in knowledge elicitation. In Morecroft J, Sterman J. (eds.) Modeling for Learning Organizations. Portland, OR: Productivity Press.

Suggested Citation:"Appendix E Integration of Data-Based Evidence and Expert Opinion in Decision Making." Institute of Medicine and National Research Council. 2006. A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap. Washington, DC: The National Academies Press. doi: 10.17226/11467.
×
Page 120
Suggested Citation:"Appendix E Integration of Data-Based Evidence and Expert Opinion in Decision Making." Institute of Medicine and National Research Council. 2006. A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap. Washington, DC: The National Academies Press. doi: 10.17226/11467.
×
Page 121
Suggested Citation:"Appendix E Integration of Data-Based Evidence and Expert Opinion in Decision Making." Institute of Medicine and National Research Council. 2006. A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap. Washington, DC: The National Academies Press. doi: 10.17226/11467.
×
Page 122
Suggested Citation:"Appendix E Integration of Data-Based Evidence and Expert Opinion in Decision Making." Institute of Medicine and National Research Council. 2006. A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap. Washington, DC: The National Academies Press. doi: 10.17226/11467.
×
Page 123
Next: Appendix F Biographical Sketches of Committee Members and Staff »
A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap Get This Book
×
Buy Paperback | $52.00 Buy Ebook | $41.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Extending the spatial and temporal boundaries of human space flight is an important goal for the nation and for the National Aeronautics and Space Administration (NASA). However, human space flight remains an endeavor with substantial risks, and these risks must be identified, managed, and mitigated appropriately to achieve the nation's goals in space. The Bioastronautics Roadmap (BR) is the result of extensive, commendable efforts on the part of NASA to prioritize research efforts to meet these challenges.

In 2003, NASA asked the Institute of Medicine (IOM), in collaboration with the Division on Engineering and Physical Sciences of the National Academies, to conduct a review of the BR. Specifically, NASA asked the committee to (1) conduct a comprehensive assessment and report of the strengths and weaknesses of the content and processes of the Bioastronautics Roadmap as applied to the missions described in the President's exploration initiative and (2) identify the unique challenges for accomplishing its goals and objectives.

In September 2004, the committee released its preliminary report to NASA entitled Preliminary Considerations Regarding NASA's Bioastronautics Critical Path Roadmap. That document presented the committee's preliminary conclusions about the strengths and weaknesses of the April 2004 version of the BR. This report, A Risk Reductions Strategy for Human Exploration of Space, builds on those preliminary conclusions and provides recommendations to NASA about how to address the issues identified by the committee.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!