Appropriate level of risk refers to whether the level of risk associated with an R&T Challenge is appropriate for a NASA research project. For example, NASA should not pursue incremental research that is of such low risk that industry could easily complete the research. Nor should NASA pursue research of great theoretical promise if the scientific and technical hurdles are so high that it has very little chance of success. A score of 1 implies that the Challenge is either very low risk (such that industry could pursue it) or extremely high risk (such that there is only a small chance of seeing any benefit without unforeseen revolutionary breakthroughs). A score of 3 implies that the Challenge either has low risk or very high risk. A score of 9 implies that it has moderate to high risk, which is a good fit to NASA’s level of risk tolerance. All NASA research should be expected to progress toward established goals, but innovation is not possible without tolerance for failure, and the pursuit of moderate-and high-risk technology is appropriate for the nation’s center of excellence for aeronautics.
The top 10 R&T Challenges for each Area, in priority order, are discussed in Chapter 3. All the Challenges are discussed in Appendixes A to E, which also contain specific milestones. The technical discussions and milestones included in this report are intended to be advisory, as it was not feasible to complete a rigorous, comparative assessment of all of the research options that might be associated for each of the 89 Challenges. The committee believes that the best approach for selecting specific research projects to fund would be for NASA to solicit proposals from industry and academia at the level of the individual Challenges.
The QFD process appears to be a rigorous quantitative process, with strict, laid-out criteria for each score. However, while each panel could consistently distinguish between what deserves a 3 and what deserves a 9, for example, some variations from panel to panel were inevitable. Furthermore, QFD is an iterative process. After initially scoring each R&T Challenge, panel members examined their results, assessed the justifications for each score for internal consistency and accuracy, and then adjusted some scores and justifications, as appropriate.
Once each panel completed the QFD process for its R&T Area, the steering committee reviewed the results and raised issues for the panels to reconsider to assure that the results were generally consistent when two panels had similar R&T Challenges. In the end, the panels and the steering committee concurred that (1) the Strategic Objectives were properly defined and weighted and (2) the Challenges were correctly scored and prioritized. Thus, although the steering committee reserved the right to change QFD scores without the concurrence of the panels, it did not find such action necessary.
The steering committee could have attempted to create a single integrated priority list of the R&T Challenges from all five R&T Areas. However, it was not practical for the committee to make extensive pairwise comparisons to assure that the scores for each R&T Challenge from each panel were consistent with the scores for dissimilar R&T Challenges from other panels. The steering committee also considered the value of having a single list of priorities and satisfied itself that (1) the results from each panel were generally consistent and well justified; (2) the high-priority R&T Challenges in each R&T Area were, indeed, high-priority items that should be included in NASA’s aeronautics R&T program; and (3) the ultimate purpose of prioritizing R&T Challenges is presumably to determine which Challenges will be funded, and that determination will depend upon budgetary factors that were beyond the scope of this study (see Appendix G).
Given the above considerations, instead of creating an integrated, prioritized list of R&T Challenges from all five panels, the steering committee decided that the best use of the limited time and resources available to complete the study would be to identify Common Themes and formulate overall findings and recommendations (see Chapters 4 and 5). Given this situation, readers are cautioned against comparing the national and NASA priority scores for R&T Challenges from different panels to determine which is more important. The steering committee firmly believes that NASA should support research in all five R&T Areas, and the priorities identified in this report can be relied on to guide research planning within each of those areas.
National Institute of Aerospace (NIA), National Strategy Team. 2005. Responding to the Call: Aviation Plan for American Leadership. Available online at <www.nianet.org/nianews/AviationPlan.php>.
National Research Council (NRC). 1992. Aeronautical Technologies for the Twenty-first Century. Washington, D.C.: National Academy Press. Available online at <http://books.nap.edu/catalog/2035.html>.