Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
B Statement of Task The Aeronautics and Space Engineering Board of the National Research Council (NRC) will form a com- mittee and three subordinate panels to assess the over- all scientific and technical quality of the Revolutionize Aviation (RA) goal area of NASA's Aerospace Tech- nology Enterprise. The committee's assessment will include findings and recommendations related to the quality and appropriateness of all NASA research in the RA goal area. This includes internal, collaborative and competitively sourced research, development, analysis, etc. While the primary objective is to conduct peer assessments that provide scientific and technical advice, the committee may offer programmatic advice when it follows naturally from technical considerations or is requested by the NASA Associate Administrator for Aerospace Technology. The committee will be assisted by the three pan- els, each of which will assess the scientific and tech- nical quality of one of the programs in the RA goal area. Each panel will provide inputs to the commit- tee report via internal working documents to the committee. Panels will meet as required during the study to receive technical presentations about the projects un- der review by their group and formulate final findings and recommendations. Panel members will also make site visits as deemed necessary in formulating the as- 116 sessment. Portions of each meeting will be highly in- teractive with NASA personnel. After completion of its deliberations and investigation, the panels will re- port to the committee on findings via internal privi- leged correspondence and working papers. 1- The committee will meet as required during the review period to plan the review process, meet with the panel members, and discuss the charge to the commit- tee and panels and to discuss panel working papers, findings, and recommendations. Meetings will involve interactive discussions with NASA personnel from the programs. The committee will develop a final report developed from panel inputs and discussions at the committee meetings. The committee's observations will follow broad themes concerning technical and scientific quality and appropriateness of research, the research performers, and the research plan. The committee and panels will evaluate the following themes: research portfolio; for- mulation of the research plan; connections to the broader community; methodology; and overall capa- bilities. Examples of specific criteria for the panels to use as appropriate are found in the Appendix. Neither the committee nor panels will make ex- plicit budget recommendations to NASA, but will in- stead comment on program content, gaps in technol- ogy, and other issues outlined above.
APPENDIX B APPENDIX Where appropriate, the panel assessments shout use specific criteria, such as the following: Research Portfolio Is the balance between fundamental and user driven research proper? Is research being conducted in the proper areas? · Are there plausible hypotheses supporting each of the research plans? Is far-term research at the forefront of science and determined to be a world-class endeavor? Is the proper amount of high-risk, high-payoff research being pursued? Is the application of fundamental science to solve real-world problems adequate? Formulation of the Research Plan ~7 gram goals, NASA's strengths, and the time horizon for the project? Are critical personnel and facilities required to support the program well defined? Connections to the Broader Community . . . Are the program's goals and objectives clearly defined and consistent with relevant documents such as NASA's Strategic Plan? Is there evidence of a clear understanding of the need by NASA's enterprises, other organizations (i.e., the FAA, DoD, etc.), or the aerospace com- Methodology munity at large for the R&D or analysis, and the potential benefits? Are the program's deliv- erables to those organizations clearly articulated and are those organizations adequately involved in the planning and review process? Can the expected benefits be accomplished by the proposed research? If not, is the path to ad- equately maturing the research clear? Is this planning well supported by sufficient decision points, downselects, customer agreements, and/ or unallocated outyear funding? Are there sufficient near-term deliverables or progress metrics from which the program can be regularly assessed? Are there sufficient off- ramps or sunsets to ensure that funding is real- Overall Capabilities located within the program or to other programs if the program does not make adequate progress towards one or more of its goals and objectives? Are the program' s plans for independent and/or external reviews adequate and appropriate? Are appropriate scientific and technical objec- tives being posed, taking into consideration pro- Is the research being accomplished with a proper mix of personnel from NASA, academia, indus- try, and other government agencies? Is the pro- gram using high-quality research performers or is there untapped talent outside the program that can be brought to bear? Is there evidence that the research plan for the area under review reflects a broad understand- ing of the underlying science and technology and of comparable work within other NASA units as well as industry, academia, and other federal laboratories? Is there evidence that the research builds appro- priately on work already done elsewhere? Does it leverage the work of leaders in the field? Is the strategy for out-of-house work (competi- tions, partnerships, etc.) well chosen and man- aged? How well crafted are the research plans for the areas under review? In general, is the use of laboratory experiment, modeling, simulation, and/or field test appropriate? How well are these methods integrated? Have the appropriate supporting system-level assessments been conclucted? Do both the researchers and managers under- stand and manage the risks involved to an ap- propriate level? Are the plans for further study reasonable and justifiable? · Is the scientific or engineering quality of the work (including work performed in academia and industry) comparable to similar world-class efforts at other institutions, and is it appropriate for the goal? · Are the qualifications of the scientific and engi- neering staff (including researchers in aca-
118 / r AN ASSESSMENT OF NASA 'S AERONA UTICS TECHNOLOGY PROGRAMS demia and industry) sufficient to achieve pro- gram goals? Are the capabilities, quantity, and state of readi- ness of equipment and facilities sufficient to achieve program goals? Are personnel, equipment, and facilities sup- plied by support contractors used efficiently; do they fill gaps in government capabilities with- out duplication? The selection of criteria for each assessment and the relative weights given to each criterion are within each panel's discretion and can vary from program to program.
