Attachment B Statement of Task
This project will produce biennial assessments of the programs within NASA's Aerospace Technology Enterprise—the Pioneering Revolutionary Technology (PRT) program, the Aviation program, and the Space Transportation program. The first review in the series will be of the PRT program group; other reviews will follow in subsequent years. Programs within the PRT group are the Enabling Concepts and Technologies (ECT) program, the Computing, Information, and Communications Technology (CICT) program, and the Engineering of Complex Systems (ECS) program.
The committee will assess the overall scientific and technical quality of the PRT program elements. These assessments will include findings and recommendations related to the quality and appropriateness of NASA's internal and collaborative research, development, and analysis. While its primary objective is to conduct peer assessments that provide scientific and technical advice, the committee may offer programmatic advice when it follows naturally from technical considerations or is requested by the NASA Associate Administrator for Aerospace Technology.
The committee will be assisted by three NRC panels that each focus on one of the three elements of the PRT program listed above. Each panel will assess the scientific and technical quality of selected programs in the element under their purview. Each panel will provide input to the committee's report via internal working draft reports to the committee. Panels will meet twice during the study to receive technical presentations about the projects under review by their group and formulate final findings and recommendations. Panel members will also make site visits as deemed necessary in formulating the assessment. Portions of each meeting will be highly interactive with NASA personnel. After completion of its deliberations and investigation, the panel will report to the committee on its findings via internal privileged correspondence and working papers.
The main committee will meet twice during the review: once to plan the review process, meet with the panel members, and discuss the charge to the committee and panels, and a second time to discuss in a closed session the working papers and findings and recommendations. This meeting will also involve interactive discussions with NASA personnel from the program. A final report will be developed from discussion at this final meeting. Before the final report is published, committee and panel members may revisit select programs within the PRT group during a short re-evaluation process. This re-evaluation will assess progress made by individual programs within the PRT which were initially deemed to be problematic.
While the committee's observations will follow broad themes concerning technical and scientific quality and appropriateness of research, the research performers, and the research plan, the panel assessments should use specific criteria, where appropriate, such as the following:
Research Portfolio
-
Is the balance between fundamental and user-driven research proper?
-
Is research being conducted in the proper areas?
-
Are there plausible hypotheses supporting each of the research plans?
-
Is far-term research at the forefront of science and determined to be a world-class endeavor?
-
Is the proper amount of high risk/high payoff research being pursued?
-
Is the application of fundamental science to solve real-world problems adequate?
Formulation of the Research Plan
-
Are the program’s goals and objectives clearly defined and consistent with relevant documents such as NASA’s Strategic Plan?
-
Is there evidence of a clear understanding of the need by NASA’s enterprises, other organizations (e.g., the FAA, DOD, etc.), or the aerospace community at large for the R&D or analysis, and the potential benefits? Are the program’s deliverables to those organizations clearly articulated and are those organizations adequately involved in the planning and review process?
-
Can the expected benefits be accomplished by the proposed research? If not, is the path to adequately maturing the research clear? Is this planning well supported by sufficient decision points, downselects, customer agreements, and/or unallocated outyear funding?
-
Are there sufficient near-term deliverables or progress metrics by which the program can be regularly assessed? Are there sufficient off-ramps or sunsets to ensure that funding is reallocated within the program or to other programs if the program does not make adequate progress towards one or more of its goals and objectives? Are the program’s plans for independent and/or external reviews adequate and appropriate?
-
Are appropriate scientific and technical objectives being posed, taking into consideration program goals, NASA's strengths, and the time horizon for the project? Are critical personnel and facilities required to support the program well defined?
Connections to the Broader Community
-
What programs or program elements should be performed in-house at NASA and be exempt from competition with industry or academia?
-
Is there evidence that the research plan for the area under review reflects a broad understanding of the underlying science and technology and of comparable work within other NASA units as well as industry, academia, and other federal laboratories?
-
Is there evidence that the research builds appropriately on work already done elsewhere? Does it leverage the work of leaders in the field? Is the strategy for out-of-house work (competitions, partnerships, etc.) well chosen and managed?
-
Is the research being accomplished with a proper mix of personnel from NASA, academia, industry, and other government agencies? Is the program using high-quality research performers or is there untapped talent outside the program that can be brought to bear?
Methodology
-
How well crafted are the research plans for the areas under review? In general, is the use of laboratory experiment, modeling, simulation, and/or field test appropriate? How well are these methods integrated?
-
Have the appropriate supporting system-level assessments been conducted?
-
Do both the researchers and managers understand and manage the risks involved to an appropriate level?
-
Are the plans for further study reasonable and justifiable?
Overall Capabilities
-
Is the scientific or engineering quality of the work (including work performed in academia and industry) comparable to similar world-class efforts at other institutions, and is it appropriate for the goal?
-
Are the qualifications of the scientific and engineering staff (including researchers in academia and industry) sufficient to achieve program goals?
-
Are the capabilities, quantity, and state of readiness of equipment and facilities sufficient to achieve program goals?
-
Are personnel, equipment, and facilities supplied by support contractors used efficiently; do they fill gaps in government capabilities without duplication?
The selection of criteria for each assessment and the relative weights given to each criterion are within each panel's discretion and can vary from program to program. Neither the committee nor the panels will make explicit budget recommendations to NASA, but will instead comment on program content, gaps in technology, and other issues outlined above.