The parallel implementation of the committee and CATE processes, shown in Figure 7.1, allowed for timely and efficient data gathering and fact finding by the CATE contractor and the committee while maintaining the independence of each activity. As one of the first activities of the survey, before the CATE process was fully developed, the committee solicited Notices of Intent (NOI) to gauge the kinds of research activities it could expect to have to assess during the course of the survey. This first step was followed by receipt of white papers and then two request-for-information cycles (RFI-1 and RFI-2), resulting in multiple submittals from candidate activities. The output of the RFI-2 process was the selection of candidates to be put forward for detailed CATE process analysis. The proposed candidates were selected by the Program Prioritization Panels (PPPs) based on scientific priorities together with a scientific evaluation of the technical approaches. The candidates were approved by the committee.

The CATE component of the process was iterative in the early stages, starting with a technical evaluation of the selected candidates and then proceeding to follow-up questions to individual project teams as required. The CATE and survey processes were linked, through direct communication between the contractor and committee and panel members, as well as presentations to the committee and PPPs. The interactions focused on ensuring the quality of the assessments by the contractor and engaging the technical expertise of the panels and the committee. Discussions between the PPPs and the cost contractor were essential to ensure that project details were not misinterpreted by the contractor. Intermediate results were then presented to the full committee in October 2009 at the committee’s fourth meeting, followed by several more iterative steps focused on reviewing the final assessments and appraisals for accuracy, realism, and consistency by the committee.

Despite the considerable interaction with the committee and panels, the survey process maintained the independence of the contractor so that its final analysis was free from undue influence by either the committee itself or by interests outside the survey. This independence was accomplished by establishing the contractor as a consultant to the National Research Council rather than a direct participant in the committee effort. Therefore, although the committee worked closely with the contractor to provide technical inputs as requested, as well as expert review and commentary, the final result was accepted and certified as independent work performed by the contractor alone. Equally important to the independence of the contractor was the committee’s responsibility for reviewing the contractor’s work and exercising its judgment in accepting the contractor’s results.

A second essential consideration affecting the CATE process was the recognition that ground-based and space-based systems are fundamentally different with respect to how they are funded and developed. This disparity profoundly influenced the methods by which the ground and space systems were evaluated and validated by the contractor. The space-based systems were evaluated statistically using the process



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement