1. Establishment of appropriate base cases. R&D programs should be compared to a base case of DOE's current R&D budget, which may be zero for a particular technology. One option that must be considered is that entities other than DOE will do the R&D; for example, in this case, private transportation companies or foreign companies or governments. One should also consider the alternative case in which no R&D is done but similar options such as conventional rail high-speed trains using current or incremental improvements on current technologies are built, or that foreign or industrial R&D efforts will produce the needed technologies.

2. Varying levels of DOE effort. How much would the proposed program be slowed down if its budget were reduced? Is there a minimum viable level of effort to make the R&D worthwhile?

3. Double counting alternative R&D approaches. For example, will advanced vehicle R&D reduce the number of gallons of gasoline that can be saved by MAGLEV?

4. Alternative NEMS scenarios. Will the value of MAGLEV change radically under different economic, technical, or policy scenarios?

In summary, then, these case studies illustrate the following points:

• Important policy options including R&D portfolio planning should be subjected to consistent scenario discipline.

• NEMS can be of value in establishing parameters for this consistency.

• Program offices must have independent analytical capabilities.

• NEMS should integrate additional detail only as the repetitive need is demonstrated, but it should be improved to remain consistent with detailed analysis once it is validated.

• NEMS modules should be designed to accommodate added detail, but all detail must not be added to NEMS and NEMS may have a role in analysis where off-line analyses are also needed.