Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 55
Review of the U.S. Department of Defence Air, Space, and Supporting Information Systems Science and Technology Program Appendix D Milestones in the Management of DoD Science and Technology Research on air science and technology, begun on a small scale prior to the establishment of the National Advisory Committee for Aeronautics in 1916, was increased dramatically just before and during World War II and the subsequent transition to jet propulsion in the 1950s. Space science and technology were predominant in the 1960s and 1970s. Information systems science and technology grew exponentially in the 1980s and 1990s. Many of the outstanding issues concerning the management of U.S. Department of Defense science and technology programs are not new. Indeed, some issues have recurred over the past 50 years. The following chronological summary covers the most significant studies and actions that have affected DoD S&T since the end of World War II. After the Manhattan Project’s success in developing the atomic bomb during World War II, DoD was convinced of the value of investing in S&T. The subsequent development of aircraft, precision bombing, proximity fuzing, radar, and many other technologies has justified that belief in the great payoffs of S&T. Vannevar Bush, a leader of science and technology during the war, advocated continuing strong peacetime support of the S&T program by the federal government in his study, Science, the Endless Frontier (Bush, 1945). Another advocate was Theodore von Karman, a world expert in aerodynamics and an early developer of rocket propulsion. Under the direction of General Henry H.(Hap) Arnold, Commanding General of the Army Air Forces, von Karman formed a team of outstanding scientists that traveled wartorn Europe and Japan and gathered enough information to fill 32 volumes. The first volume, Toward New Horizons, was delivered in December 1945 (von Karman, 1945). The needs of a technologically superior military identified by von Karman are still applicable (Box D-1). Even though Congress consistently slashed budgets for S&T from 1945 to 1957, the studies by Bush and von Karman set an agenda for decades to come. An excellent discussion of von Karman and other significant Air Force S&T studies can be found in Harnessing the Genie (Gorn, 1988). The launching of Sputnik on October 4, 1957, “the first artificial satellite in space,” provided an impetus Box D-1 Summary of von Karman’s Recommendations S&T must permeate the entire command structure. Officers must be educated in S&T. Officers working in S&T must have the same promotional opportunities as line operational officers. Management of research and development must be separated from procurement. Military S&T must have its own facilities, staff, and funding. A scientific advisory group must be established and empowered at the highest level. SOURCE: von Karman, 1945.
OCR for page 56
Review of the U.S. Department of Defence Air, Space, and Supporting Information Systems Science and Technology Program for an S&T program. Congress quickly reacted to the wake-up call by creating the Advanced Research Projects Agency (P.L. 85–325), later the Defense Advanced Research Projects Agency (DARPA), to manage U.S. space programs. Shortly thereafter, Congress realized that space operations should not be under the military and created the National Aeronautics and Space Administration (NASA), which incorporated existing facilities and research programs operated by the National Advisory Committee for Aeronautics. Defense against ballistic missiles, the detection of nuclear tests, the development of new sources of electric power, and the development of rocket propulsion were assigned to the Advanced Research Projects Agency. When John F.Kennedy came into office in 1961, he placed a new emphasis on space S&T by setting a goal of landing a man on the moon “by the end of the decade.” He also ordered a restructuring of military S&T programs. Charles Hitch, DoD comptroller, reorganized the research, development, test, and evaluation (RDT&E) budget and established the planning, programming, and budgeting system, which was the first time program elements were used as the basic budget building blocks. The so-called “Hitch Package” was based on a series of studies by Hitch at the RAND Corporation (Hitch and McKean, 1960). Budget categories, for 6.1 Research, 6.2 Exploratory Development, and 6.3 Advanced Development, were created. Various combinations of 6.1, 6.2, and 6.3 (and later 6.3A) and different names have been used over the years. The current RDT&E budget categories are defined in Box D-2. Based on studies and actions in the late 1960s and early 1970s, more controls were placed on S&T. Project Hindsight, a detailed study of the payoff of S&T, evaluated the contributions of 6.1 and 6.2 for the Bullpup, Honest John, Lance, Minuteman I, and Minuteman II missile systems, the C-141 cargo aircraft, and the Mark 56 and 57 naval mines. The study concluded that new technologies were often developed to meet specific detailed requirements. Although basic research was necessary to define potential technologies, oriented research was crucial to the development of useful systems (Isenson, 1967). Congress and the services reacted quickly to the preliminary findings of Project Hindsight (which were available in 1965 and 1966). At the same time, Senator Mike Mansfield tacked on to the 1966 appropriations bill the so-called Mansfield Amendment requiring that all S&T proposals include a written statement of future Box D-2 Current S&T Budget Activities Budget Activity 1, Basic Research. Basic research is defined as systematic study directed toward greater knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications towards processes or products in mind (6.1). Budget Activity 2, Applied Research. Applied research is defined as systematic study to gain knowledge or understanding necessary to determine the means by which a recognized and specific need may be met (6.2). Budget Activity 3, Advanced Technology Development. Includes all efforts that have moved into the development and integration of hardware for field experiments and tests (6.3). SOURCE: OMB, 2000. military applications showing “direct” and “apparent” relevance. This requirement was particularly difficult to meet for 6.1 projects, basic research (Sullivan and Heaston, 1967). To respond to Project Hindsight, the services created the Blue Ribbon Defense Panel, which made the following observation: There is no adequate mechanism to assure that funds appropriated for research and exploratory development are not diverted to advanced, or engineering development categories, or to operational systems developments. The overemphasis on mission justifications for research and development allocations and funding creates additional incentives for such diversions. (BRDP, 1970) The services then transferred all 6.1 projects into single-program funding and 6.2 projects into single-program-element funding. At the same time, many projects were combined, priorities were changed, and laboratory directors were given more latitude to manage their programs. Combining programs, however, eventually made them more vulnerable to micro-management by service headquarters, the Office of the Secretary of Defense, and Congress. In response to the Blue Ribbon Defense Panel, Secretary of Defense Melvin Laird ordered a massive reorganization and increased the emphasis on prototypes to test new technologies (see Box D-3) (Foster, 1971).
OCR for page 57
Review of the U.S. Department of Defence Air, Space, and Supporting Information Systems Science and Technology Program Box D-3 Rationale for 1971 Prototype Initiative My basic intention is to place the subject of prototyping in a proper perspective as a tool to be prudently applied. We need to make maximum effective use of the smaller defense budget. To keep costs down, we must try things out before making heavy commitments of resources. We need to find ways to improve reliability and reduce the anticipated maintainability costs of our defense systems before they are deployed. We need to keep technological innovation moving ahead, recognizing there is a continuing threat to our country’s technological leadership. We need to keep some of our design teams together so we won’t have to start over each time a new program is initiated. SOURCE: Excerpted from Foster, 1971. Although prototypes contributed greatly to the modernization of the services during the 1970s, the cost of hardware demonstrations put a great strain on the 6.3 advanced development budget; therefore, 6.3 was split into 6.3A nonsystems advanced development and 6.3B systems advanced development. Until this split, the S&T base (or technology base) was defined as 6.1 and 6.2. After the split, the technology base became 6.1, 6.2, and 6.3A. This structure worked very well throughout the 1970s and early 1980s, and many new weapon systems were fielded under the so-called modernization programs of the services. In 1981, the Defense Science Board Panel on Technology Base was asked to conduct a study on three major aspects of DoD S&T: identifying critical technologies, working closely with users to accelerate the transition of new technologies, and maintaining high-quality in-house personnel and facilities. The following sections address these three themes. CRITICAL TECHNOLOGIES Numerous attempts have been made to establish priorities for the technologies that would meet DoD needs. The Defense Science Board report identified 17 so-called order-of-magnitude technologies for which funding should be increased by an order of magnitude. In 1983, the Strategic Defense Initiative was created, causing a major shuffle of S&T throughout DoD. Then came “Technologies for Competitive Strategies” in 1987. In 1988, Congress mandated that DoD submit an annual critical technologies plan (P.L. 100–456). The first plan, which listed 22 critical technologies, was submitted on March 15, 1989 (DoD, 1989). In response to Congress’s request, another plan was submitted on March 15, 1990, with a list of 20 technologies (Box D-4; DoD 1990). In the National Defense Authorization Act for Fiscal Year 1990 (FY90) and for FY91 (P.L. 101–189), Congress asked for a critical technologies plan with a national list of the DoD’s and the U.S. Department of Energy’s priorities (DoD, 1991). The final list included 22 technologies grouped into three categories: pervasive technologies, enabling technologies, and emerging technologies (NCTP, 1991). Box D-4 1990 DoD Critical Technologies Pervasive Technologies composite materials computational fluid dynamics data fusion passive sensors photonics semiconductor materials and microelectronic circuits signal processing software producibility Enabling Technologies air-breathing propulsion machine intelligence and robotics parallel computer architectures sensitive radars signature control simulation and modeling weapon system environment Emerging Technologies biotechnology materials and processes high-energy-density materials hypervelocity projectiles pulsed power superconductivity SOURCE: DoD, 1990.
OCR for page 58
Review of the U.S. Department of Defence Air, Space, and Supporting Information Systems Science and Technology Program In 1992, DoD issued Defense Science and Technology Strategy, which announced seven S&T thrusts funded under 6.3A based on the “demands being placed on the S&T program by the users’ most pressing military and operational requirements” (DDR&E, 1992a). The 6.2 program was addressed in the separate DoD Key Technologies Plan (DDR&E, 1992b). In a parallel action, the Deputy Secretary of Defense asked the services to recommend a new approach to the management of S&T projects. The services began formal discussions on ways to strengthen interservice cooperation in their RDT&E programs and increase the use of each other’s facilities. This project, called Tri-Service S&T Reliance, was the most comprehensive restructuring of the technology base in more than 40 years. As described in a white paper (JDL, 1992), the services agreed on a taxonomy of 28 technology areas, subareas, and sub-subareas, a total of 223 technology topics. Panels were appointed for basic research and 12 technology areas. The Office of the Secretary of Defense evaluated the approach during 1993 and developed a technology area review and assessment (TARA) process in 1994. TARA established 10 DoD reliance panels and a research panel to evaluate the DoD S&T program (Box D-5), as well as an annual week-long TARA meeting, including nongovernment personnel, for comments on the DoD S&T program. The TARA process is still being used, along with a number of supporting plans: the Basic Research Plan, the Defense Technology Area Plan, the Joint Warfighting Science and Technology Plan, and the Defense Technology Objectives document (DUSD (S&T), 1999). Box D-5 Current TARA Technology Areas air platforms chemical and biological defense information systems technology ground and sea vehicles materials/processes biomedical technologies sensors, electronics, and battlespace environment space platforms human systems weapons nuclear technology The 1999 Defense Science Board Summer Study Task Force decided to take a strategic approach, focusing not on critical technologies but instead on three specific enablers: strategic agility, force protection, and information for decision superiority (DSB, 1999). The study approach was more parametric than usual and provided guidance that will have long-term value. This study may be a model for future evaluations of the technology base. TECHNOLOGY TRANSITION A prevailing theme in studies about the technology base is the question of how to transition the results of S&T projects into a military product. Despite the general desire to get new technology into the hands of the user as soon as possible, how to make the transition from a concept to a product seems to be little understood. In the case of DoD S&T, the budget is key to the process. In other words, improvements in the acquisition process, including the acquisition of S&T, must initially be implemented through the budgetary process. Figure D-1 depicts the S&T research and budget categories 6.1 through 6.3 and two system development categories: 6.4 (demonstration/validation) and 6.5 (engineering and manufacturing development). To some extent, the figure suggests a linear process by which basic and applied research provide the foundation for the development and demonstration of advanced technologies, which, in turn, enable the development of new systems or system capabilities. As a depiction of the development of any specific system, however, Figure D-1 is oversimplified. An air or space system application rarely, if ever, depends on only one 6.1 result, and basic research, sometimes done decades earlier, does not necessarily anticipate the defense systems it will enable. The process is not necessarily linear. Difficult problems may force a return from 6.3 to 6.2, and, if necessary, from 6.2 to 6.1. Other projects may jump ahead and skip stages. There may or may not be a distinguishable system-level prototype prior to the decision to enter full-scale system development. System operations may even need to call upon basic research products to solve operational problems. So, although Figure D-1 depicts a model of the relationship between the stages or categories of R&D, the actual transition of research results and technologies into systems can be much more complicated than the figure implies.
OCR for page 59
Review of the U.S. Department of Defence Air, Space, and Supporting Information Systems Science and Technology Program FIGURE D-1 RDT&E budget categories. SOURCE: Neighbor, 1999. Project Hindsight, the prototyping initiative, and a Defense Science Board study (DSB, 1981) all recognized the value of a close coupling of 6.3, prototypes, and the user. A 1986 Blue Ribbon Commission on Defense Management reinforced the need for more investment in prototypes, particularly by DARPA (CDM, 1986). A Defense Science Board study in 1987 endorsed the use of technology demonstrations, which were called advanced technology transition demonstrations (ATTDs), and recommended that at least half of 6.3A funding should be used to support ATTDs (DSB, 1987). In response to these recommendations, DoD initiated advanced concept technology demonstrations (ACTDs), so-called system-of-systems demonstrations. The planning, intermediate testing, and full field testing of ACTDs were supported by active military personnel. After it was demonstrated, ACTD equipment was left with the troops to use. In a short time, most of 6.3A was dedicated to ACTDs, and pressure was put on 6.2 to feed 6.3A. There was a trickle-down effect on 6.1, forcing most projects to adopt a short-term focus. The pressure to use commercial off-the-shelf components in developmental and fielded systems accelerated this trend (DSB, 1989). In 1998, DoD abolished the distinction between 6.3A and 6.3B and established 6.3 only. Some 6.3B programs moved into 6.4, demonstration and validation. It seems paradoxical that 6.3A and 6.3B were separated in 1972 in response to the DoD prototype initiative and that the reverse process then occurred in 1998 after the big push for ACTDs in the early 1990s. PEOPLE AND FACILITIES Many studies over the years have focused on how DoD is organized, how it manages its people, how it obtains and uses its facilities, and how it conducts the acquisition of weapons and materiel. Some of these studies and consequent changes have had a direct impact on the S&T program. All of them have had at least an indirect impact. The need for higher salaries for personnel in senior positions, adequate staffing of technical positions, technically trained military personnel in RDT&E positions, greater freedom for laboratory directors, and reorganization of the acquisition process were addressed in a report to the President in 1962 (U.S. Bureau of the Budget, 1962). The trend-setting 1981 Defense Science Board study recommended wider adoption of the approach used by the Naval Ocean Systems Center and the Naval Weapons Center (DSB, 1981). A report by the Under Secretary of Defense (Research and Engineering), a detailed overview
OCR for page 60
Review of the U.S. Department of Defence Air, Space, and Supporting Information Systems Science and Technology Program of DoD laboratories, recommended the upgrading of personnel practices, streamlining of procurement practices, modest increases in the rate of modernization of facilities and equipment, and improvements in DoD/ university relationships (USD (R&E), 1982). A report by the White House Science Council’s Federal Laboratory Review Panel, which focused on conditions throughout the federal laboratory system, made the following recommendations: the missions of the laboratories must be clarified; laboratories must be held accountable for the quality and productivity of S&T projects; and constraints on laboratories must be relieved with regard to personnel administration (WHSC, 1983). This list of areas for improvement has not changed much over the years. The Goldwater-Nichols Department of Defense Reorganization Act of 1986 (P.L. 99–433), as well as the President’s Blue Ribbon Commission on Defense Management study of 1986, had a dramatic impact on the DoD acquisition process (CDM, 1986). This was only the third major legislative action since World War II involving the organization and mission of the DoD. At the time of the Goldwater-Nichols Act, DoD was experiencing extensive growth, particularly in RDT&E funding. Changes made at the time, such as changes in the acquisition executive and program executive officer chains of command, a dual chain-of-command arrangement, have imposed a substantial burden on the much-downsized system of today. In 1988, the Office of Technology Assessment conducted a comprehensive study of the technology base over the previous 20 years (OTA, 1988). A DoD task force, working under the Institute for Defense Analyses, supported this study (IDA, 1988). Volume I of that report identified and summarized 22 other studies conducted between 1963 and 1988, especially the Defense Science Board studies of 1981 (an independent review of DoD laboratories) and 1987 (DSB, 1981, 1987). The Office of Technology Assessment report highlighted Congress’s three major concerns: the apparently lengthening time of transferring laboratory advances into effective, dependable fielded systems (Box D-6); declining U.S. leadership in vital high-technology industries; and a downward trend in the proportion of the defense budget devoted to the technology base. The Defense Science Board study of 1998 was a natural follow-on to the 1988 Office of Technology Assessment study. One conclusion of the Defense Science Board study—“DoD’s technology base is threatened by an unstable budget and an inability to attract and retain Box D-6 Transitioning Technology Many experts believe that the long delays in getting new technology into the field arise not in the technology base, but in the subsequent programs that translate the products of the technology base into new systems. Full-scale development and production times are increasing, and the longer it takes to build a system, the older the technology will be when it finally reaches the field. SOURCE: Excerpted from OTA. 1988, p. 6. Box D-7 Selected Results of 1988 OTA Study Personnel Observers in government and industry believe that DoD is finding it increasingly difficult to attract and keep the skilled management personnel necessary to the functioning of its technology base programs. This appears to be, at least in part, a result of Civil Service salary structures and Congress’ efforts to limit the movement of personnel between industry and DoD. Stability of S&T Funding Funding for technology base programs is particularly vulnerable during times of tight budgets. The rapid spend-out rates of technology base programs means that cuts in R&D go farther towards reducing deficits than similar size cuts in procurement programs. And the lack of obvious, tangible outputs from R&D projects makes the value of individual programs difficult to define. Technology base programs are particularly vulnerable to “raiding” to support programs in procurement or the later stages of development. Congress will have to determine what it thinks are proper levels of funding, which may entail acting as an advocate for technology base funding when DoD seeks to reduce it. The optimal level of funding is difficult, if not impossible, to gauge accurately. However, funding that fluctuates widely from year to year is inefficient and can be very disruptive. Congress faces the very difficult decision of whether it should be actively involved in the selection of technology base programs and the determination of specific funding levels, or whether instead it should give DoD managers wide latitude to construct programs within agreed overall funding levels. SOURCE: Excerpted from OTA, 1988, p. 5.
OCR for page 61
Review of the U.S. Department of Defence Air, Space, and Supporting Information Systems Science and Technology Program Box D-8 Funding Level for S&T No formula was discovered for establishing the optimum level of DoD investment in science and technology, but the most successful industries invest about 15 percent of sales in research and development with about 3.5 percent of sales invested in research (equivalent to DoD S&T program). This would imply that, currently, DoD should invest at least $8 billion in S&T. SOURCE: Excerpted from DSB, 1998, p. 3. high-quality scientists and engineers in laboratories and R&D centers”—echoed a conclusion of the Office of Technology Assessment report (Box D-7). With the fall of the Berlin Wall in 1989 and the subsequent drawdown of military forces and programs, the findings, conclusions, and recommendations of these studies became critical for the DoD S&T base. In 1992, the Defense Conversion Commission described the impact of the drawdown on civilian and military operations (DCC, 1992). The Morrow Defense Science Board Task Force highlighted the challenges for the future of the S&T program in two primary recommendations: the DoD S&T program should be funded at $8 billion dollars a year (Box D-8), and the majority of DoD S&T management and executive technical positions should be staffed by individuals from the private sector under the Intergovernmental Personnel Act (P.L. 91–648) and the reinstated War and Navy Departments-Professional and Scientific Service Act (P.L. 313) (DSB, 1998). A broader approach to RDT&E staffing by DoD was described in a 1999 report on streamlining the entire RDT&E infrastructure (DoD, 1999). REFERENCES BRDP (Blue Ribbon Defense Panel). 1970. Blue Ribbon Defense Panel Report to the President and the Secretary of Defense, July 1, 1970. Washington, D.C.: U.S. Government Printing Office. Bush, V. 1945. Science, the Endless Frontier. New York: Arno Press. Reprinted 1980. CDM (Commission on Defense Management). 1986. A Quest for Excellence. Final Report to the President by the President’s Blue Ribbon Commission on Defense Management, June 1986. Washington, D.C.: President’s Blue Ribbon Commission on Defense Management. DCC (Defense Conversion Commission). 1992. Adjusting to the Drawdown. Report of the Defense Conversion Commission, December 31, 1992. Washington, D.C.: Defense Conversion Commission. DDR&E (Director, Defense Research and Engineering). 1992a. Defense Science and Technology Strategy, July 1992. Washington, D.C.: U.S. Department of Defense. DDR&E (Director, Defense Research and Engineering). 1992b. DoD Key Technologies Plan, July 1992 . Washington, D.C.: U.S. Department of Defense. DoD (U.S. Department of Defense). 1989. The Department of Defense Critical Technologies Plan. Submitted to the Committees on Armed Services of the United States Congress, March 15, 1989 (revised May 5, 1989). Washington, D.C.: U.S. Department of Defense. DoD. 1990. The Department of Defense Critical Technologies Plan. Submitted to the Committees on Armed Services of the United States Congress, March 15, 1990. Washington, D.C.: U.S. Department of Defense. DoD. 1991. The Department of Defense Critical Technologies Plan. Submitted to the Committees on Armed Services of the United States Congress, March 15, 1991. Washington, D.C.: U.S. Department of Defense. DoD. 1999. A Plan to Streamline DoD’s Science and Technology, Engineering, and Test and Evaluation Infrastructure. Report of the Section 907 and Section 912(c) Senior Steering Group for Review of the RDT&E Infrastructure, July 1999. Washington, D.C.: U.S. Department of Defense. DSB (Defense Science Board). 1981. Report of the Defense Science Board Summer Study Panel on the Technology Base, November 1981. Washington, D.C.: Defense Science Board. DSB. 1987. Report of the Defense Science Board 1987 Summer Study on Technology Base Management, December 1987. Washington, D.C.: Defense Science Board. DSB. 1989. Use of Commercial Components in Military Equipment. Report of the Defense Science Board, June 1989. Washington, D.C.: Defense Science Board. DSB. 1998. Defense Science and Technology Base for the 21st Century. Report of the Defense Science Board Task Force, June 1998. Washington, D.C.: Defense Science Board. DSB. 1999. 21st Century Defense Technology Strategies. Report of the Defense Science Board 1999 Summer Study Task Force, November 1999. Washington, D.C.: Defense Science Board. DUSD (S&T) (Deputy Under Secretary of Defense (Science and Technology)). 1999. Defense Science and Technology Plans, February 1999. Washington, D.C.: U.S. Department of Defense. Foster, J.S., Jr. 1971. Prototyping: The “In Thing,” presentation by J.S. Foster, director of Defense Research and Engineering, to the Industry Advisory Committee, Pentagon, Arlington, Virginia, June 11, 1971. Gorn, M.H. 1988. Harnessing the Genie: Science and Technology Forecasting for the Air Force, 1944–1986. Washington, D.C.: Office of Air Force History. Hitch, C.J., and R.N.McKean. 1960. The Economics of Defense in the Nuclear Age. Cambridge, Mass.: Harvard University Press. IDA (Institute for Defense Analyses). 1988. Dierolf, D.A., P.H.Richanbach, K.J.Richter, and F.R.Riddell, editors. Report of the Task Force for Improved Coordination of the DoD Science and Technology Program. Volume I: Summary Report and Recommendations. Volume II: Reports of the Working Groups. IDA R-345, Alexandria, Va.: Institute for Defense Analyses, August. Isenson, R.S. 1967. Project Hindsight: Final Report on Task I, July 1, 1967. Washington, D.C.: Office of the Director of Defense Research and Engineering. JDL (Joint Directors of Laboratories). 1992. White Paper on Tri-Service Reliance in Science and Technology, January 1992. Washington, D.C.: Joint Directors of Laboratories. NCTP (National Critical Technologies Panel). 1991. Report of the National Critical Technologies Panel, March 1991. Washington, D.C.: Office of Science and Technology Policy. Neighbor, T. 1999. AFRL Vision, presentation by T.Neighbor, Air Force Research Laboratory, director, Plans and Programs, to the Committee on Review of the Department of Defense Air and Space Systems Science and Technology Program, National Research Council, Washington, D.C., December 17, 1999.
OCR for page 62
Review of the U.S. Department of Defence Air, Space, and Supporting Information Systems Science and Technology Program OMB (White House Office of Management and Budget). 2000. OMB Circular A-11: Preparing and Submitting Budget Estimates, July 19, 2000. Washington, D.C.: White House Office of Management and Budget. OTA (Office of Technology Assessment). 1988. The Defense Technology Base: Introduction and Overview. A Special Report, March 1988. OTA-ISC-374. Washington, D.C.: U.S. Government Printing Office. Sullivan, T.E., and R.J.Heaston. 1967. Relevance of European Research in Chemistry and Materials. Frankfurt, FRG: U.S. Army Research and Development Group (Europe), May. USD (R&E) (Under Secretary for Defense (Research and Engineering)). 1982. USD (R&E) Independent Review of DoD Laboratories. Prepared for Under Secretary for Defense Research and Engineering, March 22, 1982. Washington, D.C.: U.S. Department of Defense. U.S. Bureau of the Budget. 1962. Report to the President on Government Contracting for Research and Development. Washington, D.C.: Office of the White House Press Secretary, von Karman, T. 1945. Toward New Horizons. Washington, D.C.: Office of Air Force History. WHSC (White House Science Council). 1983. Report of the White House Science Council. Washington, D.C.: Office of Science and Technology Policy, Executive Office of the President.
Representative terms from entire chapter: