Chapter 1

Program, Curriculum, and Syllabi Review

BACKGROUND

The call to arms for increased educational scientific opportunities for all Americans has been sounded by many parties, including the members of the National Research Council’s (NRC’s) Committee on Prospering in the Global Economy of the 21st Century, characterized by Thomas Friedman as a “nonpartisan group of America’s most distinguished engineers, scientists, educators and industrialists” (Friedman, 2010). These opportunities need to be extended not only to students in the normal educational pipelines but also to veterans and those in active military service. There is a clear need for increased training and education in the sciences, technology, engineering, and mathematics (STEM) at all levels in the United States. This need, so clearly elucidated by President Barack Obama in his "Educate to Innovate" campaign for excellence in STEM education in November 2009 (White House, 2009), is also faced by the U.S. intelligence community. As the world becomes more technologically advanced, the need for intelligence officers and analysts skilled in science and technology intelligence (S&TI) increases.

The committee agrees that increased S&TI capability is critical given the current and future threat environment. It offers several reasons why the United States needs analysts and intelligence officers with S&TI skills:

  • The increased speed of science and technology breakthroughs;
  • The globalization of science and technology (S&T);
  • The convergence of various S&T disciplines (computer science, biology, physics, neuroscience, nanotechnology, chemistry);
  • The impact of commercial technology and its speed of dissemination; and
  • The increased capabilities of potential adversaries, including both non-state and state actors and the willingness of these parties to share with or sell to one another.

Returning veterans of the wars in Afghanistan and Iraq require additional education and training so that their experiences and skills are not lost to the nation. In light of these needs, the National Defense Intelligence College (NDIC) has created a new program in S&TI education. This effort includes the opening of a new School of Science and Technology Intelligence and the development of new educational programs (Studds et al., 2011; NDIC, 2011a).

The NRC was asked to convene a Committee for the Review of the Master’s Degree Program for Science and Technology Intelligence Professionals to consider a proposed master of science degree in S&TI. The statement of task associated with this request reads as follows:

To provide a national level perspective to the National Defense Intelligence College (NDIC) as it plans for a new master’s degree program scheduled to start



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 3
Chapter 1 Program, Curriculum, and Syllabi Review BACKGROUND The call to arms for increased educational scientific opportunities for all Americans has been sounded by many parties, including the members of the National Research Council's (NRC’s) Committee on Prospering in the Global Economy of the 21st Century, characterized by Thomas Friedman as a "nonpartisan group of America's most distinguished engineers, scientists, educators and industrialists" (Friedman, 2010). These opportunities need to be extended not only to students in the normal educational pipelines but also to veterans and those in active military service. There is a clear need for increased training and education in the sciences, technology, engineering, and mathematics (STEM) at all levels in the United States. This need, so clearly elucidated by President Barack Obama in his "Educate to Innovate" campaign for excellence in STEM education in November 2009 (White House, 2009), is also faced by the U.S. intelligence community. As the world becomes more technologically advanced, the need for intelligence officers and analysts skilled in science and technology intelligence (S&TI) increases. The committee agrees that increased S&TI capability is critical given the current and future threat environment. It offers several reasons why the United States needs analysts and intelligence officers with S&TI skills: The increased speed of science and technology breakthroughs; The globalization of science and technology (S&T); The convergence of various S&T disciplines (computer science, biology, physics, neuroscience, nanotechnology, chemistry); The impact of commercial technology and its speed of dissemination; and The increased capabilities of potential adversaries, including both non-state and state actors and the willingness of these parties to share with or sell to one another. Returning veterans of the wars in Afghanistan and Iraq require additional education and training so that their experiences and skills are not lost to the nation. In light of these needs, the National Defense Intelligence College (NDIC) has created a new program in S&TI education. This effort includes the opening of a new School of Science and Technology Intelligence and the development of new educational programs (Studds et al., 2011; NDIC, 2011a). The NRC was asked to convene a Committee for the Review of the Master’s Degree Program for Science and Technology Intelligence Professionals to consider a proposed master of science degree in S&TI. The statement of task associated with this request reads as follows: To provide a national level perspective to the National Defense Intelligence College (NDIC) as it plans for a new master’s degree program scheduled to start 3

OCR for page 3
4 Review of the NDIC’s Master’s Degree in S&TI in the fall of 2011, an NRC committee will review a draft curriculum and associated materials developed by the NDIC. The committee will produce a letter report offering advice and recommendations regarding the suitability of the draft curriculum as a basis for producing desired learning outcomes for intelligence professionals who participate in the proposed Master of Science and Technology Intelligence degree program in the School of Science and Technology Intelligence. The NDIC is an institution of higher education accredited by the Middle States Commission on Higher Education. It is authorized by the U.S. Congress to award both bachelor’s and master’s degrees, to conduct and disseminate research related to national security and intelligence activities, and to engage in outreach activities. The NDIC currently offers a bachelor of science in intelligence (BSI) and a master of science in strategic intelligence (MSSI). It also offers certificate programs in various regional and topical study areas. One such specialization is a certificate in foreign denial and deception (Studds et al., 2011). The NDIC serves both military and civilian members of the intelligence community. It is currently housed at the Defense Intelligence Agency (DIA) headquarters facility in Washington, D.C. Created in 1962 as the Defense Intelligence School, the NDIC has evolved in response to the emerging needs of its constituency and as directed by its board of visitors (NDIC, 2010; NDIC, 2011a). Most recently, in 2010, it was directed to create a school of science and technology intelligence and to develop a master of science and technology intelligence degree program (NDIC, 2011a). The curriculum under review by the committee is the material developed in response to that directive. When a new degree program is developed by the NDIC, the program must be shown to meet four distinct criteria. First, it must be shown to be necessary. In other words, the degree program must meet an unmet need that is essential for the execution of the applying agency’s mission. Second, it must be shown to be unique. It cannot duplicate existing programs that could be satisfactorily used to meet the identified need. Third, it must meet the standards for graduate degree programs met by non-federal institutions. And finally, it must allow for academic freedom for the faculty and the students so that research and classroom activities are free from undue influence and are unbiased by either job or mission (Studds et al., 2011, p. 8). From May to August 2011, the committee met and considered the material presented by the NDIC. A committee meeting was held in May, when materials were presented and the committee was able to question the faculty and administrators of the NDIC. After the meeting with the faculty and administrators, additional material was collected to provide a comparative sampling of other curricula. Because this is a unique offering in a unique setting, there is no exact replica to examine. However, curricula similar in scope and nature were reviewed, representatives of the client population were interviewed, and the program goals were examined. Detailed review comments regarding the curriculum in general and the syllabi provided to the committee are included in the next sections of this chapter. The committee’s discussion first reflects an overview of what was provided to the Committee and then the findings and recommendations follow. DISCUSSION OF THE MS&TI DEGREE Curriculum A review of the curriculum reveals that it is both unique and necessary. Comments from current members of the S&TI community revealed a true dearth of structured educational programs to create leaders in S&TI for the intelligence community. There is a distinct difference in the educational needs of S&TI leaders and S&TI analysts. Whereas analysts are required to be

OCR for page 3
Program, Curriculum, and Syllabi Review 5 deeply knowledgeable about one or more subject areas of science or technology, leaders are required to have a broad appreciation for S&TI as a whole, to understand how S&TI contributes to national security, and to know how to be effective managers within the structure of the U.S. government. It was noted that the craft of S&TI is very different from simply being a scientist, engineer, or technologist. The practitioner of S&TI at all levels needs to understand not only the underlying science but also how potential adversaries use science, both systematically and culturally. This fundamental distinction leads to a substantive difference between simply learning a science and learning S&TI. As such, this discussion of the proposed degree begins with the curriculum. The mission of the master of science and technology intelligence (MS&TI) degree program falls within the greater mission of the NDIC, which is to provide educational programs to members of the U.S. intelligence community. As a program funded by the national intelligence community, NDIC’s MS&TI program has both a professional and an academic duty to enrich the skill sets of intelligence professionals, with a specific focus on improving the nation’s ability to collect, analyze, and provide intelligence product in the areas of foreign science and technology development. Through its core courses and concentrations, the MS&TI program sets out to address some of the challenges identified in the following statement: . . . our ability to identify emerging technologies and warn about the disruptive nature of foreign developments through Scientific and Technical Intelligence (S&TI) must have a broader, worldwide focus that builds upon the traditionally peer and near-peer competitors but extends beyond to the world of non-traditional sources of technology and asymmetric adversaries. (NDIC, 2011a) Specifically, the mission of the S&TI community is to “address threats to national security arising from globalization of science and technology; identify disruptive consequences of adversarial technology adaptations; and provide a framework for effective collection and warning (Studds et al., 2011).” It follows, therefore, that a master’s degree in this area should prepare students to both contribute to the greater S&TI mission and adopt leadership roles in the community of practitioners. And in fact, the mission for this program was clearly identified as developing leaders for the field of S&TI rather than analysts. The MS&TI program is offered for full-time in-residence students and part-time students as well. Approximately half of the students admitted to existing educational programs at the NDIC study full time. Students come mainly from the population of military officers in the O-4 and higher ranks and civilians at GS-13 or above. Given this mature body of matriculating students, the program is focused on equipping managers with a broad understanding of intelligence and giving them a S&TI-specific background rather than on just developing a cadre of nationally trained analysts (Studds et al., 2011). In order to offer the MS&TI degree, the university population needs to have faculty and students with a strong technical background. In particular, “the student body would have characteristic prerequisites for graduate study that are significantly different from those needed for the advanced policy, political and military strategic intelligence of the University‘s current elective program.” The students are expected to have the “technical depth, skills, and tools to understand the potential of S&T” (NDIC, 2011a). When examined in detail, the requirements for admission to the MS&TI program are unclear. In the briefing received by the committee, it was stated that students require “undergraduate level STEM knowledge” (NDIC, 2011a) but in discussion with the faculty and administrators there was no consensus on whether the requirement for STEM undergraduate degrees or prior STEM training would be enforced. The committee feels strongly that the STEM prerequisites must be clarified, preferably in favor of requiring knowledge of STEM prior to admission.

OCR for page 3
6 Review of the NDIC’s Master’s Degree in S&TI The current educational programs at the NDIC, including the master of strategic studies in intelligence (MSSI), require that students have an active Top Secret - Sensitive Compartmented Information (TS/SCI) clearance, which enables them to access classified resources and attend classes taught at the SCI level. The requirements for clearances, the closed facility with cleared faculty, and the breadth of knowledge brought to the classroom by other students with field experience are unique aspects of the NDIC that provide a strong underpinning to the MS&TI program. Comprising a faculty to teach a new program of study is always a challenge. Upon examination, the NDIC does plan for a faculty worthy of the program. It has been building its capabilities over several years in specialist areas, which it already provides as focus areas for students in existing degree programs. Hence, a cadre of capable and tested faculty exists. Secondly, there are plans for recruiting and expanding faculty in critical areas that will provide the capacity for competently serving the incoming student body. The faculty appears to bring an appropriate set of knowledge capabilities to the task of offering the MS&TI program. According to the NDIC catalog (NDIC, 2011a): The faculty brings a wealth of knowledge and experience gained through earning advanced degrees in intelligence-related fields, and through leadership positions in the national intelligence community. They possess a wide range of expertise in Intelligence Community (IC) topics and come from varied academic and intelligence careers. Many have served on national boards and commissions to include the National Security Council, Intelligence Science Board, the Weapons of Mass Destruction Commission, and selected presidential commissions. The Defense Intelligence Agency (DIA), Department of State (DoS), Central Intelligence Agency (CIA), National Security Agency (NSA), Federal Bureau of Investigation (FBI), Foreign Denial and Deception Committee (FDDC), and National Geospatial-Intelligence Agency (NGA) are all represented on the faculty. The Air Force, Army, Coast Guard, Marine Corps, and Navy provide faculty members as service advisors for their cohorts of students enrolled in the University. CIA, DIA, NGA, NSA, and the Department of State also provide distinguished Chairs and greatly contribute to a more integrated Intelligence faculty. (NDIC, 2011a) The requirements set forth for the faculty are that they have “specific technical backgrounds and experience [including] an in-depth understanding not only of the science, but conditions and environment of government, military, and commercial technology research, development, and deployment.” Beyond this, they are also required to hold degrees suitable to the level at which they teach and advise (NDIC, 2011a). The faculty is actively engaged in curriculum development. The NDIC curriculum working group reviews all proposed course additions or changes. The process for approving new courses was described as including several distinct steps, starting with a course proposal, then conditional approval for offering the course, followed by several trial runs in special topics status, and finally, if all conditions are met, status as a permanent course offering. The NDIC requires that at least two faculty members be able and willing to teach any new course (O’Neill, 2011). The MS&TI curriculum was developed from the intelligence community’s review of competencies and needed skill sets. The Office of the Director of National Intelligence (ODNI) published a report on skills, competencies, and intelligence capabilities that served to both identify the need for the MS&TI program and develop the structure of the program (ODNI, 2008). The program covers the global impact of S&T, the challenges of globalization, and S&T- enabled symmetry and S&T-enabled asymmetry. The School of Science and Technology Intelligence “complements the capabilities of the existing College of Strategic Intelligence by providing select science and technology focused students better opportunities for effective

OCR for page 3
Program, Curriculum, and Syllabi Review 7 science-based research, focused education and interagency outreach within a science-based educational paradigm” (NDIC, 2011a, p 38). The core curriculum provides students with a comprehensive and laudable foundation in intelligence that includes the following, as partially extracted from the NDIC catalog (NDIC, 2011a): A Global Perspective – begin with an overview of globalization and strategic intelligence issues as the context for intelligence and U.S. national security. Intelligence in a Dynamic Global Environment – focus on developing and applying forecasting and analytical skills to systematically translate world events into products that identify and fill intelligence gaps. National Security – prepare students to be conversant with strategic intelligence demands, and to understand how the executive branch coordinates intelligence policy in the context of national security planning, Congressional oversight of intelligence policy, and budgeting, and how military and diplomatic consumers contribute to and defend intelligence policies and programs in both the executive and legislative processes. Integrated Skills, Competencies, and Intelligence-Related Capabilities – Critical thinking, analytic skills, communications, and intelligence related capabilities are woven throughout the content of the curricula. Engagement and Integration – Develop Interpersonal Skills, teaches students how to Engage and Collaborate with others and how to Influence and persuade others. Communication – Oral and Written Communication, Multi-Media Communication, and Conviction. Critical Thinking – Development of Situational Awareness, use of Creative Thinking, Synthesis and Explore Alternatives. Professionalism – Apply the principles of Ethics and Integrity, Initiative, and Adaptability. Mission Awareness – Understands the Enterprise Perspective, Customer Operations and Requirements, and Policy and Directives. Analysis – Use Analytic Rigor, how to Investigate, use Collection Systems and Operations, understand the intelligence Process and Exploitation capabilities, knowledge of Analytic Tools and Methods. (NDIC, 2011a) Before 2009, a non-thesis option was available for MSSI, but all students are now required to develop and present a master’s-level thesis (Studds et al., 2011). The planned MS&TI thesis will conform to academic standards as a formal, written presentation of original research examining an S&TI-related topic in the selected specialization. Students are encouraged to pursue research that contributes to the mission of the intelligence community. Theses may be classified or unclassified (Studds et al., 2011; NDIC, 2011a) The MS&TI program, therefore, requires the student to complete forty-three (43) total credit hours, thirty-six (36) of which are in course work and seven (7) are in thesis research. The course work is split between three categories: fifteen (15) common core courses, six (6) degree program required courses, and fifteen (15) elective/concentration courses (Studds et al., 2011; NDIC, 2011a). The five concentration areas currently include Weapons of Mass Destruction, Information Operations and Cyber Intelligence, Emerging and Disruptive Technologies, Geostrategic Resources and the Environment, and Foreign Denial and Deception. The five courses that make up the common core are: MCR 601. Globalization and Intelligence Issues MCR 603. Social Analysis and the Spectrum of Conflict MCR 607. Intelligence Reasoning and Analysis MCR 609. The Compound Eye: Intelligence Collection MCR 611. Intelligence and National Security Policy

OCR for page 3
8 Review of the NDIC’s Master’s Degree in S&TI These courses are all developed and available to be taught. There are faculty to teach them, established learning objectives, and structured course material. The courses that make up the five concentration areas are listed here by concentration. In the following lists, each course is annotated according to the maturity of the course offering. The annotation A refers to a course that is available, has been taught before, and is mature in program content, although it may be modified for the MS&TI degree offering. The annotation D refers to a course that is developed but has not yet been taught and is therefore considered to still be a work in progress according to the NDIC course development processes. The annotation F refers to a course that has not yet been developed but which is intended for development at a future time. Weapons of Mass Destruction —MST 663. WMD Counter-proliferation (A) —MST 665. The Biological Threat (A) —MST 666. Space and Missile Threats (D) —MST 667. The Nuclear Threat (A) —MST 669. The Chemical and Explosive Threat (A) —MST 655. Advanced Conventional and Non-conventional Weapons (A) —MST 656. The Economics of Technology (F) Of these courses, five, marked A, are considered to be developed, available to be taught, and mature in terms of content, learning objectives, and structured course material. One, MST 666, which is marked D, is considered to be under development, and one, MST 656 marked F, is considered to be a future development need (NDIC, 2011b). Information Operations and Cyber Intelligence —MST 680. Information Power and National Security (A) —MST 681. Propaganda, Persuasion, and Influence (A) —MST 682. Cyber Operations (A) —MST 683. Foreign Information and Cyber Strategies (D) —MST 684. Cyber Defense (A) —MST 687. Advanced Information Power Seminar (D) —MST 698H. Cyber Attack (D) —MST 698J. Cyber Exploitation (D) Of these courses, four are listed as being developed, available to be taught, and mature in terms of content, learning objectives, and structured course material. The four that are listed as under development (D) are MST 683, MST 687, MST 698H, and MST 698J (NDIC, 2011b). Emerging and Disruptive Technologies —MST 653. Advanced Science and Technology Intelligence (Process) (F) —MST 655. Advanced Conventional and Non-conventional Weapons (A) —MST 656. The Economics of Technology (F) —MST 657. Case Studies in Technology Transfer (D) —MST 658. Infrastructure Vulnerability Assessment (A) Of these courses, two are listed as being developed, available to be taught, and mature in terms of content, learning objectives, and structured course material. One of the remaining three is listed as “under development”: MST 657. The remaining two, MST 653 and MST 656, are listed as future development projects (NDIC, 2011b).

OCR for page 3
Program, Curriculum, and Syllabi Review 9 Geostrategic Resources and the Environment (Energy Resources and Power) —MST 672. Intelligence and the Changing Global Resource Environment (F) —MST 673. Geology and Intelligence (F) —MST 674. Nuclear and Other Alternative Energy Sources (F) —MST 675. Electrical Power Systems and Distribution (F) —MST 656. The Economics of Technology (F) —MST 658. Infrastructure Vulnerability Assessment (A) Of these courses, only one, MST 658, is listed as having being developed, available to be taught, and mature in terms of content, learning objectives, and structured course material. The remaining five courses are listed as future development efforts (NDIC, 2011b). Foreign Denial and Deception —MST 660. Introduction to Denial and Deception: History, Concepts, Issues, and Implications (A) —MST 662. Denial and Deception: Psychological/Cultural Aspects, and National Security Decision Making (A) —MST 664. Denial and Deception: Adversaries, Organizations, Activities, and Countermeasures (A) —MST 668. Denial and Deception: Tradecraft, Tools and Methodology (A) Of these courses, all are listed as being developed, available to be taught, and mature in terms of content, learning objectives, and structured course material (NDIC, 2011b). Overall, as it reviewed the proposed curriculum, the committee found that the common core courses are all available and that three of the proposed specialization areas appear to be populated enough to be offered. Two of the specialization areas, Emerging & Disruptive Technologies and Geostrategic Resources & the Environment, require significant course development efforts before they will be ready to be offered to students. Findings and recommendations are found later in the chapter. Syllabi A detailed description of the course work of the MS&TI program was provided to the committee. The courses annotated as “on hand” or “under development” all included a course description, a description of the course’s contribution to the mission, expected learning outcomes, a list of intelligence community competencies, course requirements, grading criteria, a reading list, a session outline with subjects and issues being covered, and defined learning objectives. The course content reflected in this material seems complete and thoughtfully compiled, with the exception of the structure used for the learning objectives. The committee is concerned that the learning objectives are too broadly stated and are poorly linked to the program learning outcomes. Additionally, a close analysis of the ODNI Competency Subdirectory for S&TI, which was used as a reference material for designing the MS&TI program, reveals that much is missing from the current course content (ODNI, 2008; Studds et al., 2011, p 14). Furthermore, of the material included, the committee had significant concerns about the learning objectives. These issues are discussed here and the committee’s findings and recommendations are offered in the appropriate section. Learning Objectives Learning objectives form the core of any educational program. Correctly and precisely stated expectations of what exactly a student should learn in a course allow not only the precise

OCR for page 3
10 Review of the NDIC’s Master’s Degree in S&TI engineering of course content to achieve those goals, but they also define the prerequisite knowledge of the student, the competencies of the instructor, and the end product of a program. Each of these is worth considering separately. Learning objectives should be stated using action words and precise outcomes, which must be empirically measurable. For example, a learning objective that is poorly formulated is this: “The student should understand how to write a paper.” A much better crafting of that learning objective would be “The student will write a research paper that adheres to appropriate academic standards; the paper will include a thesis statement, background material, data presentation and analysis, and a synthesis of research findings.” In the first statement of objective, there is no way to measure how well a student understands how to write a paper. In the second, not only is the expectation clearly stated but the measure of success is also clearly defined. The problem of poorly defined learning objectives creates confusion for the program objectives and the student learning prerequisites. Consider as an exemplar, learning objective for MST 684, Cyber Defense: “Gain an understanding of the vulnerabilities of the DoD and IC cyber systems.” Were this learning objective to be recrafted in active language, it could be expressed in many ways, two of which follow: (1) “Use software modeling techniques, such as functional decomposition diagrams (FDD) and data flow diagrams (DFD), to identify critical points of vulnerability in an exemplar cyber system” and (2) “Reverse engineer an identified piece of malicious software using provided tools in a laboratory environment to identify attack paths and probable target structures.” Either and both of these alternatively stated learning objectives can be measurable on an objective basis. In other words, one can measure both whether a student achieved the learning objective and how well the student achieved it. Furthermore, pre- and post- education assessments are possible to measure how much was achieved. This becomes a critical quality control function: If students are uniformly not achieving the expected level of competency, it might be an indicator that prerequisite knowledge has been incorrectly set or that the teaching process is not optimized to the student population. However, the two restatements of learning objectives differ in three important ways: first, in the prerequisite knowledge demanded of the student; second, in the capabilities they demand of the instructor; and, third, in overall learning outcomes called for by the program. These differences are critical to the administration of the MS&TI program. Both the first and second restatements are legitimate children of the original learning objective but speak to different student populations, different faculty, and different program outcomes. The obligatory prerequisite level of competency, for learning objective 1 would need to include a basic knowledge of information system structures. For learning objective 2, however, the requisite level of competency would need to be much higher; it would include knowledge of software structures, algorithmic development methods, and attack pattern analysis. This distinction then tracks back to the admissions process: requiring specific types of prior education or training, or having an identified set of prerequisite knowledge to be gained prior to admission to the program.

OCR for page 3
Program, Curriculum, and Syllabi Review 11 FIGURE 1-1 The importance of learning objectives. The learning objectives influence faculty capabilities. For learning objective 1, faculty having a general knowledge of software modeling could teach the course. For learning objective 2, however, someone with significant experience in software development, reverse engineering, and cyber threat analysis would be needed to successfully teach the course. It is far easier to find an instructor to meet the needs of learning objective 1 than those of learning objective 2, which argues for a careful examination of the program learning outcomes. If these outcomes are truly to produce graduates who can reverse engineer malware, then having faculty who can teach to learning objective 1 but not to learning objective 2 will simply not do. And it is, in fact, the overall program learning outcomes that are at stake. If the individual course learning objectives are poorly and imprecisely worded and defined, then the outcomes will range widely, depending on individual student ambitions and efforts. A more predictable graduate results from precisely defined learning objectives. Program ODNI S&TI Competencies A detailed breakdown of competencies and the courses in which they are covered is provided here. Two important caveats apply to the following discussion. First, it is recognized that MS&TI faculty are still developing courses, and the committee hopes that some of the comments included below will be addressed. Second, is that the committee understands that some of the syllabi have classified portions that were not shared with it. For this reason, the committee readily acknowledges that some of the following points may well be moot. The following core competencies seem to be well addressed in the syllabi: Engagement and Collaboration, Critical Thinking, Accountability for Results, and Communication. Engagement

OCR for page 3
12 Review of the NDIC’s Master’s Degree in S&TI and Collaboration includes information sharing and interpersonal skills, which are natural components of an educational environment. Critical Thinking is a priority area for the curriculum in general and as such the elements of it are addressed in each syllabus extensively. The elements of Accountability for Results that are addressed include continual learning, policy and directives, results orientation, and rigor. The elements of Communication, including information transfer and communicating with impact, are also natural elements of an academic program. The core competencies that are not well addressed in the syllabi include Leadership and Integrity and Management Proficiency. The elements of Leadership and Integrity identified in the ODNI Competency Subdirectory, such as Developing Others and Leveraging Diversity, are difficult to teach in a classroom environment. Indeed, the syllabi emphasize integrity from an academic perspective. The elements of Management Proficiency can be taught in a classroom environment, but they compete for time with more technical subjects. When hard choices have to be made regarding use of available time in the classroom, it would be appropriate to prioritize the material at the core of a program rather than more general topics. In this case, technical knowledge, skills, and abilities (KSAs) may be rightfully prioritized over management proficiencies, depending on the students’ backgrounds. That is not to downplay the importance of such proficiencies; on the contrary, they are vitally important to the desired end product of the MS&TI degree: a leader who is competent to manage an S&TI function. As discussed in findings, this shortcoming of the MS&TI program might be addressable using clever partnerships with other programs. Sixteen (16) technical expertise competencies (TECs) are listed in the ODNI Competency Subdirectory. Those TECs that are well-covered in the MS&TI syllabi include the following: Collection Systems Capabilities, Cryptography, Customer Operations and Requirements, Intelligence Disciplines, Research and Exploration, and Researching. The TECs that seem to be adequately covered in the syllabi include the following: Processing and Exploitation Capabilities, Project Management, Technology Integration/Insertion, Tools and Methods, and Transportation. The TECs that are not well-covered, either due to incompleteness of the topic or the absence of it altogether, include the following: Contracting/Procurement, Counter-proliferation, Intelligence Topics, Systems Engineering, and Targets: General. Given the time constraints of the program, it is probably logical to not cover everything and in this case, Contracting/Procurement is an obvious topic to leave out. However, it is appropriate to review the Counter-Proliferation TEC, described as “skill at supporting counter-proliferation operations with S&T operations” (NDIC, 2011b), for completeness. Reviewing all syllabi provided does not readily reveal any coverage of this competency. It is entirely possible that this competency is covered in MCR 601 and/or MCR 611 but, if it is, this is not explicitly stated in the syllabi and so that lack of coverage can only be conjectured. If, in fact, this competency is not covered, it may be worthwhile reviewing the syllabi to consider whether there should be more emphasis on this competency. One of the TECs that is not completely covered is Intelligence Topics. While 55 KSAs are listed under Intelligence Topics, the syllabi that were provided, which do not include sensitive KSAs, show 27 of them are not addressed. These are predominantly the KSAs that deal with biological weapons development, deployment, and testing. Because this is a critical element of the S&TI challenge today and an extremely complicated and difficult one, the NDIC should consider a more comprehensive approach to educating the MS&TI student population in biodefense capabilities. “The breadth of biological threats is much broader than commonly appreciated and will continue to expand for the foreseeable future” (Choffnes et al., 2006, p 28; NRC, 2006). This very important area of ST&I, biological threats, appears to be under-represented in the current curriculum. This deficiency is exemplified by MST665 “The Biological Threat,” which is described as “on hand” and includes two lectures on bacteria, two on viruses, and two on toxins, but the content of these lectures is not detailed. They do not appear to include basic principles of epidemiology, synthetic biology, the engineering of organisms for resistance and increased

OCR for page 3
Program, Curriculum, and Syllabi Review 13 virulence, vaccines and protective immunity, or sources of new disease agents (emerging infections). Additionally, the TEC for Systems Engineering is viewed by the committee as a critical omission. The interdependencies that characterize the global innovation landscape require for ability to understand and apply systems logic when analyzing the various bits and fragments of information collected on foreign capabilities. In particular, the integration of vastly different technologies and sciences is resulting in capabilities that can be understood only when viewed systemically. This is particularly true in the biosciences: “Other fields not traditionally viewed as biotechnologies–such as materials science, information technology, and nanotechnology–are converging with biosciences in unforeseen ways and enabling the development of previously unimaginable technological applications” (Choffnes et al., 2006, p 26; NRC, 2006; NRC, 2005). Developing these new technologies with fully open information and sharing is difficult enough; the challenge of divining what is occurring in a closed environment where clues are being gleaned from a variety of sources is vastly more difficult. Systems engineering expertise is also useful for the analytical function from a team communications perspective. Useful analysis requires effective communication among diverse individuals. Members of an analytical community must communicate with each other and with their clients. Both kinds of communication can bring into contact individuals with very different missions, backgrounds, and perspectives. Within an analytical community, a single analysis might require communication among individuals with expertise in economics, anthropology, psychology, engineering, and logistics. Each contributing discipline might have subfields and competing theories, each needing to be heard. Setting the terms of an analysis and reporting its results might require communication with clients who differ from the analysts in their objectives, careers, and education. These professional differences overlay the cultural, socioeconomic, and other differences that can complicate any communication in a diverse society. (NRC, 2011a) Expertise in systems engineering establishes an ability to “integrate and verify system elements” and analyze “processes and procedures [used] to implement operational schedules,” according CIA’s description for the job of systems analyst, available online at www.cia.gov/careers/opportunities/science-technology/systems-engineer.html, last accessed July 15, 2011. Furthermore, systems engineering expertise is increasingly being identified as a critical core competency that is presenting short supply in today’s intelligence community (NRC, 2011a; NRC, 2011b). In sum, the Systems Engineering TEC appears to be an increasingly valuable part of the intelligence analysis equation, particularly in S&TI. This discussion serves as the overarching framework for the findings and recommendations of the committee. It is worth restating that both the discussion and the findings and recommendations are based solely on the material presented and are made with the express knowledge that some material may not have been shared with the committee for reasons of sensitivity or classification, perhaps rendering some of the discussions or the findings and recommendations moot. FINDINGS AND RECOMMENDATIONS General In general, the master’s program is well thought out and gives students a solid foundation on a wide range of S&T intelligence issues. For the most part, the courses cover a lot of ground, and students will be exposed to a great deal of S&TI-related material. The course development process is inclusive and requires that at least two faculty members be available to teach a

OCR for page 3
20 Review of the NDIC’s Master’s Degree in S&TI MST 613 Science, Technologies, and Intelligence, would appear to be the fulcrum upon which the MS&TI pivots. As such, it is a critically important course. This is the course that is specifically focused on S&TI in the program, and its description in the syllabi clearly states that a primary purpose of the course is to teach how to identify disruptive technologies. In support of this goal, it would be worthwhile to have a deeper discussion on technology forecasting and tracking strategies in the course. Seeing into the future is complex and uncertain, yet there are techniques that have been developed to lend structure to such endeavors, particularly in the technological arena. A survey of forecasting methodologies would be worthwhile. Special emphasis should be placed on approaches to forecasting disruptive technology. Students should have an understanding of the appropriate use of forecasts and how to frame and present technology forecasts in analyses. The committee notes that a good teaching material would be the NRC’s Persistent Forecasting of Disruptive Technologies (NRC, 2009). Finding 17: The material included in MST 613 on forecasting is incomplete and could benefit from increased rigor, particularly with respect to disruptive technology identification. Recommendation 17: The National Defense Intelligence College faculty should add a session on disruptive technology forecasting and tracking to MST 613. Further, the NRC report Persistent Forecasting of Disruptive Technologies (NRC, 2010) should be added to the materials for teaching this course. Areas of Specialization/Concentration There are currently five of concentration areas for the MS&TI: Weapons of Mass Destruction, Information Operations and Cyber Intelligence, Emerging and Disruptive Technologies, Geostrategic Resources and the Environment, and Foreign Denial and Deception. The courses associated with these concentration areas of are at a mix of maturity levels. Tthe courses that were presented with the syllabi and found to warrant suggestions are discussed next. Those courses that were not mature or that did warrant suggestions are not covered. The concentration Geostrategic Resources and the Environment currently has no developed courses, so no concrete commentary is possible.. However, the committee supports the inclusion of this as a concentration area in the future MS&TI program because the issues involved are so important and have the potential to impact both global and regional stability, which are important elements of national security. Weapons of Mass Destruction The inexorable advances in S&T increase the importance of intelligence about weapons of mass destruction. S&T discoveries give groups and individuals who scheme to advance their causes more options to harm the United States. As an example, the knowledge to produce and deploy nuclear and biological weapons that may not be optimal, but it can still cause horrendous destruction. It is mostly held in check not by insufficient S&T knowledge but by the lack of enriched uranium or a means for delivering biological agent over a wide area. Nevertheless, as demonstrated by the effects of modest-scale agent delivery by mail, a few letters containing anthrax sent by one (probable) individual can cause multiple deaths, panic, and economic calamity (FBI, 2010). Similar disruptive attacks on food supplies, electrical grids, key modes of transportation, and economic mechanisms such as stock markets are possible, if not likely, targets that S&T may enable. MST 655, Advanced Conventional and Non-Conventional Weapons. This course should include a discussion of the importance of software in both advanced conventional and non- conventional weapons. Software subsystems are an important underlying technology of weapons

OCR for page 3
Program, Curriculum, and Syllabi Review 21 systems. Furthermore, the precision of weapons is increasingly a function of software, hardware, and firmware elements. It is increasingly important for weapons analysts to understand the underlying logic that guides and controls weapons, both from the platform and within the warhead. Additionally, new weapons are increasingly developed and tested virtually, before any physical evidence is available for either collection or observation. As such, software simulation environments become an important element of new weapons discovery. Finding 18: The MST 655 course related to the importance of software in weapon systems has a shortcoming. Recommendation 18: The National Defense Intelligence College should include software design and simulation software in MST 655, with an emphasis on the use of these tools for the design and testing of new systems. Another area worth covering in the course would be autonomous and semi- autonomous weapons systems such as UAVs. These systems are changing the way we fight on the battlefield and how we think about tactical warfare. While relatively new, these systems are being rapidly developed by many countries. Finding 19: The syllabus for MST 665, The Biological Threat, is a thin outline compared with the substance in other sections in the concentration area such as MST 667, The Nuclear Threat. Recommendation 19: The National Defense Intelligence College should greatly expand the content of MST 665 and label it as “under development” until it is improved. Emerging and Disruptive Technologies The NRC report Persistent Forecasting of Disruptive Technologies discussed in detail current forecasting methodologies, the nature of disruptive technologies, and the characteristics of disruptive technologies (NRC, 2010). Many researchers now see the importance of tracking technologies not only for how they are originally used, but how they are used by different groups of people. That NRC report stated as follows: The value of technology forecasting lies not in its ability to accurately predict the future but rather in its potential to minimize surprises. It does this by various means: Defining and looking for key enablers and inhibitors of new disruptive technologies, Assessing the impact of potential disruption, Postulating potential alternative futures, and Supporting decision making by increasing the lead time for awareness. (NRC, 2010) S&TI analysts will need to understand how various technologies fit together and how out of the box thinking may affect or change whole industries very quickly. This is not a new thought. As Arthur Schopenhauer stated over 100 years ago: The task is not so much to see what no one yet has seen, but to think what nobody yet has thought about that which everybody sees. The cultural lenses that overlay expectations color the ability to “see” potential applications and uses. It is more important to study an approach to correctly defining these potential uses than it is to review as many technology areas as possible.

OCR for page 3
22 Review of the NDIC’s Master’s Degree in S&TI Finding 20: The Emerging and Disruptive Technologies concentration area includes a very ambitious list of topics. The scope of problems is huge and rapidly changing. Employing competent instructors for these evolving topics will be challenging. Recommendation 20: The National Defense Intelligence College should reduce the number of topics in the Emerging and Disruptive Technologies concentration to a manageable set that provides core competencies across a broad range of topics without being overly ambitious in terms of content. Finding 21: The Emerging and Disruptive Technologies concentration area lacks a session on the innovative use of technologies. Obvious examples of this are flying airplanes into buildings and using the mail to distribute bioweapons-grade materials. Recommendation 21: The National Defense Intelligence College should add a session to an existing course or create a new course on the innovative use of technologies to create threat capability. Information Operations and Cyber Cyber is a difficult area to cover: technology changes very quickly and new innovations are adopted globally at a dizzying pace. Information Operations and Cyber Intelligence is a very complicated topic area that spans not only raw IO & Cyber but also Cyber as a means to conduct other forms of ST&I. In fact, the basic vocabulary of IO does not appear to be addressed in the concentration which would allow leaders to understand at least the categories, significance, and components of modern cyber warfare and information operations. Furthermore, the areas of information operations and of cyber intelligence are in fact very different areas, with different skill sets, different objectives, and different candidate students. Finding 22: The Information Operations and Cyber Concentration requires a precise definition of prerequisite knowledge for the students and for the instructors in order to create a slate of courses that are meaningful. Understanding what the graduate should be able to do is critical to defining both the learning objectives and the prerequisite knowledge requirements. Recommendation 22-1: The National Defense Intelligence College should emphasize the development of a student’s ability to learn, adapt, and deal with revolutions in cyberspace rather than emphasizing the learning of specific approaches or techniques that might work today but be obsolete within a year. Recommendation 22-2: National Defense Intelligence College students focusing in the Information Operations and Cyber concentration should be required to have an appropriate undergraduate level of education, or life skills associated with information operations and cyber, or training appropriate to the learning objectives. MST 680, Information Power and National Security. As the Internet increasingly becomes a conduit for both power projection and competition, all of the students must have prior knowledge so that this course can be conducted at the graduate rather than the introductory level. Students need an overview of how the Internet works and of how typical computer and IP networks are designed and must know about the growing dependency of everyday devices, appliances, systems, and infrastructure on IP.

OCR for page 3
Program, Curriculum, and Syllabi Review 23 Finding 23: The class should pay special attention to cloud-based architectures as well as mobile devices. Students should understand the basic strategies used to attack networks and have an understanding of defensive measures. There should be discussion of how the Internet can multiply the power of individual ideas and how this can be seen as threatening to other nation states. While week 6 is devoted to Legal considerations in Information Operations, it is extremely important to discuss the various intelligence general counsel interpretations of existing laws, policies, and restrictions. In many cases, the agencies do not have the same interpretations, especially on issues such as U.S. persons, identity, data handling, covert action, and the active use of social media. Additionally, it is important to cover the role of standards, agreements, treaties in cyberspace and the strategies other nation states such as China, Russia, and the European Union are trying to reduce the United States influence and dominance in cyberspace. There are various materials available from the U.S. government, such as through the Joint Staff publications, that can be used to illustrate these issues. Recommendation 23: The National Defense Intelligence College should add to MST 680 materials such as the Joint Information Operations Planning handbook and doctrine manuals for electronic warfare. MST 681, Propaganda, Persuasion, and Influence. New social media technologies such as Facebook, Twitter, and Google are having a global impact. Recent examples of events that show the power and influence of social media include the 2009 Iranian elections1, the Arab Spring of 20112, and the Great Firewall of China3. Analysts need to understand how social media is being used, particularly in coordinating and spreading political and social agendas. Further, they need to understand how social media are used to position and reinforce ideas and to generate and mobilize large portions of the populace around those ideas, particularly youth. A significant challenge lies in how social media are used in intelligence: At times social media blur the line between analysis and collection, and the potential for crossing that line needs to be deeply appreciated by members of the intelligence community. For example, active participation in social media could trigger specific rules and regulations that restrict how an analyst may operate, even with the most benign intentions, because some actions might be considered to fall into the realm of covert action. There is a distinct difference between reading a blog and friending or tweeting: One set of actions is passive while the other is active. There are challenges in how analysts and leaders manage and constrain their interactions with social media while still taking advantage of access to useful information. Additionally, S&TI analysts and managers need to understand meta-social media issues, including the mechanisms of how usage is measured, how activities are tracked, the measurement of impact, assessing how social media is actually being used, and the identification of supernodes. The emerging area of social media may prove to be too large to fit into one course; it might indeed need to be distributed across several courses. Not only are the preceding aspects of great importance, but there are also subtleties that need to be covered in the curriculum—including how different cultures use social media differently, how social media can be overwhelming traditional media at the same time they are feeding it, and how social media postings go “viral” (including characteristics of viral postings). 1 See http://mashable.com/2009/06/21/iran-election-timeline/ 2 See http://blogs.reuters.com/great-debate/2011/02/16/digital-media-and-the-arab-spring/ 3 See http://www.datelineshanghai.com/scaling-the-great-internet-wall/

OCR for page 3
24 Review of the NDIC’s Master’s Degree in S&TI Finding 24: The area of social media appears to be neglected both in MST 681 and in the curriculum in general. Recommendation 24-1: The MST 681 course should cover the use of social media and the importance of the social graph to show how they are used to facilitate nation-state protest, revolution, and social consciousness. The curriculum should cover the power of crowd sourcing, the effects of the digital exhaust, and social network tools such as Facebook, Google, YouTube, and Twitter. The class should also discuss Internet use to influence and shape culture and society. Recommendation 24-2: The National Defense Intelligence College faculty should consider segmenting the challenges and learning objectives associated with covering social media and parceling out the learning needs across several different courses. Finding 25: The backgrounds of the instructors for the Information Operations topics are critical to ensure that the material taught is deeper than a technical seminar from a vendor. Recommendation 25: The National Defense Intelligence College should incorporate relevant open-source information on cyber-attacks for educational purposes and for stoking the imagination of students. Candidate material should include numerous technical analyses readily available online that cover recent cyber-incidents such as the Farewell Dossier, the Stuxnet worm, the Phishing Scheme against RSA, and the anonymous attack on HBGary, as well as future incidents as they occur. Finding 26: The Information Operations syllabi reads more like political science focused syllabi than science or technology focused syllabi. For example, learning objectives are very high level and include phrases such as “gain an appreciation for” and “gain an understanding of.” The basic vocabulary of this concentration area does not appear to be addressed so that it would allow leaders to understand at least the categories, significance, and components of modern cyber warfare and information operations. Recommendation 26: The National Defense Intelligence College should repurpose the course content to be much more focused on the underlying science and technology and introduce people to the concept of cyber by exposing them to widely available hacking training for hands-on knowledge of systems compromise and manipulation. Denial and Deception The Denial and Deception concentration area is a well-developed program of study that has existed for quite some time. The courses for the MS&TI program have been repurposed from the existing curriculum for MSSI. Unfortunately, the resulting program of study that appears to be light on science and technology while strong in policy analysis. Finding 27: There is little discussion of the implications of communications technology. The use of black, gray, and white propaganda and psychological operations is conspicuously absent. (This may be because it is in sensitive elements of the program.) Social media, which can be easily used for denial and deception, are not included in the provided material. Recommendation 27: The National Defense Intelligence College should add the following content to the syllabi for MST 664:

OCR for page 3
Program, Curriculum, and Syllabi Review 25 Communications are now nearly instantaneous, ubiquitous, costless, anonymous, secure, and unlimited band-width. This enables worldwide and rapid flows of information, images, and money for good and ill. “In the global community, this leveraging power of information technology provides previously ineffective marginalized groups with additional means for both direct challenges to authority as well as indirect media for intra-group communications, planning, and management. Recent cases include the B-92 radio station in Bosnia4, the use of the Internet by the Chiapas revolutionaries to marshal global attention to their cause5, the use of tape-recorded messages by the Ayatollah Khomeini while in exile in Paris, the use of Twitter in the Iranian election protests, and the use of Facebook during the Egyptian protests6. These examples illustrate the breakdown of boundaries, both physical and customary” (Ryan, 2011). Finding 28: The current Foreign Denial and Deception concentration does not distinguish between the MSSI and MS&TI programs. Only one course, MST 604, is different, and MST 613 would be optional in MS&TI. Additionally, there is a dearth of technology and science in the syllabi. Recommendation 28: The National Defense Intelligence College should incorporate a stronger background in science and technology in the Denial and Deception concentration area by requiring one or more additional courses that focus on S&T-based based deception. CONCLUSION The need for the NDIC’s new degree program was found to be justified. The user population reflected strong support for this effort. An examination of the program also revealed that it meets the four criteria referenced: it is necessary, it is unique, it meets general academic practice, and the administrative processes support academic freedom. REFERENCES Published Choffnes, Eileen R., Stanley M. Lemon, and David A Relman. 2006. “A brave new world in the life sciences: The breadth of biological threats is much broader than commonly thought and will continue to expand.” Bulletin of the Atomic Scientists September/October: 26-33. Crawley, Edward, Johan Malmgvist, Soren Ostlund, and Doris Brodeur. 2007. Rethinking Engineering Education: The CDIO Approach. Springer Science+Business Media, pp. 40 and 113. Engineering Trends. 2006. “Historical trends and near-term predictions of statistics on degrees, enrollments and research expenditures for engineering education in the United States.” Engineering Trends Quarterly Newsletter. Summer 2006. 4 See for example “Preserving the Free Flow of Information on the Internet: Serbs Thwart Milosevic Censorship” at http://www.usip.org/events/preserving-free-flow-information-internet-serbs-thwart-milosevic- censorship-round-two. 5 See for example “Rebellion in Chiapas: insurrection by Internet and public relations” by Jerry W. Knudson from Temple University, available online at http://www.bsos.umd.edu/aasp/chateauvert/rebellio.pdf. 6 See for example “The dark side of Internet for Egyptian and Tunisian protesters”, which discusses many of these events. By Evgeny Morozov for the Globe and Mail, published Jan. 28, 2011. Available online at http://www.theglobeandmail.com/news/world/africa-mideast/the-dark-side-of-internet-for-egyptian-and-tunisian- protesters/article1887170/print/.

OCR for page 3
26 Review of the NDIC’s Master’s Degree in S&TI Federal Bureau of Investigation (FBI). 2010. Amerithrax or Anthrax Investigation Summary. February 19. Available online at http://www.fbi.gov/about-us/history/famous-cases/anthrax- amerithrax/amerithrax-investigation. Friedman, Thomas L. 2010. “Can’t keep a bad idea down.” The New York Times, October 26. Available at http://www.nytimes.com/2010/10/27/opinion/27friedman.html. Last accessed on July 20, 2011. Intelligence Science Board (ISB). 2006. Intelligence Science Board Task Force Report on the Intelligence Community and Science and Technology: The Challenge of the New S&T Landscape. November. Washington D.C.: Office of the Director of National Intelligence. Available at http://www.fas.org/irp/dni/isb/landscape.pdf. Last accessed on September 1, 2011. National Academy of Sciences, National Academy of Engineering, Institute of Medicine (NAS, NAE, IOM). 2010. Rising Above the Gathering Storm, Revisited: Rapidly Approaching Category 5. Washington, D.C.: The National Academies Press. Available at http://www.nap.edu/catalog.php?record_id=12999. Last accessed on July 12, 2011. National Defense Intelligence College (NDIC). 2011a. NDIC University Catalog 2011-2012. Washington, D.C.: Center for Strategic Intelligence Research. NDIC. 2010. National Defense Intelligence-2010 Annual Report. Available at http://www.dia.mil/college/. Last accessed on July 12, 2011. National Research Council (NRC). 2011a. Intelligence Analysis: Behavioral and Social Scientific Foundations. Washington, D.C.: The National Academies Press. Available at http://www.nap.edu/catalog.php?record_id=13062. Last accessed on July 15, 2011. NRC. 2011b. Intelligence Analysis for Tomorrow: Advances from the Behavioral and Social Sciences. Washington, D.C.: The National Academies Press. Available at http://www.nap.edu/catalog.php?record_id=13040. Last accessed on July 15, 2011. NRC. 2010. Persistent Forecasting of Disruptive Technologies. Washington, D.C.: The National Academies Press. NRC. 2006. Globalization, Biosecurity, and the Future of the Life Sciences. Washington, D.C.: The National Academies Press. Available at http://www.nap.edu/catalog.php?record_id=11567. Last accessed on July 15, 2011. NRC. 2005. An International Perspective on Advancing Technologies and Strategies for Managing Dual-Use Risks: Report of a Workshop. Washington, D.C.: The National Academies Press. Available at http://www.nap.edu/catalog.php?record_id=11301. Last accessed on July 15, 2011. Office of the Director of National Intelligence (ODNI). 2008. Intelligence Community Directive Number 652, Occupational Structure for the Intelligence Community Civilian Workforce. ICD 652, April 28. Available at http://www.fas.org/irp/dni/icd/icd-610.pdf. Last accessed on September 1, 2011. Ryan, Julie (ed.). 2011. Leading Issues in Information Warfare and Security Research, Volume 1. Reading, England: Academic Publishing International. Pages xiv and xv. The White House, Office of the Press Secretary. 2009. “President Obama launches "Educate to Innovate" campaign for excellence in science, technology, engineering & math (STEM) education. Press Release, November 23. Available at http://www.whitehouse.gov/the-press- office/president-obama-launches-educate-innovate-campaign-excellence-science-technology- en. Last accessed on July 12, 2011.

OCR for page 3
Program, Curriculum, and Syllabi Review 27 Unpublished NDIC. 2011b. National Academy of Sciences–Curriculum Review of the Master of Science and Technology Intelligence, May 2011. Washington, D.C.: Center for Strategic Intelligence Research, NDIC. O’Neil, Joseph. “The faculty’s role in curriculum development.” Presentation to the committee on May 13, 2011. Studds, Susan, Brian Smith, Joe O’Neil, Gerry Sherrill, Duncan McGill. “National Defense Intelligence College – School of S&T Intelligence.” Presentation to the committee on May 13, 2011.

OCR for page 3

OCR for page 3
Appendix

OCR for page 3