Click for next page ( 2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Executive Summary n the late l990s, the U.S. armed forces struggled to meet their recruit- ing goals and in some cases fell short. This led to the question of how the recruit-in" process and the recruiters' job could be better sup- ported in order to ensure that force strength, force quality, and the required skill mix of personnel will be available to meet ever-changing security and defense challenges. Military officials recognized that a fun- damental understanding of the youth population and of the effectiveness of various advertising and recruiting strategies used to attract them would be extremely valuable in addressing these questions. As a result, in 1999, the Department of Defense asked the National Academy of Sciences, through its National Research Council, to establish the Committee on the Youth Population and Military Recruitment. The committee is composed of 14 experts in the areas of military manpower, military sociology, psychology, adolescent development, survey methodol- ogy, behavioral theory, economics, and advertising and communication. During the first phase of its study, the committee examined long-term trends in the youth population and evaluated policy options that could improve the propensity for and enlistment in the Services. In our report, Attitudes, Aptitudes, and Aspirations of American Youth, the committee con- firmed the decline in propensity for military service among youth and identified several correlates, especially the trend in increasing college enrollments. The committee observed that current military research on advertising and recruiting often lacked long-term objectives and coordination across relevant research topics and methodologies. In this second phase, the

OCR for page 1
2 EVALUATING MILITARY ADVERTISING AND RECRUITING committee has developed an evaluation framework to assist the Depart- ment of Defense and the Services in making informed decisions on the effectiveness of various recruiting policies and mixes of recruiting re- sources. This report is the product of the committee's second phase of study. THE APPROACH The committee has identified several areas requiring more intensive study that might benefit from drawing on a variety of methodological approaches: Monitoring trends in youth attitudes, values, and propensity using surveys (Chapter 3~; Planning advertising using generative and experimental approaches (Chapter 4~; Determining optimal levels of advertising and recruiting resources and assessing the timing and levels of joint and Service-specific Advertising based on data from past or current programs using econometric methods (Chapters 5 and 6~; Determining optimum types of incentives using a combination of focus groups, surveys, and experimental approaches (Chapter 7~; and Performance management of recruiters again using a combination of methodologies (Chapter 8~. Rather than focusing on the strengths and weakness of the various methods with the intent of identifying some methods as per se superior to others, the committee's proposed evaluation framework is based on the fundamental notion that different research designs and the associated methodologies are suited to address different types of research questions. Table ES-1 provides examples of common research questions that emerge in the context of military advertising and recruiting. The framework has two dimensions. The first dimension differenti- ates between an existing or new incentive, program, or activity; the second dimension differentiates between three types of assessments. One type of assessment occurs in situations in which the outcome is not specified and the audience is asked to indicate the attractive and unattractive features; the second type deals with attitudes or behavioral intentions toward mili- tary enlistment; and the third deals with actual behavior, such as signing a contract with a recruiter. The cells in the framework list example types of questions and identify the method most appropriate for addressing each.

OCR for page 1
EXECUTIVE SUMMARY TABLE ES-1 Evaluation Framework 3 Outcome Not Specified A Priori Specific Attitudes or Behavioral Intentions Actual Behavior Question: "What does programs a target audience see as attractive or unattractive features of a program?" Method: focus groups; unstructured or open-ended surveys and interviews (Chapters 4 and 7) Existing programs Same as above Question: "What is the effect of a program on specified attitudes or behavioral intentions?" Method: surveys; experiments; . . quas~-exper~men~s (Chapters 3, 4, 7, 8) Same as above Question: "What is the effect of a proposed new program on enlistment?" Method: experiments; . . quas~-exper~men~s (Chapters 7 and 8) Question: "What is the effect of an . . exlstmg program on enlistment?" Method: econometric modeling (Chapters 5 and 6) CONCLUSIONS AND RECOMMENDATIONS Monitoring Trends in Youth Attitudes, Values, and Propensity In its first report, the committee concluded that propensity to enlist is a major direct determinant of actual enlistment. Thus, increasing propen- sity should be an important goal of the military. Monitoring surveys are well suited to measuring trends in propensity and examining the factors that contribute to changes over time. Implementation of useful monitor- ing surveys requires multiyear funding commitments and large samples of respondents. In constructing a monitoring survey to track propensity, the questionnaire must include a complete set of salient beliefs about the positive and negative consequences of joining the military. Other impor- tant content areas are the values attached to various outcomes, the expec-

OCR for page 1
4 EVALUATING MILITARY ADVERTISING AND RECRUITING tation that a particular outcome is more likely to be achieved in a military versus a civilian job, and the barriers or facilitators for enlistment deci- sions. In order to ensure complete coverage of these attitudes and beliefs, good survey designs should also conduct certain types of preliminary or exploratory studies. The committee recommends that survey research examining propensity be designed to incorporate the key determinants of propensity and that it be designed to permit analysis at the individual level. The committee proposes a program of survey research involving a commitment of at least five years. The committee recommends that con- sideration be given to undertaking a school-based survey, using cohort- sequential design, in which students are sampled in the 11th and possibly the 12th grade and regularly resurveyed until the age of 23 or 24. Planning for Advertising The purpose of advertising is to distribute information designed to influence consumer activity in the marketplace. In the military the pur- pose of advertising is twofold: to increase propensity to chose military service and to increase the likelihood of an individual choosing to join one Service over another. In the past, much of the research on military adver- tising has focused on evaluating the amount and cost of advertising, rather than on evaluating the effects of advertising content on targeted beliefs and values. In its first book, the committee found that intrinsic factors, such as duty to country, should be given increased weight in military advertising. In a strategy for developing and evaluating a range of message strat- egies, the first step is to track the competitive environment for military recruitment to detect factors affecting youth understanding and views of military service. The second step is to examine the beliefs, goals, and language of audience members using focus groups, interviews, and sur- veys. The third step is to develop and test a range of message strategies using experimental and quasi-experimental research designs. The fourth step is to allocate resources to various message strategies. Decision making during this step is informed by experiments and in-market testing. The committee recommends a program of research that follows these steps. Specifically, a program of research should begin with generative techniques to understand the concepts and language used by youth in considering alternative courses of action (e.g., higher education versus military service) and continue with survey research that measures the full range of beliefs, attitudes, and values that emerge as linked to these alter- native courses of action. The committee also recommends that advertis- ing message strategies be evaluated in terms of their effects on targeted

OCR for page 1
EXECUTIVE SUMMARY 5 beliefs and values. Such evaluation should make use of experimental designs in controlled setting and small-scale, in-market experiments. Advertising should be evaluated in terms of thematic content in order to determine whether its effects vary by content as well as by impressions and expenditures. Determining Optimal Levels of Advertising and Recruiting Resources Many of the most important determinants of enlistment supply, as well as the cost and effectiveness of existing recruiting resources and the trade-offs among them, have been addressed by a well-developed body of econometric research. Econometric methods are most appropriate for studying existing programs and for developing estimates based on natu- ral variation in resources and outcomes. There have been several studies over the past 20 years on the effects of recruiting and advertising on enlistment. The variability among these studies suggests that a consistent methodology has not been incorpo- rated. Based on our review of the literature, the committee concludes that: Recruiter productivity varies with experience, and hence sudden changes in the size of the recruiting force result in declines in average experience. Failure to incorporate recruiter experience into models of recruiter effects may bias study results. Recruiter incentives have been incorporated in supply models via recruiters' quotas, based on the assumption that increasing recruiting quotas increases effort. A more complete and realistic model of recruiter incentives is needed. Research to date has not incorporated the effects of Reserve com- petition on active-duty recruiting. The committee also concludes that the functional forms (i.e., the shape of the relationship between the recruiting incentive and enlistment) of econometric supply models have been relatively restricted. The under- lying assumptions (e.g., that each additional advertising dollar has the same effect regardless of the level of total expenditure) may not be correct, and an examination of more flexible functional forms would be fruitful. The committee therefore recommends that research on supply models make use of flexible functional forms, rather than imposed functional forms. A theme underlying all of the suggested areas for improvement is the need for better data, consistently collected and retained over time. Ideally, these data should include enlistment contract and accession data, by level of qualification and Service, at the lowest reasonable level of aggregation

OCR for page 1
6 EVALUATING MILITARY ADVERTISING AND RECRUITING and time period. The data should also include information on the re- sources and incentives that have been applied and the external factors that were in effect during the period, as well as indications of recruiting policies, incentives, and quotas. Timing and Levels of Joint and Service Advertising Econometric methods are also applicable to the questions of (1) whether there is a minimum level of advertising necessary for a cost-effective recruiting program even if it is not necessary to achieve current enlist- ment contract goals and (2) what are the proper levels of joint and Service- specific advertising. Quasi-experimental methods can also be employed to address the second question. We describe the conditions under which it would be cost-effective to advertise in the interests of future enlistment supply and review research to date on this question. While this research suggests that advertising may have effects only for a short period of time, the data available to previous researchers are limited as they do not permit the examination of time-lagged and nonlinear effects within a time period. In the committee's view, it would be a serious mistake to view the available research as sufficient grounds for drawing conclusions one way or another about the effects of current advertising on future outcomes. The committee recom- mends a focused effort to maintain advertising data in a systematic way for purposes of estimating a supply curve that incorporates the potential for both time-lagged and nonlinear advertising effects. The committee also recommends a program of research, incorporating quasi-experimental methods, to examine advertising effects over an extended period of time. Regarding the question on the appropriate levels of joint and Service- specific advertising, in our judgment, certain types of advertising themes, such as generic themes designed to increase overall propensity, are best done as a joint program, while advertising themes featuring specific ben- efits of military service are best done in the Service program. What we do not know is what level of advertising funding should be allocated to joint programs. We therefore recommend a program of research aimed at ex- amining the effects and cost-effectiveness of information-oriented versus values-oriented advertising in joint and Service advertising programs. Determining Optimal Types of Incentives The Services have many types of enlistment incentive programs in operation today, most of which are aimed at highly qualified recruits. Current incentives include education benefits as well as enlistment bonuses for various types of jobs. The Services are experimenting with

OCR for page 1
EXECUTIVE SUMMARY other incentives, particularly those that forge a closer connection between military and college pursuits, and also those that offer differing lengths of active-duty and Reserve tours. We review research on various incentive programs, both old and new, in terms of the different types of research designs that have been or are being used for studying this topic: survey, experimental, and econo- metric. The central message is that each of the evaluation methodologies discussed in previous chapters can play a useful role in addressing differ- ent questions that policy makers may ask about current or proposed incentives. It is also important to note the value of combining approaches in examining a particular program. For example, focus groups can be used to explore prospective enlistment options, which are then tested with a large survey of youth or in pilot tests employing experimental designs. Performance Management of Recruiters Recruiter performance management encompasses the range of issues and decisions that face Service recruiting managers as they organize to meet their mission. Service recruiting managers establish systems to select and train recruiters, to open recruiting offices in specific locations, to establish production goals, to motivate recruiters with various incentive programs, and to monitor and assess recruiter performance. In some of these areas, such as developing recruiter selection programs, there is a large body of ongoing research in both the military and civilian sectors. In other areas, such a developing effective incentive programs, research efforts are minimal. In many respects, the problems of performance management faced by the military are no different from the problems faced by private industry. However, the environments are distinctly different, and the military faces many restrictions that do not apply in the civilian sector. As a result, some of the existing research from the professional literature will be of limited use. Ideally, the military should undertake a continuous and systematic evaluation of each aspect of performance management system individu- ally and in combination. Specifically, the committee recommends: Continued research on the development of effective recruiter selection strategies, in conjunction with a consideration of career incen- tives for service as a recruiter. Expansion of the Services' evaluation of overall training of recruiters to include the study of other informal development opportunities. In particular, assessment and improvement of the supervisory and coaching

OCR for page 1
8 EVALUATING MILITARY ADVERTISING AND RECRUITING skills (to include on-thejob training) of those who train recruiters may be a fruitful approach. A program of research aimed at evaluating the effects of goals on recruiter behavior and outcomes. Research to develop a complete model of recruiter performance and to develop performance appraisal instruments and feedback pro- cesses based on this model.