Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 9
1
~ ntroduction
During the late l990s, the armed forces began having difficulty
meeting recruiting targets, particularly for highly qualified
recruits youth with high school diplomas and above-average
aptitudes who are essential for an effective fighting force. Recruiting
problems were most severe during 1998 and 1999, when most of the
Services experienced accession shortfalls. These shortfalls were especially
alarming to military planners, given the major force reductions of the
l990s and the lower accession requirements that followed (Sellman, 1999~.
In 1999, at the request of the U.S. Department of Defense (DoD), the
National Research Council formed the Committee on the Youth Popula-
tion and Military Recruitment. During Phase I of its study, the committee
was asked to examine long-term trends in the youth population and evalu-
ate policy options that could improve the propensity for and enlistment
in the Services. In its Phase I report, Attitudes, Aptitudes, and Aspirations of
American Youth, the committee confirmed the decline in propensity for
military service among youth and identified several correlates, especially
the increasing trend in college enrollments. The report made specific
recommendations about changing advertising programs and recruiting
policies to improve propensity and enlistment.
During the earlier phase, the committee observed that military re-
search on advertising and recruiting programs is often opportunistic and
fragmented, lacking coordination and long-term objectives. In this report,
the committee aims to fill this gap by proposing a comprehensive evalua-
tion framework to assist and improve research on recruiting policies and
advertising programs. This report contains results, conclusions, and rec-
ommendations from the committee's Phase II study.
9
OCR for page 10
10
EVALUATING MILITARY ADVERTISING AND RECRUITING
PURPOSE OF THE STUDY
The primary objective of this study is to help DoD improve its re-
search on advertising and recruiting policies. It is anticipated that in the
coming decade DoD will field and test new advertising and recruiting
initiatives designed to improve the recruiting outlook and help avoid the
shortfalls of the past decade. In order to discover the most promising
policies, in the committee's view the department needs a comprehensive
research and evaluation strategy based on sound research principles that
will ensure valid, reliable, and relevant results.
Good research should begin with developing a clear statement of the
problems to be studied, because the nature of the policy questions influ-
ences and shapes the most appropriate research designs. While a given
research question can be studied using a variety of approaches and tech-
niques, some research designs in our view are more promising than others
for particular research questions. For example, focus groups are most
useful when determining what a target audience finds attractive about a
program, whereas econometric methods are appropriate when studying
the effects of an existing program on enlistments.
Good research requires more than sound methodology; it must also
be grounded in solid theory and valid concepts. In the case of recruiting
and advertising policies, the most important theory concerns individual
decision-making processes, particularly the factors than influence job and
career decisions. The research program proposed here reviews and devel-
ops the most relevant theories for enlistment decisions.
Finally, advertising and recruiting research must produce results that
are useful to military planners. Research results must be available in a
timely manner to help planners make decisions among competing pro-
posals and to assist during the programming and budgeting cycles of the
department. In the committee's view, the research and evaluation studies
proposed in the following chapters can meet these needs.
The remainder of this introduction is an overview of issues and
methods and proposes a framework for selecting the most appropriate
method to address various types of research questions.
.
THE EVALUATION PROBLEMS
The committee's work during Phase I led us to conclude that there are
a number of critical problem areas or topics needing more intensive study.
Some problems arise because of the need for ongoing, up-to-date infor-
mation that can serve as early-warning indications of potential recruiting
problems or that can point to areas in which improvements are needed.
Other problem areas are important because they are central to improving
OCR for page 11
INTRODUCTION
11
the overall recruiting climate. We have selected six areas to be the central
focus of this report.
Monitoring Trends in Youth Attitudes, Values, and Propensity
The issue of youth attitudes has, in fact, received considerable em-
phasis from DoD. Youth attitudes, values, and propensity for military
service were monitored by the Youth Attitude Tracking Studies (YATS)
from 1975 to 1999. A new effort to survey youth attitudes on a quarterly
basis began in 2001 as part of the Defense Marketing Research program
(Sellman, 2001~.
Further monitoring of youth attitudes is important for several reasons.
First, this information can be used as leading indicators of future changes
in enlistment behaviors. For example, YATS documented a downturn in
youth propensity that presaged the 1998-1999 recruiting shortfalls by at
least six years.
Second, the YATS methodology was not optimal, as was documented
in the committee's letter report of tune 2000. The once-per-year format
hampers the assessment of effects on propensity arising from specific
events, such as the Persian Gulf and the latest Iraq wars. In addition,
YATS administered different attitude and value questions to different
subsamples of youth at different times, making it very difficult to analyze
the relationship between attitudes or values and enlistment propensity.
Finally, it is not clear that YATS adequately covered the domain of values
and attitudes that influence career decisions.
DoD should continue to place a high priority on studies of short- and
long-term trends in youth attitudes, values, and propensity. The new
Defense Marketing Research program (Office of Assistant Secretary of
Defense [Force Management Policy], 2000) has improved the frequency of
administration, but issues remain regarding the collection of trend data
on key values and attitudes relevant to military service. The methodology
used in these studies should ensure timely, reliable, and relevant infor-
mation that can be used for anticipating potential problems in enlistment
supply.
Developing Effective Advertising Themes to
Increase Youth Propensity
The committee's Phase I report concluded that current advertising
strategies and themes are not designed to maintain a base level of propen-
sity, and this may be contributing to the decline in youth propensity for
military service in light of alternatives (e.g., college, employment).
OCR for page 12
12
EVALUATING MILITARY ADVERTISING AND RECRUITING
Accordingly, one of our most important recommendations was to develop
effective advertising themes to help increase the propensity of youth for
military service.
Solving this problem requires two types of research. The first is devel-
opment of promising advertising themes using generative research tech-
niques; the second is thoroughly and rigorously testing the market impact
of the selected themes. Again, the Services and DoD have conducted
research in this area, but many improvements can be made in the meth-
odologies applied to date, and these improvements can have a real impact
on the development of effective advertising themes and message delivery
strategies.
Optimal Levels of Recruiting Programs and Resources
The Services have many existing policies and programs designed to
maintain adequate levels of enlistment supply. These include traditional
and on-line advertising, various incentive packages (education benefits
and enlistment bonuses), and number and location of recruiters. The
policy problem is how to allocate these resources to provide effective and
efficient approaches for maintaining a given supply of highly qualified
recruits.
Results of optimal mix studies are extremely useful for allocating
resources among various existing programs during the budget planning
cycles. While the Services have conducted some research in this area,
many existing studies have not adopted the strongest research designs or
incorporated the most reliable data. Improving the quality and reliability
~ en · 1 1 1 1 · · r- ~ rr · 1 1 · ~ · 1 en
of this research would nave s~gn~cant payoff In helping to provide the
most efficient mix and level of existing recruiting resources.
Optimal Investment in loins and Service Advertising
The timing and level of advertising are somewhat related to both
advertising planning and optimizing the levels of recruit programs and
resources, but we discuss them separately to emphasize two important
areas of research that have not been adequately studied to date.
First is the issue of level the minimum level of advertising needed to
maintain youth propensity even during periods when enlistment supply
is good. Service enlistment goals can fluctuate from year to year for vari-
ous policy reasons, and enlistment supply is also influenced by changes
in external factors (e.g., unemployment). During good supply years, mili-
tary planners often reduce recruit advertising below levels necessary for
OCR for page 13
INTRODUCTION
13
maintaining the propensity base, thereby creating an advertising vacuum
that lowers propensity and supply in difficult years.
The second issue is the question of the optimal combination of joint
(DoD-wide) and Service-specific advertising. Some advertising themes,
particularly those designed to maintain or increase the base level of pro-
pensity (e.g., intrinsic values such as duty to country), may be accom-
plished more efficiently as a joint program, while other advertising themes
(e.g., specific benefits of serving in a particular Service) are more appro-
priate for Service-specific programs. What we do not know, because of a
paucity of research, is what combination of joint and Service advertising
would be optimal for meeting both types of goals.
Improving Enlistment Incentives
The Services offer a wide assortment of enlistment incentives at the
present time, including benefits for college education and enlistment
bonuses for particular military jobs. For the most part, these incentives
are similar to those introduced at the inception of the All Volunteer Force
in 1974. Given the many competing alternatives facing youth today, par-
ticularly the increasingly popular college option, there may be other types
of incentives that would enhance enlistment supply. For example, some
Services are experimenting with programs that tie military duty more
closely to college education, and other incentives being considered in-
volve changing the length of active-duty and reserve commitments with
the potential for enrollment in civilian national service. Research on new
types of incentives, or new combinations of incentives, is very important
to meet the increasing competition from college and other work oppor-
tunities.
Performance Management of Recruiters
Existing research has been very suggestive, but not definitive, about
the impact of recruiter organization and performance on enlistment sup-
ply. By recruiter organization we mean such policies and practices as the
number of recruiters, their geographic location, how they are selected
from and returned to other military units, how they are trained, their pay
and benefits, and the setting of recruiting goals or targets. While there is
some existing research on recruiter organization and performance, com-
prehensive studies do not exist. There is potential for substantial payoffs
from research that focuses on how the performance management of
recruiters can increase enlistment supply in a very cost-effective manner.
OCR for page 14
14
EVALUATING MILITARY ADVERTISING AND RECRUITING
THE EVALUATION FRAMEWORK
Our evaluation framework is based on the fundamental notion that
different research designs are suited to answer different types of research
questions. What appear to be disagreements about the relative merits of
econometric models versus experiments versus survey methods tend to
diminish if the question is framed as "What research approaches are best
suited to answering the following research question?" rather than "What
is the best approach to doing research?"
We outline a number of common types of research questions that
emerge in the context of military recruiting. Note that this is not intended
as a general or complete taxonomy of research questions; other types of
questions may arise in other research contexts.
The research questions are placed in a two-dimensional framework
(Table 1-1~. The first dimension indicates whether the research question
focuses on an existing incentive, program, or activity or a proposed new
incentive, program, or activity (hereafter summarized generally as "pro-
gram"~. For example, an existing program might be a current advertising
campaign, while a new program might be a proposed new enlistment
. .
ncenhve.
The second dimension differentiates among three types of assess-
ments of new or existing programs. The first type focuses on settings in
which the outcome variable of interest is not specified a priori. It addresses
what a target audience sees as attractive and unattractive features of a
current or proposed program. The second type of assessment focuses on
specific attitudinal or behavioral intent variables, such as one's perceived
benefits of military service or one's intention to enlist. It addresses the
effect of existing or proposed programs on attitudes or behavioral inten-
tions. The third type of assessment focuses on actual behavioral outcomes,
such as an enlistment decision or contact with a recruiter. It addresses the
effect of existing or proposed programs on a behavior of interest.
Each of the cells in this two-dimensional framework lends itself to
some research methods more than others. Some cells may preclude the
use of certain methods.
The framework identifies focus groups or open-ended surveys and
interviews as the most viable method when the research question seeks
opinions about and reactions to a current or proposed program. The use
of such methods makes sense when the researcher has not specified in
advance the attitudinal or behavioral variables of interest. The reactions
obtained from individuals in focus groups may be completely unantici-
pated by the researchers. Such a strategy is perhaps most generally useful
in the early stages of considering a new program, although it may be
usefully applied to existing programs, particularly those that were imple-
OCR for page 15
INTRODUCTION
TABLE 1-1 Evaluation Framework
15
Outcome Not
Specified A Priori
Specific Attitudes
or Behavioral
Intentions
Actual Behavior
Question: "What does
programs a target audience
see as attractive or
unattractive features
of a program?"
Method: focus groups;
unstructured or
open-ended surveys
and interviews
Existing
programs
Question: "What is
the effect of a
program on
specified attitudes
or behavioral
intentions?"
Method: surveys;
experiments;
. · .
quas~-exper~men~s
Same as above
Same as above
Question: "What is
the effect of a
proposed new
program on
enlistment?"
Method:
experiments;
. · .
quas~-exper~men~s
Question: "What is
the effect of an
existing program
on enlistment?"
Method:
econometric
modeling
mented without the use of systematic evaluation in their initial design
and implementation. Chapter 5 offers further development and illustra-
tion of the use of such qualitative research methods in the context of
developing advertising campaigns.
The framework identifies surveys as a useful method when the re-
search question deals with the effect of existing or proposed programs on
attitudinal or behavioral intent variables of interest. Survey questions can
be administered in various media, from paper-and-pencil questionnaires
to telephone interviews to administration via computer. The key distinc-
tion with the previous category is that the researcher has clearly specified
in advance the attitudinal or behavioral intent variable of interest. Thus if
a portfolio of possible new recruiting incentives is being considered, and
the researcher is interested in the effects of each on an individual's stated
likelihood of enlistment, the use of survey methods would be particularly
appropriate. Note that experimental and quasi-experimental designs can
be usefully employed in such settings, with different groups of partici-
OCR for page 16
16
EVALUATING MILITARY ADVERTISING AND RECRUITING
pants asked to respond via survey to different proposed incentives.
Survey methods are also useful in monitoring whether existing programs
continue to generate the same attitudinal reaction over time.
It is often the case that the attitudinal or behavioral intent variables of
interest are not of primary interest in and of themselves; they are of inter-
est because of a documented link to actual behaviors of interest, such as
enlistment. Because of this link to behavior, insight into the potential
effects of programs can be obtained prior to actual program implementa-
tion. Thus, for example, a wide array of different bundles of recruiting
incentives could be presented to research participants to obtain insight
into which is seen as most attractive. Note, though, that while informa-
tion on the relative attractiveness of different options can be quite useful
in deciding which ones to implement or to research more extensively, this
method does not result in a point estimate of the effects of the program on
enlistment (without research linking survey responses to actual enlist-
ment, as discussed in Chapter 7~. Chapter 3 further develops and illus-
trates the use of survey methods.
The final two cells in the framework focus on behavior; in the military
recruitment context, enlistment is the behavior of greatest interest. It is in
research questions focusing on enlistment behavior that the distinction
between the evaluation of new programs and the evaluation of existing
programs becomes a critical issue. Econometric modeling methods are
well suited to the evaluation of ongoing programs, as they are based on
actual variations in a set of observable conditions and relating that varia-
tion to behavioral outcomes of interest. For example, the effects of adver-
tising expenditures on enlistment can be examined by relating changes in
expenditures over time to patterns of enlistment, statistically controlling
for other relevant time-varying features. Chapters 5 and 6 further develop
and illustrate econometric modeling.
In contrast, determining the effects on enlistment of a new program
prior to full-scale implementation or the differential effects of alternate
possible new programs are not amenable to examination via methods
relying on observed variation in the features of interest, because the
programs have not yet been implemented. Thus approaches involving
manipulation of features of interest are needed, namely, experimentation
(manipulation with random assignment) or quasi-experimentation
(manipulation without random assignment). For example, different
advertising content could be used in different geographic regions to
examine effects on enlistment. Chapters 4 and 7 further develop and illus-
trate experimental and quasi-experimental approaches.
While the framework presented here identifies broad types of re-
search questions for which particular research methods are well suited, it
is not our intent to draw bright-line distinctions among the methods.
OCR for page 17
INTRODUCTION
17
Another message of this volume is the relevance of multiple methods to
many key issues in military recruiting. Chapters 6, 7, and 8 all illustrate
areas in which multiple methods are useful. In addition, aspects of different
methods can usefully be combined. For example, quasi-experimentation
does not ensure the equivalence of conditions that are obtained via random
assignment, and some of the quantitative tools of econometric modeling
can be very useful in controlling for differences among conditions.
In the next chapter we elaborate more fully on similarities and differ-
ences between econometric and experimental approaches to research.
Again, it is the nature of the research question that should drive the
choice of research method or methods, rather than allegiance to any par-
ticular methodological orthodoxy.
Representative terms from entire chapter:
military service
