Part III:
The Cost/Performance Trade-off Model

As stated in the overview of this report, the JPM Project and the related cost/performance trade-off model grew out of the misnorming of the ASVAB, the congressional interest in the relationship between job performance and recruit quality, and the lack of a clear, coherent framework for establishing recruit quality goals. Throughout the 1980s, the continuing question was: How much quality is enough? With the current level of high quality, the downsizing of the force, and the supply of high-quality recruits exceeding demand, the question now becomes: What level of quality is most cost-effective? The cost/performance trade-off model has been developed as a tool to aid analysts and policy decision makers in answering questions about recruit quality needs and in justifying the costs associated with selected quality mixes. Here we provide a context for the personnel planning process.

Personnel planners in the Services attempt to staff the force structure—divisions, air wings and battle groups, and the supporting infrastructure—with the numbers and types of people necessary to maintain desired readiness levels As part of this process, the Services must annually recruit numbers of new entrants to the enlisted force to replace those who leave or retire and to reflect planned growth or shrinkage in the overall size of the force. In today's All-Volunteer Force, the military services annually recruit about 200,000 young men and women to become soldiers, sailors, airmen, and marines. These young people receive basic military training and specialized skill training in more than 900 different military occupations. For



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop Part III: The Cost/Performance Trade-off Model As stated in the overview of this report, the JPM Project and the related cost/performance trade-off model grew out of the misnorming of the ASVAB, the congressional interest in the relationship between job performance and recruit quality, and the lack of a clear, coherent framework for establishing recruit quality goals. Throughout the 1980s, the continuing question was: How much quality is enough? With the current level of high quality, the downsizing of the force, and the supply of high-quality recruits exceeding demand, the question now becomes: What level of quality is most cost-effective? The cost/performance trade-off model has been developed as a tool to aid analysts and policy decision makers in answering questions about recruit quality needs and in justifying the costs associated with selected quality mixes. Here we provide a context for the personnel planning process. Personnel planners in the Services attempt to staff the force structure—divisions, air wings and battle groups, and the supporting infrastructure—with the numbers and types of people necessary to maintain desired readiness levels As part of this process, the Services must annually recruit numbers of new entrants to the enlisted force to replace those who leave or retire and to reflect planned growth or shrinkage in the overall size of the force. In today's All-Volunteer Force, the military services annually recruit about 200,000 young men and women to become soldiers, sailors, airmen, and marines. These young people receive basic military training and specialized skill training in more than 900 different military occupations. For

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop the military services and the taxpayer, this represents the beginning stages of a significant investment in recruiting resources, training resources, and compensation. Not all applicants are equally capable of completing basic training or the skill training necessary in the technically demanding jobs constituting an increasing proportion of the modern armed forces. The success achieved by the Services in recruiting the right kinds of young men and women will determine not only how large this investment will be, but also, in part, how effectively the armed forces will meet the challenge of defending the nation's interests. For these reasons, the Services have an incentive to be selective in recruiting only those who are likely to succeed. For individuals in the youth population, the opportunity to serve one's country is an important part of citizenship. It should not be limited or denied without compelling reasons. Moreover, for many of the nation's youth, service in the armed forces offers the chance to obtain valuable experience and training that could be a major determinant of the potential recruit's future economic well-being. Policies that determine how selective the Services will be in choosing applicants, therefore, are important, and entail a complex array of trade-offs. More selective recruiting policies tend to reduce the effective size of the youth labor market from which the Services may recruit, raising recruiting costs. Presumably, the investment in higher recruiting costs resulting from a more selective policy yields a return in the form of a more capable recruit—one who is more likely to complete training successfully and to perform well on the job. But, because these more selective policies result not only in higher initial recruiting costs, but also deny service opportunities to some willing applicants, the case for more selective recruiting policy should be well grounded in logic. Moreover, the empirical links between higher selectivity, performance, and cost should be well-established. The Services set their recruit selection policies in terms of applicants' scores on the Armed Forces Qualification Test (AFQT) and an applicant's level of education. The AFQT consists of 4 of the 10 subtests of the Armed Services Vocational Aptitude Battery (ASVAB), the enlistment screening and classification test administered to all applicants. The primary educational criterion is whether the applicant possesses a high school diploma. Those high school graduates scoring in the top half of the distribution on the AFQT (Categories I–IIIA) are considered high-quality applicants. Recruiting goals are set in terms of the proportion of high-quality recruits desired. A more selective recruiting policy typically means attempting to recruit a larger portion of high-quality applicants. The applicant's score on the AFQT, as well as education status, determines whether the applicant is qualified for the Service. In addition, the Services use various combinations of the ASVAB subtests to form composite scores that are

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop relevant to particular jobs or occupational categories. The applicant's score on the relevant composite must exceed the minimum score established by the Service for that occupation in order to be accepted for training in that particular occupation. Hence, the composite scores determine which jobs or occupational groups the applicant is qualified for. In practice, an individual's AFQT score is highly correlated with most other composite scores, so that the higher-quality applicant, as measured by AFQT, also meets the minimum qualifications for more jobs. A large number of studies have demonstrated that high-quality recruits—those with above average scores on the AFQT and a high school diploma—perform better in the military, whether performance is measured by training outcomes, job performance tests, speed of promotion, or first-term attrition. But high-quality individuals also cost more to recruit in the volunteer force environment, in which the military must compete with other employers for the services of talented individuals. Therefore, the Services attempt to set recruit quality goals that balance the higher performance and lower attrition costs of high-quality individuals with their increased recruiting costs. Figure 1 depicts the trade-off between costs and performance that the Department of Defense faces in setting recruit quality goals. As the level of recruit quality increases, both expected military performance and recruiting costs rise. Higher recruiting costs are offset by decreases in attrition-related costs. This figure also illustrates the two empirical linkages that must be established to provide the quantitative measures of the trade-off that are necessary for policy decisions. First, we must know how different levels of recruit quality affect military performance—the top half of the figure. And second, we must understand how changes in average recruit quality affect the components of personnel costs shown in the bottom half. Understanding these linkages is important for two reasons. First, there should be a solid rationale, grounded in performance and cost differences, for choosing among applicants for military service. To deny a citizen the opportunity to serve his or her country is a serious matter that must be justified with compelling reasons. Second, Congress, as the agent of the taxpayer, has insisted that DoD be able to justify, in terms of increased military performance, the costs of a higher-quality enlisted force. Understanding these linkages is necessary to achieve the maximum return from a declining defense personnel budget. The first paper in this section, prepared by D. Alton Smith and Paul F. Hogan, provides an overview of the cost/performance trade-off model. The second paper, by Paul F. Hogan and Dickie A. Harris, presents a discussion of the policy and management applications of the model.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop Figure 1 Accession quality cost/performance trade-off.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop The Accession Quality Cost/Performance Trade-off Model D. Alton Smith and Paul F. Hogan INTRODUCTION This paper describes a model that defines the quantitative linkages between recruit quality, military performance, and personnel costs and uses that information to determine the cost/performance trade-off options available to the Department of Defense (DoD). The model was jointly developed by the Systems Research and Applications (SRA) Corporation and the Human Resources Research Organization (HumRRO) for the Office of the Assistant Secretary of Defense (Personnel and Readiness). It builds directly on the results from the Joint-Service Job Performance Measurement/Enlistment Standards (JPM) Project, which developed and implemented handson job performance tests for enlisted personnel. The Accession Quality Cost/Performance Trade-off Model has many potential applications in accession planning. In particular, it can assist policy makers in: Building an accession program, which is the target number of accessions by quality category and occupation group and the associated recruiting resource requirements. Evaluating the performance and cost implications of changes in the accession program caused by variation in manpower requirements, recruiting market conditions, or budgets for Service recruiting efforts. Efficiently setting job classification standards, the cutoff scores con-

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop structed from the Armed Forces Vocational Aptitude Battery (ASVAB) used to screen entrants into particular occupations. Assessing the savings potential associated with new processes and tools for selecting and classifying military recruits. Other papers in this volume describe these applications in more detail and provide examples of the results generated by the model. This paper is organized as follows. The first section describes the components of the model and identifies some of the strengths and weaknesses of the analytical approach. This is necessarily a summary. More details on the structure of the model and the underlying research can be found in McCloy et al. (1992). To facilitate its use in policy analysis, we have implemented the analytical model for all four Services in a microcomputer software program, which is described in Human Resources Research Organization et al. (1993). The second section uses the model in two different types of validation tests: comparing model results against actual accession cohorts and varying individual elements in the scenario defining an optimization run. The final section provides a summary of the paper. DESCRIPTION OF THE MODEL The structure of the model is shown in Figure 1. It selects, for a set of occupation groups, the number of accessions by recruit quality category that minimizes the sum of recruiting, training, and compensation costs while meeting first-term performance and personnel strength goals. Because the quality-performance and quality-cost relationships vary significantly across the Services, a separate version of the model has been developed for each Service.1 It is important to emphasize that the model does not choose the best overall recruit quality level, only the cost-minimizing level for a given amount of performance. Selecting the best overall level requires information on the dollar value of performance so that performance gains can be weighed directly against increased costs. In a market setting, in which profit-maximizing firms compete for the services of employees, one can argue that the compensation paid to workers represents the value of their performance. It is much more difficult to make the same claim for public- 1   We are not the first to develop an accession cost/performance trade-off model. The essential structure for such a model is described by Steadman (1981). Cost/performance tradeoff models for a small number of Army occupations were implemented and tested by Armor et al. (1982) and Fernandez and Garfinkle (1985). Accession quality models are also currently under development by several of the Services.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop Figure 1 Structure of the model. sector organizations that do not compete. Without using what must be arbitrary performance valuations, the model can be used to define the efficient cost and performance combinations, an improvement in the information currently available to policy makers.2 In mathematical terms, the model is a constrained minimization problem with the three elements: accessions by recruit quality category and occupation, which are the variables for which we solve; performance and personnel strength goals, which define the constraints on the problem; and personnel costs, which comprise the objective function. We describe the model in terms of these three elements. An explicit mathematical statement of the model is found in the technical appendix at the end of this paper. 2   For an example of a recruit quality model that employs a performance valuation function, see Nord and Kearl (1990).

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop Variables: Accessions by Recruit Category and Occupation Group The model solves for 360 variables for each Service—accessions in 10 recruit quality categories for each of 36 occupation groups. Recruit Categories Although there are many ways to categorize recruit quality, our choice should be driven by the policy applications of the model. Therefore, we use the Armed Forces Qualification Test (AFQT) score and high school graduation status, the two characteristics used to establish recruiting goals and measure recruiting performance for DoD. In particular, 10 recruit quality categories are defined by the interaction of 5 standard AFQT score groups (called Category I, II, IIIA, IIIB, and IV) and 2 high school graduation groups (those with high school diplomas and those without).3 Additional categories are often used in managing the recruiting process. For example, all Services track enlistments separately by gender because some occupations, by regulation or statute, cannot be filled by women. Although the model would be more useful to accession planners if its recruit categories were also defined by gender, there is little research measuring the linkage between female accession quality and recruiting costs. Without information on both the quality-performance and quality-cost relationships, we cannot add more recruit categories to the model. Occupation Groups Both the quality-performance and quality-cost linkages vary with military occupation, which means that the minimum-cost level of recruit quality will also vary by occupation. As an example, consider two occupations with different training costs but the same overall performance goal. The least-cost solution for staffing these occupations will have fewer recruits, of a higher average quality, in the occupation with high training costs. Because of their lower turnover, fewer high-quality recruits have to be trained to generate the same amount of performance, conserving training resources. Given these differences, it is important to understand how high-quality recruits should be allocated across occupations. Ideally, the aggregate quality requirement for a Service would be determined as the sum of the levels of quality needed in each enlisted occupation 3   Those high school graduates without regular diplomas, such as individuals with General Educational Development (GED) certification, are grouped with the nongraduates, who have similar performance and attrition characteristics.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop in that Service. This would yield not only aggregate recruit quality goals but also targets for the process of classifying recruits into occupations. We use grouped, rather than individual, occupations in the model for two practical reasons. First, as described by McCloy in this volume, the performance data used in our model cannot support distinct estimates of the relationship between performance and entry characteristics for every enlisted occupation in DoD. And second, increasing the number of occupations also increases the number of variables in the model. With up to 350 occupations in each Service, finding a solution to the constrained optimization problem becomes time-consuming enough to detract from the model's usefulness as a policy analysis tool. There are 36 occupation groups in the model, which are defined hierarchically. At the top, occupations are divided into nine groups based on one-digit DoD occupation codes.4 These codes provide a common classification of occupations across Services, facilitating the definition of performance and strength goals. Each of these nine groups is further divided into four subgroups on the basis of training costs and an index of occupation characteristics.5 These subgroups were chosen to increase the within-group homogeneity of the occupation groups on variables that are central to the cost-minimization problem, such as training costs. The more homogeneous the groups, the less error is introduced into the recruit quality solution by the aggregation of individual occupations. Constraints: Performance Goals by Occupation This is one of the more complex parts of the model. We look first at the definition of performance used in the model, then describe how performance is measured by occupation group and recruit quality category, and finally discuss the setting of performance goals. Definition of Performance In the model, performance for an occupation group is the sum of expected hands-on performance test scores over the first term of service for 4   The one-digit DoD occupation codes included in the model are: infantry, gun crews, and seamanship specialists; electronic equipment repairers; communications and intelligence specialists; health care specialists; other technical and allied specialists; functional support and administration specialists; electrical/mechanical equipment repairers; craftsmen; and service/supply handlers. 5   Specifically, a performance index based only on occupation characteristics is constructed from the performance equations described below. This index indicates how a given individual's performance would vary if assigned to different occupations.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop recruits assigned to that group. Specifically, Pi, the performance value for occupation group i, is defined as where Aij is the number of accessions from recruit category j into occupation group i. The size of an accession cohort for a Service is equal to the sum of all A's, which, in fiscal 1990, ranged from 90,000 for the Army to 34,000 for the Marine Corps. is the survival rate to year of service (YOS) t for a category j recruit in occupation group i. Survival rates are the proportion of accessions still in the military at time t; they range from 0 to 1 and decrease with time in service. The model uses a four-year initial term of service to calculate performance values. is the predicted hands-on performance test score at YOS t for a recruit from category j entering occupation group i. The test scores are the percentage of tested task steps completed correctly and can be roughly interpreted as the percentage of the job done correctly. Therefore, the scores can potentially range from 0 to 100. We discuss the approach used to predict these scores below. The performance value for an occupation group is the result of two calculations. The term in parentheses is the survival-weighted sum of predicted performance test scores over the first term for individuals from a particular recruit category enlisting in that occupation group. As a simple example, assume that AFQT Category I high school graduates enlisting in the Communications and Intelligence group typically stay for four years, making the survival rates all 1, and score 75 on the performance test each year. The expected first-term performance value for these individuals would equal 300.6 In the second part, occupation group performance is calculated as the weighted sum across recruit categories of these expected scores, with weights equal to the number of accessions from each category. Occupation group performance is a function of the number and quality distribution of recruits into that group, with quality affecting performance both through the hands-on performance test score and the probability of 6   Specifically, the performance value equals the test score of 75 for first year times the survival rate for that year of 1, plus a score of 75 for second year times the survival rate of 1, etc.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop attrition. Although the use of hands-on tests represents an improvement over performance metrics based on training outcomes or job knowledge tests, this measure of occupation performance has four limitations: First, it reflects only the ability to perform selected tasks for each job at the entry level. It ignores other dimensions of military job performance, such as leadership potential. Second, the same amount of performance for an occupation can be obtained with different combinations of the size and average quality of an accession cohort. As a practical matter, staffing requirements restrict the flexibility to trade off between the number of recruits and their average quality. For example, each tank requires a crew of four to operate efficiently, regardless of the ability of individual crew members. Third, equation (1) assumes that the first and last high-quality individual added to an occupation yield the same gain in performance. In fact, the benefit to group performance of that first high-quality recruit is probably greater than the marginal benefit of those that follow.7 Fourth, military performance is, in most cases, a function of both personnel and equipment performance. When force structure (i.e., the number and type of military units) can be varied, the appropriate level of recruit quality will likely be different than that estimated for a fixed force structure.8 Fortunately, we can work around some of these limitations by using strength constraints in the model, as described below. Measuring Performance Constructing performance values in the model would be straightforward if observations on hands-on performance test scores were available for all occupations, by recruit category and time in service. However, the cost of developing and administering hands-on performance tests limited the data available from the JPM Project. Counting all four Services, we could use results from just 24 occupations to estimate the relationship between recruit 7   For a conceptual discussion of military performance measurement, see Black (1988). 8   For example, see Daula and Smith (1992). This study estimates the minimum-cost level of recruit quality needed to meet a given level of tank force performance, for which performance is measured by scores on a firing range. Recognizing the role of equipment leads to a higher recruit quality requirement because increasing the quality of tank gunners and commanders allows the same performance requirement to be met with fewer tanks, at a considerable cost savings.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop TABLE 12 Service Aptitude Composites, Fiscal 1990 ASVAB Subtests Army Composites Navy Composites Marine Corps Air Force Mechanical Comprehension (MC)   General Technical General Technical General (AR + WK+ PC) Arithmetic Reasoning (AR) Electrical (GS+ AR + MK + EI) Electrical Electrical Electrical Numerical Operations (NO) Clerical (PC + WK + AR + MK) Clerical Clerical Administrative (NO + CS + WK + PC) Math Knowledge MK) Mech/Maint (NO + AS + MC + EI) Mechanical (AS + MC + WK + PC) Mech/Maint (AR + AS + MC + EI) Mechanical (GS + 2AS + MC) General Science (GS) Combat (AR + CS + AS + MC) Basic Electronics (GS + AR + 2MK)     Paragraph Comprehension (PC) Field Artillery (AR + CS + MK + MC) Engine/Boiler/Machine (AS + MK)     Auto/Shop (AS) Operations/Food (NO + AS + MC + WK + PC) Mechanical Repair (AR + AS + MC)     Electronics Information (EI) Surveillance/ Communications (AR + AS + MC + PC + WK) Submarine (AR + MC + PC + WK)     Word Knowledge (WK) Skilled Technical (GS + MK + MC + PC + WK) Communications Technician (AR + NO + CS + WK + PC)     Coding Speed (CS) General Maintenance (GS + MK + EI + AS) Hospital (GS + MK + PC + WK)     The model does not include the 10 ASVAB subtests, or the composites built from those subtests, as variables or dimensions in the model. How-     fies, but appears especially well qualified based on his or her composite scores. Actual classification decisions also take into consideration the current staffing of the occupation, the convening dates for training classes, and the recruit's preferences, so that the matches that are made are less than perfect from the more narrow criterion of classification efficiency.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop ever, constraints can be set in the model on the minimum proportion of high-quality recruits in each of the occupational groups, but a floor on the minimum AFQT score in the occupational group cannot be directly imposed. Ostensibly, then, it would be difficult to use the model to analyze occupation-specific, minimum composite scores or enlistment standards. However, the model does use the AFQT as the overall measure of aptitude and recruit quality and the AFQT is itself a composite score consisting of the mathematical and verbal subtests: paragraph comprehension (PC), word knowledge (WK), arithmetic reasoning (AR), and mathematics knowledge (MK). Moreover, the individual subtest scores on the ASVAB are positively correlated, some of them highly so. Hence, in general, for a given composite, the higher the cutoff score, the higher the average quality of recruits entering that occupation, as measured by the AFQT. We are able to analyze the effect of job-specific enlistment standards, defined by minimum cutoff scores on relevant composites, by approximating the effect that the cutoff scores would have on the minimum proportion of high-quality recruits entering that occupational group. We do this by establishing two links: a link between applicant's composite scores and the applicant's AFQT score and a link between the AFQT-equivalent minimum cutoff score of an occupation and the minimum proportion of high-quality recruits in that occupation. The first relationship is based on a strong correlation, in most cases, between an applicant's AFQT score (which itself is the linear combination of four ASVAB subtests) and composite scores. We can predict, with a reasonable degree of accuracy, the AFQT-equivalent of the various composite scores. The second is based on a statistical relationship between the AFQT-equivalent minimum score and the observed proportion of high-quality recruits in that occupation. Hence, we set enlistment standards in terms of a minimum percentage of high-quality recruits that, in turn, is statistically related to the underlying composite scores. It is the relationship between the composite cutoff score and an estimate of overall high-quality recruits, as measured by the AFQT, that we wish to explore using the model. To do this, we use the overall correlation between AFQT and the various composite scores to construct an AFQT-equivalent of each composite score. We then use the observed relationship between the AFQT-equivalent cutoff score for the occupational category and the proportion of high-quality recruits in the occupation to analyze enlistment standards and the implications of changing those standards. Recognizing throughout that the relationship between underlying occupational enlistment standards, as measured by cutoff scores on the composites, and the proportion of high-quality recruits in the occupational category is less than perfectly precise, the types of analyses that can be conducted using the model include:

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop Comparisons of the proportion of high-quality recruits in each occupational category prescribed by the model to the proportion that occurs under current enlistment standards, under otherwise similar conditions; The implications for overall recruit quality goals and for the costs of imposing a minimum proportion of high-quality recruits implied by current enlistment standards; and The implication for overall recruit quality goals and for the costs of raising (or lowering) enlistment standards in one or more occupational categories. We describe below how the model can be used to provide insights into the implications of occupation-specific enlistment standards. In particular, we can compare the costs of an unconstrained solution—one with no occupation-specific minimum standards—to a solution in which the equivalent of the current, occupation-specific enlistment standards are imposed, the costs associated with raising (or lowering) one or more cutoff scores, and the likely distribution effects of inefficiently high standards (according to the model) in some occupational groups. The Empirical Linkages: Linking Occupational Enlistment Standards to High Quality12 Two types of equations will connect occupation-level enlistment standards, as measured by composite cutoff scores, to the proportion of high-quality recruits in an occupational category. These are a set of equations relating AFQT to composite scores and an equation relating the proportion of high-quality recruits (% HQ) in an occupation to the cutoff score: and The first equation is estimated using data from individual applicants and is a regression of an individual i's AFQT score on that individual's score on composite j. The second equation, estimated at a more aggregate level, establishes the relationship between the cutoff score in an occupational category as its predicted AFQT-equivalent and the percentage of high-quality recruits (AFQT Category I–IIIA high school graduates) in the occupa- 12   This section provides a brief overview of some of the underlying empirical relationships developed to link the model to enlistment standards.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop TABLE 13 Regression of AFQT Scores on Composites with Goodness of Fit As Measured by R2 Army   Navy   Marine Corps   Air Force   Composite R2 Composite R2 Composite R2 Composite R2 General technical. 94 General technical .94 General technical .88 General technical .97 Electrical .89 Electrical .89 Electrical .89 Electrical .90 Clerical .98 Clerical .65 Clerical .85 Administrative .66 Mech/maint .61 Mechanical .65 .Mechanical .65 Mechanical .49 Combat .72 Basic electronics .93         Field artillery .86 BT/EN/MM .72         Operations/food .72 Mech rep. .65         Surveill/comm .77 Submarine .88         General maint .74 Comm. tech. .80         Skilled tech. .89 Hospital .92         tional category. The AFQT cutoff score is obtained by using the first equation to translate the composite score minimum to an AFQT score. In general, the relation between AFQT and composite score is quite strong, as measured by R2, for most composites, as illustrated in Table 13. Occupations in the model are aggregated into nine groups corresponding to one-digit DoD occupation codes. Using the relationship estimated between composite scores and AFQT, we compute the AFQT-equivalent of the minimum cutoff score for each DoD occupation code. This is done by computing the weighted average of the AFQT-equivalent cutoff score for the Service occupations within a DoD occupation category, in which the weights are the proportions of recruits entering that occupation in fiscal 1990. Table 14 shows the AFQT-equivalent minimum score required for each DoD occupation category using this method. Finally, occupation-level quality constraints in the model (the equivalent of minimum enlistment standards) are not in terms of AFQT scores, but rather are in the form of the proportion of high-quality recruits—the proportion of AFQT Category I–IIIA high school graduates. Moreover, one of the important outcomes of enlistment standards by occupational group is the proportion of high-quality recruits that enters that occupational category, partly as a result of the standard. For this reason, we developed an equation predicting the proportion of high-quality recruits that enter an occupation as a function of the AFQT-equivalent cutoff score. A regression equation describing the relationship between the weighted-average AFQT minimum cutoff score in an occupation and the proportion of high-quality recruits in that occupation is estimated by using the actual

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop TABLE 14 AFQT-Equivalent Average Enlistment Standards by DoD Occupation Code Average Implied AFQT Cutoff Scores DoD Occupation Code Army Navy Marine Corps Air Force (0) Infantry, gun crews 32.13 39.08 29.32 32.13 (1) Electronic equip, rep. 55.29 62.96 67.23 63.3 (2) Comm/intel 43.45 51.97 45.45 49.88 (3) Medical/dental 44.87 44.22 — 45.22 (4) Other technical 45.56 55.51 48.01 46.64 (5) Functional support/admin. 43.18 61.13 54.06 45.78 (6) Elect/mech. repairers 46.48 45.22 45.88 44.65 (7) Craftsmen 38.60 50.14 38.23 37.89 (8) Service/supply handlers 40.71 38.78 31.84 35.36 TABLE 15 Proportion of High-Quality Recruits as a Function of AFQT-Equivalent Enlistment Standards Service Intercept AFQT Cutoff Score Coefficient R2 Army 12.7 1.18 .49 Navy -19.9 1.51 .57 Marine Corps 19.4 1.11 .78 proportion of high-quality recruits by occupational category in fiscal 1990. For three of the Services, the proportion of high-quality recruits is regressed on the estimated AFQT-equivalent cutoff score for fiscal 1990.13 The results are shown in Table 15. Interpreted literally, these equations imply that a one-unit increase in the average enlistment standard of an occupational category, as measured by the AFQT-equivalent, is associated with a 1.18 percentage point increase in the proportion of high-quality recruits assigned to that category in the Army, a 1.51 percentage point increase in the Navy, and a 1.11 percentage point increase in the Marine Corps. Using these equations, we can estimate the percentage of high-quality recruits implied by the AFQT cutoff scores in effect in fiscal 1990 and compare this per 13   The Air Force is omitted because fiscal 1990 data were not available to us at the time of this writing.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop TABLE 16 Percentage of High Quality by Occupational Category, Fiscal 1990   Army   Navy   Marine Corps   Occupation Code Actual Predicted Actual Predicted Actual Predicted 0 56.8 50.6 45.2 39.1 50.0 51.9 1 81.8 77.9 78.4 75.1 90.9 94.0 2 72.7 63.9 63.8 58.5 62.5 69.8 3 78.5 65.6 58.8 46.8   — 4 61.1 66.4 76.5 63.9 87.1 72.6 5 63.5 63.6 59.8 72.4 76.9 79.4 6 56.7 67.5 53.5 48.3 68.9 70.3 7 53.1 58.2 39.7 55.8 68.6 61.8 8 52.6 60.7 22.6 38.6 49.6 54.7 centage to the actual percentage of high-quality recruits in that category. This comparison is shown in Table 16. Applying the Cost/Performance Trade-off Model to Enlistment Standards It should be emphasized that the model was not specifically designed for the micro analysis of occupation-specific enlistment standards, but for more programmatic issues involving aggregate recruit quality goals and recruiting resources. Hence, we recommend that the particular applications of the model that follow be considered experimental in nature, with a goal of examining the potential of the model in this direction and, if promising, the modifications that would be necessary to realize this potential. The performance goals and performance equations in the model do not capture all of the factors that influence readiness and job performance. Enlistment standards, represented by externally imposed, minimum cutoff scores for entry into an occupation, may capture factors affecting performance and readiness that are not included in the model. But there does not literally exist a composite cutoff score such that all those who score below that score will be unsuccessful, while all those above that score will be successful. Instead, the trade-offs are between probabilities of success or higher levels of performance and higher recruiting costs associated with more restrictive entry standards. In principle, enlistment standards by occupation should be determined by weighing the expected costs and the anticipated benefits. The model can highlight some of the cost implications of enlistment standards and changes in those standards and possibly help managers improve the way in which standards are set and revised.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop Effect of Current Enlistment Standards on Recruit Quality and Cost In this example, we compare an optimal fiscal 1990 Army accession plan when there are no explicit enlistment standards by occupation—the unconstrained case—with the same scenario when actual enlistment standards are approximated by the implied minimum proportion of high-quality recruits shown in the ''Predicted" column for the Army in Table 16. These proportions are an estimate of the effect of the fiscal 1990 minimum composite scores, discussed above, translated into minimums in terms of high quality. First, the model is run without placing any constraints on the proportion of high-quality recruits within the occupational categories. This is equivalent to having no explicit enlistment standard except that implied by the cost-minimizing solution. Next, accessions are constrained to the actual fiscal 1990 level of 84,400 for the Army. Both the performance goals and the accession goals are held constant in the analysis. The average expected performance per staff-year in fiscal 1990 was 56 percent. First-term staff-years, however, are unconstrained. In the unconstrained case, there are no minimum high-quality percentages set by occupational group. The alternative case imposes the fiscal 1990 enlistment standards on each occupational category. The case is precisely the same as the unconstrained case except that constraints on the minimum proportion of high-quality recruits that enter each of nine occupation groups are set to equal the percentages in the "Predicted" column from Table 16 for the Army. These minimums are the implied representation of minimum composite scores in the model. A comparison of the two cases is shown in Table 17. The results indicate that the imposition of this approximation to enlistment standards raises the proportion of high-quality recruits accessed by about 10 percentage points, from 50.9 percent in the unconstrained optimum to 60.7 percent in the enlistment standards case. The unconstrained case also suggests that higher-quality is warranted in combat arms (infantry, gun crews, and seamanship), electronic equipment repairers, and communication and intelligence, compared with the allocation in the enlistment standards case, with lower portions of quality recruits allocated to the other occupational groups. One might presume that if it were optimal to have a high-quality proportion that exceeds the minimum enlistment standard, the allocation would be made. This is not necessarily the case, as the results indicate. The required quality allocation due to the enlistment standards in some occupations raises the marginal cost of high-quality recruits to all other occupations. Hence, while the unconstrained case suggests that 74 percent high-quality recruits is optimal for combat arms, when minimums are imposed in other occupations, the optimal, given the allocation to other

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop TABLE 17 Effect of Enlistment Standards on Recruit Quality and Cost: Army Example Occupations Unconstrained Case Fiscal 1990 Enlistment Standards Percentage High-Quality Recruits     Infantry, gun crew, seamanship 74.0 50.6 Electronic equipment repair 100 77.9 Communications, intelligence 100 64.0 Health care 0 65.6 Other technical 60.4 66.5 Support and administration 22.6 63.6 Mechanical equipment repair 0 67.5 Craftsmen 0 58.2 Service and supply handlers 20.1 58.2 All 50.9 60.7 Costs     Recruiting $ 482.2M $ 578.3M Training 1,532.9M 1,534.7M Total 6,573.0M 6,645.8M Occupations, falls to 50.6 percent, which is just sufficient to satisfy the minimum. Reducing Enlistment Standards What are the effects of reducing the AFQT-equivalent, minimum enlistment standard by, say, 10 AFQT percentile points? Using the statistical relationship between AFQT-equivalent enlistment standards and the proportion of high-quality recruits in an occupation, we can solve for the new, lower enlistment standards implied by the reduction. These lower standards are entered into the model to establish a third case. The results of running this case are shown in Table 18, which includes the previous two cases for comparison. With the reduction in enlistment standards there is a concomitant reduction in costs and the proportion of high-quality recruits accessed. Recruiting costs fall by about $72 million, compared with the case in which the original standards are imposed, and the proportion of high-quality recruits falls by about 7 percentage points. Interestingly, the proportion of high-quality recruits in the occupations in which the unconstrained case suggests high quality is most efficient—electronic repair and communications and intelligence—rises. This occurs because the marginal cost of high-quality recruits to those occupations is reduced as high-quality recruits

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop TABLE 18 Reduction in Enlistment Standards Occupation Unconstrained Case Fiscal 1990 Enlistment Standards 10 AFQT-Equivalent Point Reduction in Fiscal 1990 Enlistment Standards Percentage High-Quality Recruits       Infantry, gun crew, seamanship 74.0% 50.6% 38.8% Electronic equipment repair 100% 77.9% 100.0% Communications, intelligence 100% 64.0% 71.7% Health care 0% 65.6% 53.8% Other technical 60.4% 66.5% 58.6% Support and administration 22.6% 63.6% 51.8% Mechanical equipment repair 0% 67.5% 55.7% Craftsmen 0% 58.2% 46.4% Service and supply handlers 20.1% 58.2% 48.9% All 50.9% 60.7% 53.1% Costs       Recruiting $482.2M $578.3M $501.9M Training $1,532.9M $1,534.7M $1,533.5M Total $6,573.0M $6,645.8M $6,591.1M are released from occupations in which enlistment standards are inefficiently high, according to the model. An implication of these results is that inefficiently high enlistment standards in one or more occupations raises the cost of high-quality recruits to other occupations, potentially reducing the proportion of high-quality recruits in those occupations below what would have been optimal in the unconstrained case. SUMMARY We have examined some potential applications of the Accession Quality Cost/Performance Trade-off Model from several different perspectives: As a vehicle for explaining to Congress the rationale for an accession program and its relation to personnel readiness over the first term of service; As a tool for evaluating a program; As a model for developing an accession program; and As an aid in setting enlistment standards. Its actual contribution to the defense community in any of these areas de-

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop pends on the willingness of analysts and researchers to work with the model. The purpose of this paper is to suggest, by examples, that the effort may be worth the cost. We believe that the model, as it currently exists, provides an excellent logical framework for analyzing recruiting and first-term performance issues, perhaps providing new insight. Most important, the model may be helpful in articulating the reasons for programmatic choices regarding recruiting goals, resources, and first-term performance. The model clearly can be improved, particularly by refining some of the underlying empirical relationships. Continued use of the model, however, is a necessary condition for improvement. Only through its use will its strengths and weaknesses be shown and its value be demonstrated. REFERENCES Human Resources Research Organization, Systems Research and Applications Corporation, and Lewin/VHI, Inc. 1993 Accession Quality Cost-Performance Tradeoff Model (CPTM) Guidebook. Prepared for the Office of Accession Policy, Assistant Secretary of Defense, Force Management and Personnel. Lockman, Robert F. 1978 A Model for Estimating Premature Losses. In R.V.L. Cooper, ed., Defense Manpower Policy: Presentations from the 1976 Rand Conference on Defense Manpower. R-2396-ARPA. Santa Monica, Calif.: The Rand Corporation. McCloy, R.A. Harris, D.A., Barnes, J., Hogan, P.F., Smith, D.A. Clifton, D., and Sola, M. 1992 Accession Quality, Job Performance, and Cost: A Cost/Performance Trade-off Model" Report No. FR-PRD-92-11. Alexandria, Va.: Human Resources Research Organization.

OCR for page 101
Modeling Cost and Performance for Military Enlistment: Report of a Workshop This page in the original is blank.