As described in Air Force Instruction 36-2110, Total Force Assignments, “the primary goal of the Air Force Officer Assignment System is to assign the right officer to the right position at the right time to meet [Air Force] mission requirements” (USAF, 2018, p. 465). Currently the Air Force assignment process considers, to varying degrees, inputs such as job requirements, qualifications, career development, and duty location. However, room for improvement remains. This Appendix provides additional details to supplement approaches mentioned elsewhere in this report to assist the Air Force to improve the assignment process and expand recent initiatives in officer assignments into enlisted assignments. Specifically, this Appendix describes the potential utility of an increased use of preference-informed matching in job assignment to improve person-job fit and positively impact retention of high-quality Airmen, especially those with critical skills. As such, it offers alternative ways of thinking about and designing assignment algorithms that the Air Force may find useful to improve or replace those currently in use.
Many of the current Air Force assignment procedures have grown out of the historical low-tech assignment tool consisting of a whiteboard covered with colored sticky notes, a long-standing system later augmented by spreadsheets. Often assignment teams worked with very little information about job requirements and candidate preferences. Although candidates with particular preferences or special needs could sometimes have these recognized by having their current commander (i.e., the “losing commander” who would be losing them, but who knew them) advocate for them to the assignment team, there were few ways of communicating preferences in a general and easily-used way.
In recent years the Air Force and other Services have moved toward somewhat more market-oriented assignment procedures, such as the Talent Marketplace developed for use by the officer assignment system,1 that make it easier for candidates and also for hiring authorities to share information and express preferences. This approach is shaped by the idea that sometimes the mutual needs and preferences of the candidate and the hiring authority could be better expressed and met. But the equivalent of whiteboards and sticky-notes still plays a role, as the information needed for hiring authorities and candidates to gain information with which to form and express preferences is still limited.
This Appendix briefly outlines ways in which preferences of candidates and/or hiring authorities can be added to other requirements of a job assignment system (e.g., those dictated by law, policy, mission need, development plans), so as to better satisfy the needs and desires of both hiring authorities and service members. In each case, the Air Force would first establish job requirements and candidate qualifications to determine which candidates are qualified for which jobs. Preferences could then be solicited from candidates for the jobs for which they are qualified, and/or from hiring authorities for the qualified candidates. These preferences would reflect information that is not included in requirements and qualifications but that nevertheless influences the satisfaction of the hiring authorities and candidates with their assignments.2 The job assignment system would facilitate the sharing of information to allow submitted preferences to reflect actual preferences as closely as possible: for example, candidates might be allowed to submit personal statements and other relevant material (e.g., particular tasks accomplished), and assignments might be described in as much relevant detail as is available.3
1 Information about the Talent Marketplace was provided to the committee during the site visit to Joint Base San Antonio-Randolph (November 5–8, 2019) and a follow-up conference call (November 18, 2019).
2 That is, candidates’ preferences can add information both about desires for particular kinds of assignments, and about constraints or concerns related to, for example, children in school, spousal employment, current job satisfaction, which are not captured in administrative data. Similarly, hiring authorities’ preferences can reflect information about candidates from interviews, references, and personal statements that may not be captured in administrative data. (For Airmen in the Reserves, more information about their day jobs may fall into this category.)
3 The exchange of information on which informed preferences can be based cannot be expected to happen spontaneously. The Army has been experimenting with Talent Marketplaces for several years, and still reports low rates of resume submission, especially resumes that are informative of special knowledge and skills, and many units only rank a small number of candidates for a small number of jobs. So the information part of the marketplace will need to be designed and monitored, and part of this task will involve educating Airmen and hiring authorities in its use.
Roughly speaking, when making assignments, four kinds of algorithms are commonly considered: prioritized choice (sometimes called serial dictatorship), two kinds of stable preference matching, and constrained optimization. Each of the algorithms discussed below is illustrated with the following simple example throughout: four hiring authorities are each seeking to fill one position, and all drawing from a set of four candidates available at that time.
Note that this is a simplified example. These systems can be adapted in a straightforward way to the case of hiring authorities seeking to fill many positions. Thus this discussion can apply to both reassignments (of the kind addressed by the Talent Marketplace) and initial assignments of new enlisted personnel to positions, and of new officers to officer development categories.4
PRIORITIZED CHOICE (“SERIAL DICTATORSHIP”5)
Under this system, the Air Force establishes a priority order either among the available jobs or among the available candidates. It also establishes which candidates are qualified to fill which job.
4 A good deal is known about extending this simple model to one in which hiring authorities are seeking to fill multiple positions. It is somewhat more difficult to extend this approach in a simple way to a fully dynamic matching model in which jobs and candidates become available fairly continuously, with the ability to keep a job unfilled or a candidate waiting for some interval to produce a better matching than can be obtained immediately, although something is known about this also.
If jobs are prioritized, then the hiring authority for each job establishes a preference list for qualified candidates. Then the highest priority job is assigned to the qualified candidate it ranks highest (prefers most) among all the candidates, the second-highest priority job is assigned to the qualified candidate it most prefers among those not yet assigned, and so forth, until the lowest priority job is assigned its most preferred candidate among those not yet assigned (if any candidates remain).6
If candidates are prioritized,7 then each available candidate is asked for a rank-ordered preference list over the available jobs for which they are qualified. The highest priority candidate is assigned to his/her most preferred job, the second-highest priority candidate is assigned to the job he/she most prefers among those that remain (and for which she/he quali-
6 If some job has not indicated any preferences among candidates qualified for that job, then it is desirable to fill that job as late as possible (e.g., when only one candidate qualified for the job remains). That is, lower priority jobs that have expressed preferences are allowed to choose first so long as qualified candidates remain for the higher priority jobs that find all qualified candidates equally desirable.
7 For example, by seniority, class rank, or shortness of time since last deployment.
fies), and so forth until the lowest priority candidate is assigned his/her most preferred job among those not yet assigned (if any jobs remain).
Benefits of Prioritized Choice
The resulting distribution of assignments is “Pareto optimal,” which means that no other assignment is preferred by all members of the prioritized side. This is true whether jobs or candidates are prioritized. When jobs are prioritized (and hiring authorities choose) then no distribution of assignments exists that would fill all jobs with preferred candidates. Similarly, when candidates are prioritized and choose, no other distribution of assignments exists that all candidates would prefer.8
8 Note that to say that a distribution of assignments is Pareto optimal is only a weak recommendation for those assignments, but to say that a distribution of assignments is not Pareto optimal is a strong argument that the assignments might be done better. That is, an assignment of positions is Pareto optimal for Airmen if no other set of assignments would give every Airman a more preferred position, but it might be that some alternative (also Pareto optimal) distribution of assignments would give many Airmen more preferred positions and only a few (or even a single Airman) a less preferred position. On the other hand, if an assignment of positions to Airmen is not Pareto optimal, then this means that it would be possible to reassign positions in a way that would give each and every Airman a preferred position (for which they are qualified), which suggests that the relevant Air Force goals could be met in a way that would increase the satisfaction of all the Airmen who are subject to assignment in this period. (Slightly modified forms of Pareto optimality work similarly; for example, we often want comparisons between two distributions of assignments to depend only on those Airman who get different assignments, and so the relevant comparison is whether some Airman can
Another property of prioritized choice is that it is safe for members of the prioritized side (i.e., those who are choosing based on preferences) to reveal their preferences truthfully. For example, in a situation where jobs are prioritized, a job owner who does not get the candidate he listed as his first choice would not have done better by not listing that first choice. The same is true of job seekers in situations where candidates are prioritized. This means that there is no need for participants to be strategic about the preferences they reveal.
Drawbacks of Prioritized Choice
The resulting distribution of assignments does not consider the preferences of both candidates and hiring authorities; rather, only the prioritized side makes choices according to preferences. Neither does it balance the preferences of those with high priorities and those with low priorities. Instead, the most highly prioritized are assigned a top choice from those available, while those with low priorities may receive very low choices.
Note on Tie-breaking
Whether candidates or jobs are prioritized, when some preference lists have ties (i.e., some jobs or candidates are equally preferred) there are ways of breaking ties that make it possible to satisfy others’ preferences more fully than if ties are broken arbitrarily.
Under stable matching systems, the Air Force establishes which candidates are qualified to fill which jobs, and elicits preferences from both candidates and hiring authorities (i.e., candidates are asked to rank jobs for which they are qualified, and hiring authorities are asked to rank qualified candidates, in order of preference).
Deferred Acceptance Algorithms (Pairwise Stability)9
In contrast to some matching procedures that put a premium on what participants indicate is their first preference, stable matching procedures try to treat all parts of the preference ordering in a balanced way. This makes it safer for participants to reveal their true preferences (rather than
get more preferred assignments without giving any Airmen less preferred assignments.) See Roth and Sotomayor (1990) for more information.
concentrating on what is the best assignment they can get if they claim it is their most preferred). “Deferred acceptance” algorithms get their name because they accomplish this by deferring final assignments until preferences of all parties have been considered. As described in Chapter 5, the Air Force’s newly developed Talent Marketplace leverages a deferred acceptance algorithm as a preliminary step to improve the officer assignment process.10
Candidate-Proposing Deferred Acceptance Algorithm
The following should be read as the description of a computerized matching algorithm that takes as its inputs the preference lists (rank-order lists) submitted by Airmen over positions, and by hiring authorities over Airmen, after both hiring authorities and Airmen have had adequate time and information to formulate their preferences. The algorithm will proceed in steps. For ease of communicating these steps, they are written as if the Airmen and positions are taking actions, but their only actions are to submit their preference lists, after which all actions are carried out by computer:
Step 0.1: Airmen and positions (hiring authorities) privately submit preferences.
Step 1: Each Airman “applies” to his first choice. Each position “holds” (but does not yet finally accept) its most preferred candidate among those who have so far applied. Any remaining applicants are rejected.
. . .
Step k (generic intermediate step): Each Airman who was rejected in the previous step applies to his next choice if one remains. Each position considers the applicant it has been holding together with its new applicants and “holds” the most preferred. Any remaining applicants are rejected (including one who may have been “held” at a previous step).
Stop: The algorithm terminates after the first step at which no application is rejected. At that point each Airman is assigned to the position (if any) that is holding him when the algorithm stops. (Note again that all final acceptances are deferred until there are no more applications or rejections, hence the name “deferred acceptance algorithm”).
10 Information about the Talent Marketplace was provided to the committee during the site visit to Joint Base San Antonio-Randolph (November 5–8, 2019) and a follow-up conference call (November 18, 2019).
Position-Proposing Deferred Acceptance
This is the same as above but with the roles of applicants and positions reversed—i.e., at each step each position offers itself to its highest-ranked
applicant, applicants “hold” their most preferred position among those offered to them, rejected positions offer themselves to their next most preferred applicant, etc.
Benefits of Deferred Acceptance Algorithms
DA algorithms use the information contained in the preferences of both candidates and hiring authorities, and they produce what are called stable matchings, which do not have “blocking pairs” (i.e., there is never a service member and Air Force job that would have both, mutually, preferred each other) or “justified envy” (in which a lower priority candidate receives a job preferred by a higher priority candidate with equal qualifications). This approach also renders it safe for members of the proposing side to reveal their preferences truthfully. Deferred acceptance algorithms have been used to match new doctors to their first positions in the United States, and in other health care labor markets, and to match children to schools in a number of American cities (see Roth 2002, 2008 and the references cited there, and Roth, 2015).
Drawbacks of Deferred Acceptance Algorithms
The blocking pairs the Air Force needs to be most concerned with for retention do not involve Air Force positions and service members. They instead involve private-sector jobs and service members who might choose to separate from the Air Force to take a private-sector job instead of the
offered Air Force assignment. Because of this, it is not clear that the form of stability produced by deferred acceptance algorithms is the best goal for an Air Force assignment system. Eliminating blocking pairs involving Airmen and alternative assignments within the Air Force comes at a cost, since a stable matching (i.e., one with no such blocking pairs) may not be Pareto optimal for candidates (i.e., it may be possible to give all of some groups of candidates assignments for which they are all qualified and which they all prefer, which might better facilitate retention of service members who have no further military obligation). This is worth further study, particularly if (as is now the case) deferred acceptance algorithms are being employed to generate an initial matching that is then modified by assignment teams.
Trading in Cycles (One-Sided Group Stability)
After any procedure for making initial assignments (including hybrid procedures in which an algorithm suggests an assignment that is then manually modified), we can search for ways to improve the set of assignments by allowing “trading” by candidates in cycles such that candidate i prefers (and is qualified for) job i + 1 to his initial assignment (to job i) for a cycle of candidates i = 1, . . . n, n + 1 = 1. That is, after making an initial assignment the system searches for cycles in which each candidate could “trade up” to a job he prefers. The algorithm would continue until no further cycles of this sort exist (so the process would be a Pareto improvement for candidates—i.e., some candidates would be helped by getting assignments they preferred, and no candidates would be harmed by getting assigned to a position they liked less).
CONSTRAINED OPTIMIZATION (“Integer Programming”)
Constrained optimization (also known as “integer programming” maximizes an objective function, subject to some constraints. Constraints can include all the requirements (e.g., how many jobs need to be filled by what kinds of people), while the objective function could measure, for example, satisfaction as expressed by rank-order lists or by some other measure of Air Force productivity.
A benefit (and a drawback) of this approach is that it offers maximal flexibility. This flexibility is the result of two opportunities for choice: (1) the choice of what to maximize (in the example below, this is a weighted sum of matches, with weights that can be chosen to reflect combinations of candidate and position preference ranking), and (2) the choice of what constraints to impose (which can reflect not only job qualifications in a binary way, but also the degree of qualification and distributional constraints over candidates assigned to different positions (or multiple candidates assigned to the same position).
In general, a particular set of constraints may not be feasible (e.g., it might not be possible to fill all of the available positions with qualified Airmen from among those available at a given moment). So sometimes constraints need to be made more flexible, and this can sometimes be done by consecutively solving related optimization problems (e.g., first maximizing the number of positions that can be filled, and then maximizing candidate satisfaction subject to filling this number of positions). The fact that at least some variables are constrained to take on only integer values gives this kind of optimization the name “integer programming.” (The RAND algorithm for officer specialty selection described in Chapter 4 of this report is an integer program.)
This compact formulation offers vast flexibility, which is good if used very carefully but can also present problems with interpretation and maintenance. Consider for example the objective function: Maximize the weighted sum (over all i, j) of Σ wij(xij) where the wij are weights (numbers). This allows a different weight wij to be put on each possible person-job match xij. So, for example, the number wij could reflect something about Airman i (e.g., his skills, time since last deployment) as well as something about position j (e.g., the relative urgency of filling it with someone who meets certain requirements). If there are n Airmen and m positions to be matched
at this time, that means that there could be as many as m x n different weights, each of which could be determined by some formula having to do with attributes of both the airman and the position. While this allows maximal flexibility, it can quickly become hard to audit, particularly if how the weights are chosen is not well described or recorded. As situations change, if the weights are not transparent, the whole assignment system may become opaque. So particular attention has to be given to documenting how the weights in the objective function are calculated (which may involve a separate set of formulas, distinct from the integer program itself). That is, because of the great flexibility in formulating optimization models of this sort, clearly documenting them becomes important for maintaining their usefulness.11
SOME GENERAL OBSERVATIONS ON INCENTIVES IN ELICITING PREFERENCES
Devising a system where all parties can safely and straightforwardly report their preferences (and to have those preferences used effectively in making assignments) is not possible in general. This is all the more true when participants will list preferences only over a subset of possibilities (e.g., when there are too many possible assignments for candidates to rank them all, so that choices have to be made about which to rank, and not just about in which order to rank them). However, there are some lessons learned in matching and assignment that have quite general applicability. They tell us that big incentives to manipulate preferences in obvious ways should be avoided. For example:
- Do not employ rules that penalize participants for submitting long preference lists.12
- Do not employ rules that penalize participants for failing to be assigned their first choice (e.g., rules that make it much less likely in
12 For instance, in constrained optimization, consider an objective function that seeks assignments that give participants their highly ranked choices, on average. Suppose some participant lists only 1 choice, then it will be useful to regard all his other (unranked) alternatives as being tied for his 2nd choice, instead of regarding all his unranked alternatives as being equal to, say, his 100th choice. The reason is that the latter design choice would give him an incentive to rank only one choice, to force the assignment system to avoid assigning a 100th choice, but if all his unranked choices are regarded as his second choice, then it would not be costly for the objective function to assign him to one of them, and so if he has actual preferences, he would do better to indicate which of those assignments he would prefer.
that case to get one’s second choice than if the second choice had been listed as the first choice).13
RESEARCH, DEVELOPMENT, AND MAINTENANCE OF ASSIGNMENT SYSTEMS
Current assignment systems, as well as any new or modified systems, generate data on placements and their immediate and longer-term consequences. In addition, new procedures offer the possibility of collecting data from experiments, or non-experimental comparisons. That is, when a new procedure is introduced initially for some parts of the Air Force, data can be collected comparing the new experience with previous experience (e.g., do candidates list more assignments in their preference lists after some new information-sharing procedure) and with parts of the Air Force still using the prior procedures. In accordance with the Data Priority included in this report’s Flight Plan, these data need to be collected and organized.
To better understand and increase retention, future research is needed to understand how assignment systems influence decisions to separate from the Air Force. For example, research is needed to gather data and investigate procedures for post-assignment adjustment and repair (particularly if assignments are declined in favor of separation). Potential questions to explore include:14
- Are there particular assignments that predictably lead to separations that are unnecessary/undesired by the Air Force?15
- What if the service member resigns rather than accept appointment? Should mechanisms be introduced and employed to help with “reclaiming” high-value Airmen who have submitted their resignations (and letting go of those who are not high value)?
To summarize, the Talent Marketplace has to also become an information marketplace that allows position owners and Airmen to make appro-
13 There are “immediate acceptance” algorithms that, if a participant does not receive his first choice, put him aside until everyone else who might receive their first choices has been satisfied, during which time his other ranked choices may all be taken. In such a system, it is not safe for participants to truthfully reveal their first choice: instead, they have to calculate how likely they are to get each of their choices if they claim it is their first choice.
14 Some of which may already be focused on by the new Office of Labor and Economic Analysis (OLEA), U.S. Air Force Academy, directed by Col. Justin Joffrion, USAF. Information received during call between Col. Justin Joffrion, USAF, AF-A1, and Alvin Roth, Committee Member, March 13, 2020
15 For example, the impact of assignment to regions with poor public schools on Airmen with school-age children.
priate information available to each other in order to develop informed, accurate preferences. The key point is that matching, and the overall functioning of the human capital system, may be improved by developing new methods of sharing preferences as well as new algorithms for taking preferences into account.16
Ferguson, M.D., R. Hill, & B. Lunday. (2020). A scenario-based parametric analysis of the army personnel-to assignment matching problem. Journal of Defense Analytics and Logistics, 4(1), 89–106.
Gale, D., & L.S. Shapley. (1962). College admissions and the stability of marriage. The American Mathematical Monthly, 69(1), 9–15.
Roth, A.E. (2002). The economist as engineer: Game theory, experimental economics and computation as tools of design economics. Econometrica, 70(4), 1341–1378.
Roth, A.E. (2008). What have we learned from market design? Economic Journal, 118(527), 285–310.
Roth, A.E. (2015). Who gets what—and why: The new economics of matchmaking and market design. Boston, MA: Houghton Mifflin Harcourt.
Roth, A.E., & M. Sotomayor. (1990). Two-sided matching: A study in game-theoretic modeling and analysis. Cambridge: Cambridge University Press.
Sonmez, T., & T.B. Switzer. (2013). Matching with (branch-of-choice) contracts at the United States Military Academy. Econometrica, 81(2), 451–488.
Switzer, T.B. (2011). A tale of two mechanisms: US Army cadet branching. (Unpublished master’s thesis.) Pontificia Universidad Católica de Chile, Santiago, Chile. Available: http://economia.uc.cl/wp-content/uploads/2015/07/tesis_tswitzer.pdf.
USAF (U.S. Air Force). (2018). Total Force Assignments. Air Force Instruction 36-2110. Available: https://static.e-publishing.af.mil/production/1/af_a1/publication/afi36-2110/afi36-2110.pdf.
16 Much of the academic literature on matching assumes that institutions that allow participants to form accurate preferences already exist. One of the tasks facing the Air Force is to develop such institutions in parallel with the development of the Talent Marketplace.
This page intentionally left blank.