Assessment of Staffing Needs of Systems Specialists in Aviation (2013)

Summary

Within the Federal Aviation Administration (FAA), the Airway Transportation Systems Specialists (ATSS) personnel maintain and certify the equipment in the National Airspace System (NAS). According to the definitions set forth by the Office of Management and Budget in Circular A-76, the certification work of ATSS is considered to be “inherently governmental”; that is, tasks performed to maintain and certify the NAS may only be performed by federal government (in this situation, FAA) employees. The Technical Operations service unit of the FAA includes more than 9,000 employees, two-thirds of whom are ATSS personnel.¹ In fiscal year 2012, Technical Operations had a budget of $1.7 billion. Thus, Technical Operations includes approximately 19 percent of the total FAA employees and less than 12 percent of the $15.9 billion total FAA budget (DOT, 2012).

Technical Operations comprises ATSS workers at five different types of Air Traffic Control (ATC) facilities: (1) Air Route Traffic Control Centers, also known as En Route Centers, track aircraft once they travel beyond the terminal airspace and reach cruising altitude; they include Service Operations Centers that coordinate work and monitor equipment. (2) Terminal Radar Approach Control (TRACON) facilities control air traffic as aircraft ascend from and descend to airports, generally covering a radius of about 40 miles around the primary airport; a TRACON facility also includes a Service Operations Center. (3) Core Airports, also called Operational Evolution Partnership airports, are the nation’s busiest airports. (4) The General National Airspace System (GNAS) includes the facilities located outside the larger airport locations, including rural airports and equipment not based at any airport. (5) Operations Control Centers are the facilities that coordinate maintenance work and monitor equipment for a Service Area (Eastern, Central, Western) in the United States (Grant Thornton, 2011).²

__________________

¹Rich McCormick, director, Labor Analysis, FAA, presentation on Labor Analysis to the Committee on Staffing Needs of Systems Specialists in Aviation, October 19, 2012.

²Ibid.

At each facility, the ATSS personnel execute both tasks that are scheduled and predictable (e.g., performing regular preventive maintenance, conducting scheduled certification of equipment and facilities of the NAS, upgrading equipment, standing watch³) and tasks that are stochastic⁴ and unpredictable in occurrence (e.g., detecting an adverse event or outage and then repairing and returning certified equipment to use after the event). These tasks are common across the five ATSS disciplines: (1) Communications, maintaining the systems that allow air traffic controllers and pilots to be in contact throughout the flight; (2) Surveillance and Radar, maintaining the systems that allow air traffic controllers to see the specific locations of all the aircraft in the airspace they are monitoring; (3) Automation, maintaining the systems that allow air traffic controllers to track each aircraft’s current and future position, speed, and altitude; (4) Navigation, maintaining the systems that allow pilots to take off, maintain their course, approach, and land their aircraft; and (5) Environmental, maintaining the power, lighting, and heating/ air conditioning systems at the ATC facilities (FAA, 2011b). Because the NAS needs to be available and reliable all the time, each of the different equipment systems includes redundancy so an outage can be fixed without disrupting the NAS.

The 2012 FAA Modernization and Reform Act mandated the National Research Council to appoint an ad hoc committee to study the assumptions and methods the FAA uses to estimate the number of ATSS personnel needed. This committee was appointed with the statement of task shown in Box S-1.

Rather than establish a standard for staffing models (a staffing model can be used to estimate the number of ATSS employees needed to fulfill the FAA’s mission), the committee decided to identify relevant factors and considerations necessary to create a model that will yield a staffing number by a reasoned, scientifically sound approach. To accomplish its tasks, the committee received briefings and

__________________

³“Standing watch” refers to monitoring for adverse events and unscheduled outages of the equipment.

⁴Randomly determined; having a random probability distribution or pattern that may be analyzed statistically but may not be predicted.

materials from the FAA and the union representing ATSS personnel, Professional Aviation Safety Specialists (PASS). It also visited several ATC facilities, collected stakeholder input via a public webpage (most comments came from ATSS personnel), and examined several staffing models and the factors that are necessary input into the models. These models and the framework of human-systems integration (HSI) factors guided the committee’s deliberations and its assessment of various modeling approaches. In the context of this report, a model depicts how different factors such as workload level or equipment repair time interact to determine optimal staffing levels (NRC, 2006).

STAFFING FACTORS AND MODEL CRITERIA

In addition to the inherent mix of scheduled and unpredictable tasks performed by ATSS personnel, staffing needs are further complicated by the FAA’s planned implementation of the Next Generation Air Transportation System, or NextGen, which will continue to integrate new technology with current systems in the NAS. Input received from various stakeholders, along with the information received from the FAA regarding the primary factors to consider when developing a staffing model, informed the committee’s work. One of the committee’s major tasks was to identify the factors that should be included in the model; these related not only to manpower staffing but also to the human resources aspects that feed into modeling. The following nine factors were identified:

1.   The time required for new hires, as well as ATSS personnel who are certifying on new equipment, to complete formal training at the FAA training center in Oklahoma City; the time required for them to receive on-the-job training (OJT) from more experienced ATSS personnel and complete the certification activities; and the increased workload on ATSS personnel who provide OJT

2.   The distance an ATSS employee must travel to some remote facilities and the time required to make the trip

3.   The potential environmental challenges involved in maintaining equipment that is near a hazardous area or located in places that experience severe weather

4.   The time dedicated to serving in the military reserves or taking other forms of leave, including family and medical leave

5.   The ability of ATSS personnel to meet all the job demands on their time without being unduly fatigued

6.   Safety requirements that include the need to have two (or more) workers in some situations (e.g., working on high-voltage equipment)

7.   Problems with the current FAA time reporting systems, such as Labor Distribution Reporting and the Remote Monitoring and Logging System, which provide data that are deficient for estimating the time ATSS personnel spend on daily tasks

8.   Upcoming retirements from an aging workforce⁵

9.   Other requirements on ATSS personnel time, such as nontechnical training and administrative tasks

In addition to identifying the factors to consider when building an accurate staffing model, the committee reviewed critical steps in the modeling process, including all of the following steps:

__________________

⁵The committee notes that attrition/accession models are best handled as separate human resources algorithms based upon the staffing model targets.

1.   Following a comprehensive study and design process that incorporates the major workload factors at an appropriate level of detail, links workload to the time required to complete the tasks, and completes the six steps of the logical design process (i.e., feasibility, familiarization, measurement design, measurement, analysis and model selection, and implementation)

2.   Incorporating key model considerations, including choosing the right type of model to use all relevant input and provide all required outputs

3.   Attending to quality factors, such as ensuring that the model is transparent, scalable, easy to use, relevant, and valid (NRC, 2006:33)

The committee used the above criteria to examine existing ATSS staffing models including the Windows Staffing Standards Analysis System (WSSAS) and the Tech Ops District Model. Although these models meet some of the criteria, overall the committee found them lacking in some areas and recommends development of a new model using the steps and factors outlined in the report. The committee also reviewed recent efforts by FAA to examine its past modeling efforts and to assess its needs for future modeling through a contract with the Grant Thornton consulting firm. The Grant Thornton reports build on the WSSAS, which is a deterministic⁶ model, and suggest improvements to the key data sources. Finally, the committee reviewed the steps necessary to successfully implement a new model, including estimates of the time needed for each activity (e.g., development, testing, preparation, implementation, validation, and monitoring results), as well as the role of FAA staff, funding requirements, and consideration of other resources within the model.

REPORT FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS

The following findings, conclusions, and recommendations are shown in order of appearance in the report chapters and do not necessarily reflect their order of importance.

ATSS Staffing and NextGen

The Committee understands that the FAA’s planned transition to NextGen is under way, but that the maintenance requirements are not yet publicly specified. An effective staffing model goes beyond documenting today’s required ATSS staffing levels; instead, a robust and accessible staffing model should be capable of incorporating such additional inputs as they become available, not only with respect to NextGen but also with respect to inevitable continuing technological advances.

Finding 2-1: Changes to the NAS such as NextGen and Minimum Operational Network (MON) implementation and unspecified decommissioning policies all make the amount of work to be performed by ATSS personnel ambiguous and complicate the FAA’s task in developing an appropriate staffing model for ATSS.

Conclusion 2-1: Developing and using a successful staffing model to predict future outcomes will be limited by unknowns such as decommissioning policies for legacy equipment, installation of NextGen equipment, and consolidation of facilities.

__________________

⁶A deterministic model is a mathematical function of multiple inputs such as the type of equipment and the nature of the task to be performed. That is, a single value of a model outcome variable can be derived from a set of values for all the input variables.

Recommendation 2-1: The FAA should ensure that ATSS staffing models will incorporate new information about the unknown factors that affect ATSS staffing such as NextGen as it becomes available, consider their staffing implications, and use appropriate modeling techniques to plan for contingencies.

Critical Factors Affecting ATSS Staffing

The job duties and work environment for ATSS personnel, represented by PASS, present special challenges and issues that should be addressed by an effective staffing model. By reviewing documents related to the job of the ATSS personnel and considering the perspectives of a wide array of stakeholders, including the systems specialists themselves, the committee identified several critical factors affecting demand for systems specialists that result not only from the unusual requirements of this particular job series but also from the demands placed on ATSS personnel by the external aviation environment. Although the committee focused much of its effort on the manpower domain, it noted several human resource areas to address.

Finding 2-2: A number of human resource issues, such as hiring procedures, training requirements, retirements, and military obligations, affect the number of qualified ATSS personnel who are available at any point in time to maintain the NAS.

Conclusion 2-2: The committee concludes that human resource issues such as retirements and succession planning considerations should be addressed in conjunction with any comprehensive manpower staffing model.

Recommendation 2-2: In accordance with the principles of human-systems integration, the FAA should build a robust staffing model that takes into account all of the following aspects of the ATSS job series, in addition to the time that ATSS personnel spend on preventive and corrective maintenance tasks:

•  Training issues, time to schedule training, the time required to attend training, and the time of experienced ATSS personnel necessary to provide OJT

•  Travel time to and from work sites

•  Environmental challenges

•  Time dedicated to military reserve service or family and medical leave

•  Fatigue mitigation plans

•  Safety factors

•  Labor Distribution Reporting deficiencies and other data deficiencies

•  Aging workforce and succession planning considerations

•  FAA’s Next Generation (NextGen) system

•  Nontechnical task demands

The Staffing Modeling Process for ATSS

The committee had several discussions on what would be a valid and useful model and how it could estimate the number of ATSS personnel necessary for maintaining the NAS in a safe, efficient, and effective manner. These discussions did not focus on a total number of ATSS personnel, but there was an understanding that, should the model generate a number greater than 6,100, the anticipated FAA budget would need to be quickly increased to avoid negative implications for the safety and efficiency of the NAS. In that case the FAA would have to employ risk mitigation to compensate for the less than

sufficient staffing while the requisite budget was approved and additional staff was hired, trained, and assigned to the appropriate sites.

Conclusion 3-1: Dedicated budget requirements for the ATSS personnel are likely to result from application of any comprehensive manpower staffing model and will need to be addressed.

Leaders of the model design and development process should intentionally look backward and forward during the phases of a logical design process so that the end result is logical, valid, and compliant with the stated purpose. In particular, data examination during the measurement and analysis phases may necessitate revised data collection procedures or additional research into some of the factors that affect ATSS staffing levels. Moreover, those who develop the model need to plan for future improvements to the model as modeling methodology and data collection procedures evolve and as the FAA obtains greater understanding of the causes of variability in the tasks performed by ATSS personnel. Insights gained at any phase can redirect the study in unexpected ways; thus, the study team needs to be not only flexible but also seasoned in handling unexpected situations and understanding which approaches are most likely to be effective in a given situation.

Recommendation 3-1: The FAA should execute a modeling process that allows for future improvements in data modeling techniques and applicability.

Assessment of the FAA’s Previous Modeling Efforts for ATSS

To provide a sound basis for recommendations for future ATSS staffing models, the committee compared prior FAA staffing models and the Grant Thornton proposed modeling approach with each other and against the set of criteria for valid staffing models compiled by the committee. The deterministic WSSAS model was based on valid structure and suitable types of data and data sources. Essentially, the WSSAS incorporated the pieces of equipment at each site, the number of technician hours required for each corrective or preventive maintenance action on each piece of equipment, and the probability of failure for each type of equipment. These measures were combined to establish the maintenance workload for each piece of equipment, and workloads were summed across all equipment at a work site. Allowances were added for technician unavailability, and a factor was added for other required activities. Time data came from time studies and consensus judgments by subject matter experts, supplemented by contractor-provided times for new equipment. Neither the reliability of input data to the model nor the validity of the output staffing levels was measured. The reports generated were useful to supervisors and administrators, and the WSSAS model was reasonably transparent to all users. However, updating the model was expensive, and the lack of resources for updating eventually made the model unsustainable.

Conclusion 4-1: The approach represented by the original WSSAS model included many of the important variables (e.g., equipment counts and task durations together with failure rates and allowances) to determine the staffing required at each site.

Finding 4-1: The WSSAS model does not appear to contain stochastic elements in places where these may have been appropriate.

Finding 4-2: The WSSAS model made no prediction of outcomes such as the impact of staffing levels on NAS availability and safety.

The Tech Ops District Model is an allocation model only (that is, it aimed only at distributing available personnel resources effectively irrespective of their collective adequacy to maintain a safe and effective NAS), rather than a sufficiency model (i.e., a model designed to predict the resources needed to sustain system performances at an acceptable level). The committee therefore found it to not be an appropriate basis for moving forward toward a valid model for staffing. It is a regression model that only shows, at best, how a number of variables are related in a statistical manner to then-current staffing levels by district. The Tech Ops District Model was never validated against outcome measures. Any work regression model, without measurement of actual hours required to perform the task, will necessarily preserve the status quo in terms of staffing, so that the performance consequences will remain the same and may not meet current or future performance requirements. The committee recommended against using the Tech Ops District Model as a source of either the modeling framework or data for future work and did not consider it further as a basis for FAA staffing models.

Finding 4-3: The Tech Ops District Model was an allocation model and not a sufficiency staffing model.

Conclusion 4-2: The Tech Ops District Model is not an adequate framework for future work.

The Grant Thornton approach builds on the WSSAS model but suggests improvements to the key data sources for the future model. It appears to capture the relationships between equipment and maintenance staffing, and it has provisions for idiosyncratic factors (variances) affecting staffing. Although the Grant Thornton design represents distinct improvements over WSSAS, the committee has two concerns with an approach built on the basis of the WSSAS model. First, corrective maintenance in the ATSS job often occurs in an unscheduled, stochastic manner, and the WSSAS model does not account for the intrinsically stochastic nature of these events. Even if some elements like mean time between component failures can be predicted, some randomness or unpredictability remains, related to the timing and location of a specific failure requiring corrective maintenance. Before dismissing various probabilistic aspects of maintenance work and methods, these stochastic elements and their relationship to adequate and safe staffing levels need to be understood and thoroughly explored to determine if and how they should be included in the model design.

Second, like the WSSAS and Tech Ops District models, the proposed model does not predict the consequences or results of staffing at alternative levels. It would be advisable to gather data necessary to study stochastic properties of outages, required time to repair, and shift profile dynamics. It would also be helpful to explore potential linkages of key internal Technical Operations performance metrics to various levels of staffing allocations.

Finding 4-4: The Grant Thornton prospective model builds on the earlier WSSAS model, and thus inherits some of its strengths and limitations. Its strengths include the same elemental data structure, supplemented with more recent data partly derived from existing data bases.

Conclusion 4-3: Based on the latest description of the proposed model in the Grant Thornton report, the limitation of the Grant Thornton approach is the plan for a deterministic model that does not consider the implications of stochastic elements. Further, it makes no predictions of outcome measures such as NAS equipment availability and safety.

Future Modeling for ATSS Staffing

The committee applied the modeling approaches and criteria developed in this report from staffing model knowledge to the WSSAS model, the Tech Ops District Model, and the proposed Grant Thornton approach to modeling. First, the factual basis of the three models was tabulated and assessed in terms of the criteria. Next, other sources of successful modeling in similar situations were assessed for the insights they might add to the future models of ATSS staffing developed by the FAA.

The committee can summarize the attributes and the evaluation of models based on these attributes as a set of statements about what are good criteria for the FAA’s future modeling efforts. The best science currently available supports the following recommendations for a valid and usable model.

Recommendation 4-1: The FAA should develop a new ATSS staffing model based on the modeling framework and criteria developed in this report. The model should be developed using a model structure that is based on equipment inventory, failure rates, and time to perform each task and should include any valid allowances and accommodations. The model structure should include both deterministic and stochastic estimates for variables such as task duration, as appropriate. The developed model structure should be based on the different specialties of ATSS technicians, rather than providing just an overall staffing level at each facility.

Recommendation 4-2: The FAA should develop a model that captures stochastic elements, unless it can be demonstrated that stochastic aspects of the maintenance process have no material effect on the staffing. For example, some tasks may exhibit multiple deterministic durations of identifiable elements, rather than strictly stochastic durations.

Recommendation 4-3: The FAA should incorporate data for the model that are appropriate to the duration and frequency of the tasks modeled and to its data collection capabilities. Specifically, the FAA needs a process to systematically validate through direct observation both historical estimates of task durations and estimates by subject matter experts.

Recommendation 4-4: The FAA should ensure that the ongoing data collection and input essential for model use do not place an unacceptable burden on data providers.

Recommendation 4-5: The FAA should ensure that output reports from the system predict consequences such as overall NAS availability time, deferred preventive maintenance activities, and overtime required.

Recommendation 4-6: The FAA should ensure that output reports are tailored closely to the needs of FAA’s internal users at multiple organizational levels, in order to increase transparency of, and user trust in, the model.

Implementing a Future Model for ATSS Staffing

The committee discusses six steps, each of which comprises a number of activities that should be undertaken to increase the likelihood of a successful model launch. Each activity requires time and resources. Ideally, the FAA and its staffing model experts would add to these major steps as needed and elaborate on each, specifying what needs to be done by whom and creating a schedule to be followed for each step.

Step 1 includes all the activities associated with the actual development of the model. Step 2 involves a pretest of the model prior to full implementation, comparing it to five criteria by which a staffing model should be evaluated: transparency, scalability, usability, relevance, and validity. As review and improvement of the staffing model is a continuous process, these five criteria should be kept in mind once the model is implemented and is being validated and amended.

Step 3 focuses on the activities necessary to prepare the FAA to transition from the current state of staffing to a new process for establishing staffing levels via a carefully constructed staffing model. The FAA will need to develop a detailed plan to implement the new staffing model and create the timeline for the transition, checking to see what other events might affect the implementation. Step 4 encompasses the actual implementation and rollout of the model according to the plans laid out in Step 3, all of which should logically be synchronized with key FAA human resources, training, and budget processes to the extent possible. Step 5 is the post-implementation evaluation of the quality of the staffing model against the five standards of transparency, scalability, usability, relevance, and validity. All of the quality standards are important, but validity is perhaps the most critical. Unless the results of the staffing plan allow the FAA to maintain the NAS, nothing else is relevant.

Recommendation 5-1: The FAA should prepare a timeline that details all the activities associated with model development and implementation that must be completed and should ensure that the resources necessary to accomplish each are available.

Step 6 includes the activities that are necessary to sustain the model over a period of time: monitor the staffing model, evaluate its adequacy, and make adjustments as needed. The staffing model that is produced as a result of this study will be more effective if it is periodically reviewed and updated to reflect the changing environment of ATSS work. If a review indicates that the staffing model is outdated, modifications should be made and the cycle of implementation should begin anew. An effective and useful modeling process takes into account the lessons that have accrued from each round of the implementation process and incorporates them into the next iteration of the staffing model, resulting in a spiral development process.

Recommendation 5-2: Once implemented, the FAA should continue to monitor the effectiveness of the staffing model by collecting data from multiple sources and should make adjustments as needed to enhance the accuracy of the model. In addition, the model should be adapted as changes to the major components of the model are made, such as changes in the tasks performed, equipment used, and training processes.

This page intentionally left blank.