Executive Summary

The Federal Aviation Administration (FAA) is responsible for ensuring safety in all U.S. civil aviation. Two offices within the Aviation Safety (AVS) organization enforce and maintain aviation safety regulations and promote safety in aviation. The Flight Standards Service (AFS) is charged with overseeing aviation operations, maintenance, training, and other programs, and the Aircraft Certification Service (AIR) is charged with ensuring the safety of aircraft design and manufacture. In 2005 there were over 3,600 aviation safety inspectors (ASIs): about 3,450 in AFS and 175 in AIR.

The number of these inspectors employed by the FAA has changed little over the past several years, although aviation industries have been expanding and changing rapidly. Also, the FAA has made more use of what it calls designees—nongovernment employees certified to act on behalf of the FAA—to assume some of the responsibilities formerly assigned to aviation safety inspectors. There is concern in several communities that the staffing levels of safety inspectors may not be adequate to fulfill the FAA’s responsibilities. The Aviation Subcommittee of the Committee on Transportation and Infrastructure of the U.S. House of Representatives responded to these concerns by mandating the current study in the Vision 100—Century of Aviation Reauthorization Act.

The Committee on FAA Aviation Safety Inspector Staffing Standards was consequently established at the National Research Council to examine the models and methods used to determine inspector staffing needs for these two FAA units. The objective of the study is to determine the



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Staffing Standards for Aviation Safety Inspectors Executive Summary The Federal Aviation Administration (FAA) is responsible for ensuring safety in all U.S. civil aviation. Two offices within the Aviation Safety (AVS) organization enforce and maintain aviation safety regulations and promote safety in aviation. The Flight Standards Service (AFS) is charged with overseeing aviation operations, maintenance, training, and other programs, and the Aircraft Certification Service (AIR) is charged with ensuring the safety of aircraft design and manufacture. In 2005 there were over 3,600 aviation safety inspectors (ASIs): about 3,450 in AFS and 175 in AIR. The number of these inspectors employed by the FAA has changed little over the past several years, although aviation industries have been expanding and changing rapidly. Also, the FAA has made more use of what it calls designees—nongovernment employees certified to act on behalf of the FAA—to assume some of the responsibilities formerly assigned to aviation safety inspectors. There is concern in several communities that the staffing levels of safety inspectors may not be adequate to fulfill the FAA’s responsibilities. The Aviation Subcommittee of the Committee on Transportation and Infrastructure of the U.S. House of Representatives responded to these concerns by mandating the current study in the Vision 100—Century of Aviation Reauthorization Act. The Committee on FAA Aviation Safety Inspector Staffing Standards was consequently established at the National Research Council to examine the models and methods used to determine inspector staffing needs for these two FAA units. The objective of the study is to determine the

OCR for page 1
Staffing Standards for Aviation Safety Inspectors strengths and weaknesses of the methods and models that the FAA now uses in developing staffing standards and projections of ASI staffing needs and to advise on potential areas for improvement. The term “staffing standards” is used in this report to denote the FAA’s concept of sheer numbers of personnel required to fill specified jobs, without regard for quality or skill levels. While the term is often used to refer to levels of qualifications or skills needed by individuals for particular jobs, it is not used in that way by the FAA. The FAA uses “staffing standards” to refer to the numbers of personnel of various job categories deemed appropriate to staff its facilities.1 The distinction between individual and collective standards represents a long-standing functional demarcation used in the professional human resource and staffing community. Determining and providing the number of personnel in various categories that an organization needs to accomplish its goals (the current AVS focus) is a manpower planning and management function, whereas describing and classifying jobs and establishing the knowledge, skills, and abilities (KSA) requirements for each are considered human resource or personnel management functions. The formal task statement from the FAA’s contract with the National Research Council reads: Critically examine the current staffing standards for FAA Aviation Safety Inspectors and the assumptions underlying those standards. The committee will confine its study to ASIs only; other inspector jobs will not be considered. The committee will not consider issues of compensation, work rules, or similar labor relations matters. Gather information about the ASI job series and about the specific factors that may characterize the FAA as an organization and the ASI job series that would influence the choice of methods that might best be used to develop staffing standards; for example, it will compare engineered to performance-based staffing standards. Review the staffing models, methodologies, and tools currently available, and some of those in use at other organizations with important similarities to the FAA, and determine which might be applicable or adaptable to the FAA’s needs. Propose models, methods, and tools that would enable the FAA to more accurately estimate ASI staffing needs and allocate staffing resources at the national, regional, and facility levels, particularly in light of the occasional but urgent need to reallocate resources on short notice. Estimate the approximate cost and length of time needed to develop the appropriate models. 1 Throughout the report we have tried to use the term “staffing” to denote only manpower-related functions or issues.

OCR for page 1
Staffing Standards for Aviation Safety Inspectors Since this statement of task is directed exclusively toward manpower planning and management issues (i.e., the adequacy of staffing “standards” and models), the committee focused its efforts accordingly. However, we recognized from the outset that any attempt to assess or improve the FAA’s overall staffing operation would be impossible without first engaging a critical set of human resources and personnel management issues (e.g., the adequacy of job descriptions, qualifications, performance measures). Simply put, unless one knows precisely what the various inspectors are expected to do, what qualifications they must have to do it, how the quality of their performance is distinguished, and what the consequences of different performance levels are, it is impossible to determine how many inspectors the system needs to function properly—no matter how appropriate and sophisticated the staffing models that are used. Since the level of effort required to address these human resources and personnel management issues extends well beyond this committee’s charge and resources, yet is so critical to the ultimate goal, we deemed it our responsibility to articulate clearly the need while focusing our attention on the specific charge. The committee, therefore, proceeded to develop criteria for key features and functions of a staffing model to meet the FAA’s needs. We then evaluated several staffing model approaches, some that are used or proposed by the FAA as well as those of several other public-sector organizations, against those criteria. The results of this evaluation inform our recommendations for the FAA’s future staffing model development. In this way we have addressed the formal charge while emphasizing that these modeling recommendations are contingent on seriously engaging the human resource and personnel management issues. CONCLUSIONS AND RECOMMENDATIONS Important Factors to Be Considered in Model Development The Changing Aviation Landscape The changing U.S. and global aviation landscape has important implications for ASI staffing. Five features are expected to be especially important drivers of future staffing needs for ASIs: introduction of advanced flight deck and air-ground technologies, increasing number of variants and derivatives of aircraft and systems, continuing growth in regional carrier and general aviation operations,

OCR for page 1
Staffing Standards for Aviation Safety Inspectors outsourcing and offshoring of maintenance, and new manufacturing tools and techniques. Changing FAA Policy and Business Practices, Attrition, and Retirements Four areas of change within the FAA are particularly noteworthy with respect to their implications for future ASI staffing needs: the increasing use of designees, the shift to a system safety approach, regulatory changes, and attrition and retirements of safety inspectors. FAA Human Resource Management Practices The committee’s first goal is to “critically examine staffing standards for FAA Aviation Safety Inspectors and the assumptions underlying those standards.” We detected the following qualitative issues related to current human resources (as distinguished from manpower) management practices that were intrinsically linked to the determination of the demand for aviation safety inspectors, and that should be addressed before the development of a staffing model is attempted: new knowledge demands of system safety approaches, new skill demands to support new ways of working with industry, new and continuing technology knowledge demands, failure of job specifications to adequately capture current job requirements or to accommodate changes, and lack of job performance criteria. Evaluation of FAA Models All organizations base staffing decisions on a paradigm of the underlying production process, whether they do so explicitly or not, and whether it approaches reality or not. This conceptualization is often referred to as a staffing model. The committee’s charge is cast in those terms—we were asked to examine and evaluate the FAA’s models as well as possible alternatives for the staffing of ASIs. The Holistic Staffing Model was initiated by AFS in response to a safety review conducted following the 1996 crash of a ValuJet aircraft. As the model design evolved, it became clear that AFS could not implement the model cost-effectively, and the effort was discontinued in 2002. More

OCR for page 1
Staffing Standards for Aviation Safety Inspectors recently, the Automated Staffing Allocation Model (ASAM) was developed by AFS as a forecasting and planning tool for staffing needs. In AIR, a different staffing approach is used for the inspectors responsible for aircraft design and manufacture. AIR’s staffing standards primarily depend on a work recording system that lists activities, products, and services to record inspectors’ work. The system has also used time standards (in hours) based on job task analysis and observation and on actual hours recorded in labor reporting systems. Conclusions In reviewing the two comprehensive models of staffing demand— ASAM and the aborted Holistic Staffing Model—against the features we consider essential, the committee concludes that both fall short in certain areas. ASAM, as currently structured, appears deficient in at least the following respects: It does not predict the consequences of staffing shortfalls at any level, a primary criterion for most staffing models. It fails to account for some important factors affecting inspector workload. Many of its key parameters derive from expert judgment and have not been empirically validated. Hence it is not an empirically based model. The holistic model, while potentially closer to the structure we consider appropriate, was lacking in the following respects: The documentation suggested statistically weak univariate methods for estimating parameters. It appears to be quite detailed in its structure, almost certainly exceeding a level that available data could support. There was no discernible plan for formal model validation, nor was it clear precisely what predictions the model would make regarding the consequences of alternative levels of staffing. Most importantly, the model was never developed or tested, so its potential utility and validity are unknown. Recommendations In view of the above limitations and of good software development practices, neither the ASAM nor the holistic model offers a promising departure point for a future staffing model. We therefore recommend that any effort directed toward improving the FAA’s approach to ad-

OCR for page 1
Staffing Standards for Aviation Safety Inspectors dressing the need structure for ASIs begin afresh. Both ASAM, as it exists, and the holistic model, as it was described, provide a rich store of knowledge that would be useful in developing a new modeling approach. In particular, the existing documentation and the knowledge and experience of the FAA staff involved in the development of these models should be tapped in any new development effort. We emphasize, however, that the total cost (financial and otherwise) involved in attempting to preserve the basic structure of the ASAM model, or to replicate the holistic approach, while overcoming their limitations would far exceed that of creating a model with all the essential features unencumbered by such considerations. Evaluation of Potential Alternatives Adapted from Other Organizations In addition to the models that the FAA presented for review, the committee examined manpower and staffing models from other organizations for their potential relevance to our task. Because of the proprietary nature of staffing for any business, we had a very limited opportunity to review and document staffing models from private industry. However, we were able to review a number of public-sector manpower planning models, tools, and processes that resemble the staffing situation for safety inspectors in at least some respects. They include airport security staffing, distributed service networks, Air Force manpower assessments, Army manpower analysis and modeling, Navy manpower modeling, the U.S. Environmental Protection Agency, state courts, and the FAA’s Air Traffic Controller staffing model. Conclusions Our analysis of approaches used by these organizations that bear a resemblance to the FAA, as well as of the approach used by the FAA’s air traffic control organization, reveals that in certain key respects there is little potential for direct transfer—or adaptation—of any of these models to the staffing situation for aviation safety inspectors. In each case, unique features far outweigh the common ones, and the solutions that have evolved reflect that diversity. Recommendations The staffing challenge for aviation safety inspectors is sufficiently distinctive to rule out the option of importing, in whole or in part, an

OCR for page 1
Staffing Standards for Aviation Safety Inspectors already developed model. Therefore, improvement over current practices can be achieved only through development of a new model, drawing on both the FAA’s previous experience with modeling efforts and careful consideration of the salient model properties described in this report. Model Development in Light of Unit Differences Conclusions The staffing situations in the two FAA units responsible for aviation safety are markedly different. With over 3,400 inspectors widely distributed across functional and geographical job domains and the obvious deficiencies in its ASAM model, AFS is clearly in a position to benefit from (and justify the cost involved in) developing a new model. By contrast, it would be difficult to justify a costly modeling effort for staffing the fewer than 175 inspector positions in the AIR organization, especially since the approach currently in use appears to be generally satisfactory. The challenge facing AFS is considerably larger and more complex. To be demonstrably effective, a staffing model must incorporate accurate representations of workforce supply and demand, applying well-designed algorithms to produce accurate projections of staffing needs and the consequences of staffing shortfalls. It should also enable frequent updates and changes to the work processes. Most important of all, it must be integrally linked to appropriate measures of individual and system performance, without which its validity and utility cannot be established. Recommendations AIR. The committee recommends continued effort aimed at improving the work recording systems for the AIR inspectors rather than development of a new staffing model at this time. Should significant changes in the workload drivers appear in the future, the current staffing approach might warrant another review. AFS. We recommend that the FAA initiate a systematic effort to make a fresh analysis of the staffing need structure and develop a suitable model, using recently developed software tools and methods. We elaborate this recommendation below. The approach should draw on the FAA’s previous experience with ASAM and the Holistic Staffing Model, but it should not attempt to preserve the architecture of either of these models. The development process should include the following phases:

OCR for page 1
Staffing Standards for Aviation Safety Inspectors requirements definition, model specification, model development, and model verification and validation. We recognize that the initial development and appropriate testing of a new model will require an up-front investment and take time. However, weighing it against the long-term benefits afforded by a model capable of estimating overall staffing needs, optimal distribution, and understaffing consequences, we recommend making that investment, after careful consideration of the factors discussed below. The modeling effort should be undertaken with the goal of supporting FAA decision making in both sufficiency decisions (predicting the resources needed to sustain system performance at an acceptable level) and allocation decisions (distributing available resources equitably and effectively irrespective of their collective adequacy). Most importantly, it should be empirically based, although certain relationships may necessarily be established through expert judgment—at least initially. We recommend that the model designers explicitly consider which aspects of the model should be process-based and which based primarily on statistical relationships. For example, routine tasks that have a long history of performance could probably be modeled statistically, while new or modified tasks may require more detailed process modeling. More precise specification than this would require a far more comprehensive analysis than was possible within the scope of the present study. Appropriate measures of system and individual performance are essential for both the development and validation of any improved staffing approach, irrespective of model properties and features. We therefore recommend that a concerted effort be invested at the outset in developing meaningful performance indices. We recognize that this is not a simple matter, and that heavy reliance on expert judgment at all levels will be necessary in order to devise measures that are both meaningful and widely accepted. One of the most significant weaknesses in the current modeling practices is the burden of entering data to populate the model prior to making predictions. Hence we recommend that any future model should be designed so that it can be supported by institutionalized administrative databases, not by ad hoc surveys or other extraordinary data sources. In addition to model features per se, the FAA and those assisting

OCR for page 1
Staffing Standards for Aviation Safety Inspectors with the development of a staffing model must consider the following additional issues, mostly of a practical nature, that bear on model implementation. Cost. It was difficult for the committee to estimate the cost of designing, developing, operating, and maintaining a staffing model for aviation safety inspectors with the information available to us, and the estimate presented here necessarily is based on assumptions about the modeling environment. Our best estimate is that it will take about $600-800K to design and build the modeling tool, $300-400K to initially populate the model with data and develop mechanisms to keep the data updated, and perhaps $100K per year to keep the model and data current. There are many uncertainties that could affect the cost, including the availability of the needed data in easily usable form, the number of variables and level of detail that must be represented in the model, and the choice of method and of model developers. Time. The previous experience of committee members suggests that one to two years will be required to go through the development process that we recommend, if it is performed by an FAA-contractor team. Development of a model using only FAA in-house resources would be very likely to take longer. By the end of this period, a working model should be in place. However, the model itself will continue to evolve over time as data are accumulated on its functioning, and inevitably adjustments will need to be made. We recommend that the development of a new model be undertaken in full recognition of this evolutionary requirement. Organizational constraints and culture. Perhaps the most critical determinants of a model’s long-term value are the organizational constraints and culture in which it is introduced and maintained. If the model is perceived as merely creating work for its own care and feeding (e.g., gathering data to populate it) rather than as a valued aid to decision making, it will fail. Therefore, once a model is developed, its value must be actively promoted to those using it and affected by it. Any effort of this sort will also be accepted most readily if those who are affected by its use are involved in the process of its development. ASIs and their managers (and perhaps the Professional Airway Systems Specialists organization) should be consulted from the beginning, and they should have significant roles in the model’s design, development, and implementation. The management of the modeling effort will be critical to its success, cost, and timeliness. The managers should be skilled and experienced in managing software development projects and committed to the success of the effort.

OCR for page 1
Staffing Standards for Aviation Safety Inspectors Finally, the FAA should address the human resource issues associated with changes to the position of aviation safety inspector and to AFS business processes before developing a new staffing model. Available data and improvements. Whenever possible, the FAA should seek and take advantage of available data to populate new staffing models. Since a model’s predictions are only as good as the data that go into it, careful consideration should be given to the relative costs of modifying existing data-gathering systems versus creating new ones. Resources. The committee’s sense is that one of the failings of past staffing models is that they required a commitment of resources for development, maintenance, and use beyond what FAA or AFS management was able or willing to provide. This is not a problem unique to the FAA. Modeling endeavors often begin with great ambitions and expectations, only to be undone by the weight of the work that is required to realize their ambitious goals. We strongly recommend that during the model design phase the FAA should focus on the tradeoff between what it is willing to invest (both for development and for maintenance and use of the model) and what it is expecting to gain. Constant and explicit consideration of this trade-off is imperative during the early stages of staffing model design and development. The gains should be viewed from the perspective of the breadth of the staffing questions that the model will be able to answer as well as the validity and utility of the answers it is able to yield. In a word, we recommend that the FAA conduct a serious cost-benefit analysis for any new modeling effort that is proposed, undertaking this effort only if it is institutionally committed to the development and maintenance of such a model.