Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 46
Staffing Standards for Aviation Safety Inspectors 3 Approaches to Staffing Having established a conceptual frame of reference for staffing models, we are in a position to begin the examination and ultimate assessment of the actual approaches of the Federal Aviation Administration (FAA) as well as selected examples from organizations with somewhat comparable staffing situations. Representing as it does the essence of our charge, yet encompassing three fairly distinct perspectives, our in-depth analysis is divided into three chapters. This chapter is primarily descriptive—a review and analysis of systematic approaches to staffing, past and present, inside and outside the FAA. Chapter 4 focuses on the specific factors (many of them unique) that contribute to the staffing need structure for aviation safety inspectors (ASIs), along with their modeling implications. Chapter 5 brings the consideration of realities and requirements together in articulating our conclusions and recommendations to the FAA. History of FAA Modeling Efforts During the past several years, the Flight Standards (AFS) and Aircraft Certification (AIR) offices of the FAA have engaged in several efforts to improve the approaches they use to determine levels of staffing for aviation safety inspectors. Although referred to internally as “staffing standards,” they all fall within the committee’s definition of staffing models discussed in Chapter 2 and constitute efforts to represent the human resources “need structure” as conceptualized there. All are tools for use in manpower planning, determining how many staff of various types or
OCR for page 47
Staffing Standards for Aviation Safety Inspectors categories are needed at AFS or AIR facilities. They are not used in individual hiring, assignment, promotion, or other such personnel decisions. Concurrent changes in the nation’s aviation environment have presented the FAA with a number of significant challenges in this effort. These include: variations in workload drivers and work environments across the regions and the individual facilities in which aviation safety inspectors work, changes over time in the geographic distribution of workload, along with the costs and morale issues associated with relocating personnel and offices to adapt to such changes, changes in the skills required of inspectors driven by changes in the aviation industry and the FAA’s approach to maintaining aviation safety, and resource constraints as the FAA has absorbed reductions in funding, either absolute or in proportion to expanding aviation operations. Another challenge is the cost and the personnel time and effort required to implement any new method of developing staffing standards. The worth of any method or model is dependent on the quality of the data populating it, and the timely collection and validation of suitable data can be difficult and costly. Foremost among the data required in a staffing model are those describing the work that the staff—in this case, the ASI workforce—is expected to accomplish. In particular, analyses of some kind must be performed to document what inspectors do on the job and how long it takes them to do it, and record-keeping systems must be designed and implemented to capture these data. As jobs and tasks change, the data yielded by earlier analyses become obsolete, so analyses must be performed repeatedly to provide accurate information—a very costly process often requiring outside contractors. Even with well-designed systems, the procedures, forms, and data entry effort needed to continuously document task performance and resource use can be onerous. Such work often is perceived as a nonproductive use of inspector and manager time in a resource-constrained environment. During the past few years, the systems used by the FAA for labor-hour reporting (important sources of workload inputs for any staffing model) have undergone some noteworthy changes. The new systems have been devised primarily to improve resource tracking, and their utility for providing the essential input to staffing models is questionable. A system called Program Tracking and Reporting Subsystem (PTRS) has long been used in AFS to record the work activities performed. A new system called
OCR for page 48
Staffing Standards for Aviation Safety Inspectors BOX 3-1 What is LDR? LDR is a new financial management tool to help give us a more accurate picture of FAA’s major costs. Each employee and manager will identify the time he or she spends on various projects and activities. Time will be reported in one-hour blocks. Why are we doing this? FAA is required to do this, in large part because historically it has been difficult to quantify our costs. LDR will support Cost and Performance Management (C/ PM) and the Cost Accounting System (CAS), which are also just around the corner. C/PM will allow us to better allocate and manage resources. This program will ultimately change how we view and understand our contribution to the FAA vision and mission. Also, it will help us make better business decisions. The data we input into LDR will feed into the CAS, which will generate reports reflecting the total cost per project or activity. Together, LDR and CAS data should give us better information for decisions when we project budgets, estimate staffing levels, assess employee skill mix, and recruit. Management will be able to better defend the need for added resources and will be able to cite hard supporting evidence. SOURCE: Available: http://www.faa.gov/ahr/super/LDR.cfm. Labor Distribution Reporting (LDR) was introduced FAA-wide as a labor-hour reporting (i.e., timesheet) system beginning in fiscal year (FY) 2002. LDR has recently been enhanced; the number of codes used to identify work performed has been expanded to allow more fine-grained tracking of work times. Employees are required to record their work in both the PTRS and the LDR systems. Box 3-1 presents a brief description of LDR from the FAA’s web site. LDR aggregates work information differently from PTRS, recording hours as the primary entry, rather than activities. The first version provided less detail than PTRS as well, causing difficulties for staffing models that are based on documentation of labor hours by task. Although LDR has not been proposed as a source of model data, we note these features as a caution, in case other recording systems are discontinued or changed in ways that make them unsuitable data sources and LDR becomes the sole documentation of task performance times. Other difficulties that arise in the use of any labor reporting system data as input to represent workload in staffing models result from the fact that only the
OCR for page 49
Staffing Standards for Aviation Safety Inspectors work actually performed is reported; no record is produced of work that should have been performed but was not done because of resource constraints or other limitations or work that was done but simply not reported. Changes in the way tasks are classified, aggregated, and reported can make it difficult to track changes in workloads for particular tasks or groups of tasks over time. There is no direct translation between the PTRS and LDR reporting systems. Staffing Models in AFS Until 1995, AFS used a system of complexity points as a surrogate for workload to develop staffing standards. Complexity points were originally developed by the U.S. Office of Personnel Management for use in job evaluation and setting levels and pay grades of various jobs. The FAA publication Position Classification Guide for Aviation Safety Inspector Positions (Air Carrier and General Aviation)—FG-1825 (Federal Aviation Administration, 1998) provides an example of the complexity points system and its use in its Appendix 1, Complexity Report. By combining complexity points for tasks with the requirements for, and records of the frequency of, task performance and other inputs, AFS calculated what they deemed at the time to represent a rough approximation of actual workloads and staffing needs. In 1995, AFS determined that this methodology was not as effective as desired, especially as jobs and workloads were changing, and the AFS management team began considering alternative staffing models. Since the late 1990s, the FAA has experienced declining staffing resources in proportion to increasing demand, and AFS has been obliged to develop staffing methodologies aimed at allocating available resources equitably and effectively (allocation models), rather than determining staffing requirements needed to sustain system performance at what is deemed a priori an acceptable level (sufficiency models). In 1998, development of a new staffing system, called the Holistic Staffing Model, was initiated in response to the recommendations of the 90-Day Safety Review (Federal Aviation Administration, 1996), following the 1996 crash of a ValuJet aircraft. The approach underlying this initiative was patterned after a system then in use by some AFS regional offices, and model development was undertaken with contractor assistance. Documentation provided to the committee (IBES, 2000) indicates that inputs to the development of the holistic model were to include automated systems information (e.g., labor reporting system data), task inventories developed at headquarters and in the field, structured interviews, and self-reporting surveys completed by inspectors. As the model design
OCR for page 50
Staffing Standards for Aviation Safety Inspectors evolved, it became clear that AFS could not implement this model cost-effectively, and the effort was discontinued in 2002. More recently, AFS has developed a model called the Automated Staffing Allocation Model (ASAM). ASAM, developed by an FAA work-group under the auspices of the AFS Human Capital Council, incorporates some formulas used by the southern and southwestern regions in their local staffing projections. ASAM is implemented in a Microsoft Excel application. ASAM does not appear to have involved a new task analysis effort, as the holistic model would have, and it relies heavily on the complexity-point computations that are believed to be ineffective in an environment of changing jobs and workloads (Federal Aviation Administration, 2003b). AFS is now implementing ASAM as a forecasting and planning tool for staffing needs, although it is not being used as a standard for authorizing staffing levels. The actual staffing levels are set in regional offices, with guidance from the ASAM model and from other sources. The ASAM model is still undergoing refinement as its early results are evaluated. Both the holistic model and ASAM are described in detail later in this chapter. Staffing Models in AIR In AIR, a different staffing approach is used for manufacturing inspectors.1 A staffing model for AIR was developed in the mid-1990s and first implemented in 1997 through updates to Order 1380.49, Staffing Standards for Aviation Safety Inspectors (see Order 1380.49D, 2002, for the current version) (Federal Aviation Administration, 1995, 2002). These standards have not been used officially since some time in 2004, according to information we obtained from AIR and Professional Airways Systems Specialists (PASS). Under Order 1380.49 (Federal Aviation Administration, 1995, 2002), AIR employed a work recording system called the Manufacturing Inspection Management Information System (MIMIS), consisting of a list of 78 activities, products (reports, certificates, etc.), and services, to record inspectors’ work. The system also used time standards (in hours) that had 1 This information on AIR staffing standards is derived from a briefing to the committee by Deane Hausler of AIR 530 on March 8, 2005, personal communication with James Pratt, an ASI and a PASS representative at the Cleveland, OH, Manufacturing Inspection District Office, and documentation provided to the committee by the FAA or accessed on the FAA web site. The committee understands that the staffing system in AIR is still undergoing review and that changes are likely.
OCR for page 51
Staffing Standards for Aviation Safety Inspectors been developed for tasks, products, etc., based on job task analysis and observation and on actual hours recorded in labor reporting systems. In addition to inputs from work recording systems, AIR uses information gathered from division and field office managers and from customers (manufacturers) to forecast staffing needs. The staffing projections generated at headquarters using all of these inputs serve as guidance to the divisions and offices as they make their staffing decisions given available resources, but they are not used to mandate staffing levels. Unlike MIMIS, the recently implemented LDF system originally recorded hours worked using only 25 major activities, said to account for about 80 percent of the activities recorded under MIMIS. LDR does not record products and services completed. The Certificate Management Information System (CMIS) records inspections performed, while MIMIS is still used to record work products at this time. AIR management had planned to discontinue the use of MIMIS and staffing standards reporting in FY 2004, but questions concerning the adequacy of the LDR system for AIR purposes prompted delay of that decision. As of 2005, staffing standards reporting was being retained, although recent enhancements to the LDR system may affect this decision.2 Of course the FAA’s human resource management functions include selection, assignment, training, and development of inspectors with the particular skills needed for their individual jobs. ASI position announcements note the specific assignment for which hiring is anticipated, often with special knowledge, skills, abilities, and experience requirements specified. But these human resource management functions are not addressed by FAA staffing standards. Analysis of the ASAM The purpose of the ASAM is to improve the allocation of ASI and other AFS staffing resources across regions and across offices within regions in AFS. It does this by providing an estimate of total staffing requirements for flight standards district offices, certificate management offices in the region, and international field offices (Federal Aviation Administration, 2005c). Staffing demand is estimated by using algorithms relating demand factors, such as number of certificates, registered aircraft, commercial airports, etc., either to staffing directly or to staffing 2 As an example of the ambiguity about labor reporting in AIR, it is interesting to note that Order 2700.37 establishing LDR was published in 2001. Order 1380.49D, issued in 2002, is now shown on the FAA regulatory library web site as having been cancelled by Order 2700.37. A new MIMIS guide was issued in 2004.
OCR for page 52
Staffing Standards for Aviation Safety Inspectors hours required, which are then converted into full-time equivalents. The original relationships between demand factors and staffing are based on expert judgment and experience, rather than data obtained through some empirical process. It should be noted that although the model captures some of the factors that contribute to inspector workload and overall staffing needs, a number of important drivers are either underrepresented or neglected completely. For example, designee staff are not directly included in the model as workload drivers for inspectors. Recent changes and evolution in workload demands and how they are staffed are not fully incorporated into the model. The model’s data and algorithms lag behind in fully incorporating the Air Transportation Oversight System (ATOS), for example, as well as some new relationships in general aviation staffing and maintenance/repair inspection. The emphasis on work sampling, for example, is probably a pre-ATOS carryover, at least in part. Incomplete though it may be, the model’s estimate of total staffing demand is compared with total ASI staffing resources in order to guide distribution decisions. The ratio of staffing resources to estimated staffing demand is applied to the model’s estimate of demand at the district and office level to yield an index of the staffing resources that the district or office should have. In practice, however, this index does not represent a requirement or directive for allocations at the district or office level; rather, it is regarded primarily as a basis for management discussions and possible negotiation (Federal Aviation Administration, 2004b). From everything the committee was able to determine regarding its current use, therefore, ASAM is basically an allocation rather than a sufficiency model. That is, it is not used to determine the appropriate overall level of staffing, but rather to ensure that available resources are allocated reasonably, which means in proportion to estimated workload. Deviations between actual and estimated staffing do not necessarily result in corrections, but they may form a basis for subsequent adjustments at the discretion of management. Model Parameters and Logic The ASAM model builds staffing demand from the bottom up, relying heavily on two types of relationships. The first consists of a large number of identities representing “core” staffing principles that were established by definition, by custom, or by simple rules of thumb (e.g., one manager per field office). These equations are location specific, so they generate staffing demands at specific locations. The second type of relationship is somewhat more analytical and empirical, at least in theory. Staffing demand is estimated as a function of
OCR for page 53
Staffing Standards for Aviation Safety Inspectors workload measures. Algorithms (some of them from the Position Classification Guide complexity report) (Federal Aviation Administration, 1998) relate variables that generate the demand for staffing, such as the number of pilots requiring certification or the number of airports, to hours required. Table 3-1 shows a page of an ASAM worksheet with documentation from a sample file provided to the committee by AFS. In some cases the equations go directly from the workload driver to full-time equivalent positions (FTEs). Demand for staff hours of service of particular types is estimated from the underlying factors generating demand. The algorithms are typically (though not all) linear, with coefficients or parameters relating demand measures to service hour requirements. The sources or rationales for some of the algorithms are documented in the literature the committee was provided; for other algorithms, these are not documented. FTEs are estimated from hours by dividing total hours demanded by an estimate of service hours that can be supplied annually by a full-time ASI in the respective category. Supervisor requirements are determined by identities that relate number of FTE inspectors of a given category to numbers of people needed to supervise them, based on predetermined supervisor to staff ratios. Overall staffing demand at a given location is computed by summing all of the sources of demand. Some positions—officer managers and supervisors—are based on identities, while others are derived empirically from underlying factors presumed to generate demand. It is important to note that although the values used in this computation may be empirically determined, the relationships among variables (the parameter or coefficient estimates) typically are not; rather, they are based on expert judgment and past experience (often the most reasonable source). Order 8400.10 is a source for some factors, such as supervisor to staff ratios and hours available per FTE (Federal Aviation Administration, 1994). Data Sources Three data sources are used in the ASAM model. The first is a staffing questionnaire, versions of which are administered at the district and field office levels (Federal Aviation Administration, 2004c). This instrument requests data on actual (assigned) staffing, authorized staffing, and information on workload as measured, typically, by the number and size class of carriers supported. It constitutes the primary data source for the ASAM index of actual or current staffing. A second data source is derived from direct demand measures— “aviation services data” provided by both the region and headquarters. These data have important direct effects on the staffing requirement estimates produced by the model. Workload-related data for the region or
OCR for page 54
Staffing Standards for Aviation Safety Inspectors TABLE 3-1 Sample Page from ASAM Worksheet Documentation Section Item Question Field Name Table 2: Aviation Services Index Aircraft Based in FSDO Geographic Area Conversion to Hours [CodeE_Hours] Table 2: Aviation Services Index Accidents / Incidents / Eir / Complaints (Ops)B (Code F) [CodeF] Table 2: Aviation Services Index Accidents / Incidents / Eir / Complaints (Ops)B (Code F) [CodeF_Hours] Table 2: Aviation Services Index Accidents / Incidents / Eir / Complaints (Aw)C+D (Code G) [CodeG] Table 2: Aviation Services Index Accidents / Incidents / Eir / Complaints (Aw)C+D Conversion to Hours [CodeG_Hours] Table 2: Aviation Services Index Accidents / Incidents / Eir / Complaints (Av)C (Code H) [CodeH] Table 2: Aviation Services Index Accidents / Incidents / Eir / Complaints (Av)C Conversion to Hours [CodeH_Hours]
OCR for page 55
Staffing Standards for Aviation Safety Inspectors Actual Number Authorized or Derived Value Notes Derived from [CodeE] Data Source =IF([CodeE]<100,[CodeE]*0.5, IF([CodeE]<200,[CodeE]*0.4, IF([CodeE]<400,[CodeE]*0.35, IF([CodeE]<500,[CodeE]*0.33, IF([CodeE]<1000,[CodeE]*0.28, IF([CodeE]<2000,[CodeE]*0.16, IF([CodeE]<3000, [CodeE]*0.16, IF([CodeE]<4000, [CodeE]*0.088, IF([CodeE]<5000, [CodeE]*0.077, IF([CodeE]<10000,[CodeE]*0.037, ”Error”)))))))))) This is meant to generate the number of hours inspectors in the region must spend each year inspecting the data element. The hours value is driven by the 8400.10 handbook chart on page 6-5. It is meant to ensure that there is a 95% confidence that the surveyed group is similar to the entire population. Same as [Code_B] (Certificated Airmen (Pilots)) =[Code_B] The value being used is the Certificated Airmen (Pilots) value ([Code_B]). Derived from [CodeF] Data Source =([Code F]*.02)*11 This is direct from ASAM spreadsheet. The source of formula is unknown. Same as [Code_B] (Certificated Airmen (Pilots)) =[Code_B] The value being used is the Certificated Airmen (Pilots) value ([Code_B]). Derived from [CodeG] Data Source =([Code G]*.02)*10 This is direct from ASAM spreadsheet. The source of formula is unknown. From Data Source =[CodeB]*0.1 The value being used is the Certificated Airmen (Pilots) value ([Code_B]) times 0.1. Derived from [CodeH] Data Source [CodeH_Hours]= ([Code H]*.02)*10 This is direct from ASAM spreadsheet. The source of formula is unknown.
OCR for page 56
Staffing Standards for Aviation Safety Inspectors facility, such as the number of public use airports in a region, the number of air carriers of each type, and many other indicators are included. Finally, the third data source is “national” data, collected at the nationwide level and provided by FAA headquarters. Additional key sources of information supporting the model, though not of current data inputs, are the FAA Inspector Handbooks (Orders 8300.10, 8400.10, 8700.1, etc., Federal Aviation Administration, 1994, 2003a, 2004a, 2005c, 2005d, undated) and the Work Program Guidelines (Federal Aviation Administration, 2005a), which provide specifications for many of the tasks to be performed by inspectors. As background to understanding ASI staffing, it is also important to know that ASI work is divided by the FAA into three categories or levels of priority: (1) required work, (2) planned items, and (3) demand items. Required work is a top-priority workload for each fiscal year that is required by policy. It is largely independent of external environmental conditions. Planned items are specific items generating workload that are determined for the year at the regional or field office level. Demand items are performed on an as-needed basis. They are typically generated by incidents, accidents, and the demands of the public and require rapid response. Basis for the Relationship Between Workload and Staffing There is little direct documentation of the basis of the relationship between workload and staffing in ASAM. The relationships appear to be determined by administrative judgment or policy. Some relationships are taken from the Complexity Report (Federal Aviation Administration, 1998) and have not been updated in recent years.3 Simple logic suggests that workload should be one (if not the sole) major determinant of quantitative staffing requirements; hence the precision with which this relationship can be specified is a vital consideration in the evaluation of any staffing model. Clearly, when it is available, empirical evidence constitutes the most credible basis for estimating the relationship. For ASAM, it appears that a combination of historical experience, administrative judgments, and policy or regulations rather than current empirical evidence underlie the estimates. References to certain instructions, handbooks, and other sources provide some documentation, 3 The model clearly uses complexity points in estimating demand. However, the Position Classification Guide: Aviation Safety Inspector Positions (Federal Aviation Administration, 1998) states that “the complexity report is used to determine the complexity of an individual inspector’s assignment, not to make conclusions regarding the overall staffing needs of an office” (part VII, Position Management).
OCR for page 57
Staffing Standards for Aviation Safety Inspectors but clearly the basis for most of them is the judgment of experts. Expert judgment can, of course, yield valuable information, but in the absence of empirical verification, the validity of such estimates is indeterminate. In addition to its questionable precision, the ASAM model provides no estimates of predicted performance; that is, it does not indicate the consequences of being understaffed (or overstaffed) relative to the model’s prescriptions in terms of workload cycle times, or backlog, or safety. Analysis of the Holistic Model4 The Holistic Staffing Model was proposed in the mid-1990s as an improved way to estimate staffing demand for the AFS. It was never made operational, but the design of the surveillance submodel was well developed. We are reviewing it here in some detail because we think that important lessons for future modeling efforts can be learned from this modeling effort. The basic concept underlying the holistic model is that staffing demand is best established from the bottom up, starting at the level of activity (i.e., functional elements of work determined through systematic job or task analysis). The more variable and complex the to-be-described job domain is, along with the context in which it operates, the more challenging and costly the effort required to adequately represent it. The holistic model was undertaken in full recognition of this challenge, including the fact that it would be necessary to identify and measure a number of key factors controlling the demand for specific activities in order to provide credible estimates of staffing requirements. Such factors are referred to throughout this report as workload drivers. The demand for aviation safety inspectors is estimated in the holistic model by estimating the activities of inspectors by 13 office types5 and the three categories of tasks—required, planned, and demand—discussed earlier. Basic factors affecting the demand for work, such as the number of certificates, public use airports, and aircraft, generate the demand for various activities. Then ASI inspector requirements are estimated by dividing total activities by the average number of the identified activities that can be performed by an inspector. To this requirement for inspectors, administrative and support staffing is added. A fixed component of administrative and support staffing, one that is approximately constant over time, is distinguished from a 4 This discussion is based largely on a contractor report, “Flight Standards Service Holistic Staffing Model: Final Report,” prepared June 11, 1999, by IBES (2000). 5 The office types include small, medium, and large variants of commercial and general aviation, mixed, ATOS and others.
OCR for page 58
Staffing Standards for Aviation Safety Inspectors variable component, one that may vary with changes in workload. The first component is entered as “current staffing” under the assumption that the current administrative and support staffing levels are correct for the current workload. The variable component is estimated as ratios to current operational staffing. As that staffing changes, support and administration staffing based on average ratios between particular administrative and support staff and inspectors changes also. Hence, the variable component of support and administration staff changes indirectly in response to workload changes, if the number of inspectors changes. The model estimates and equations are based on a sampling approach in which typical office types are constructed. Based on the parameters estimated for the typical offices, the relationship between demand and staffing is then applied to offices across the regions. The holistic model, in concept, allows the estimation of AFS staffing demand, serving as a sufficiency model. As demand factors change, the model would be responsive and estimates of staffing requirements would change. In principle, it could help estimate changes in staffing demand as aggregate workload changes and as relative workload shifts among regions and field offices. Since it was never actually implemented, of course, whatever potential it may have was never realized. We therefore present the following description of model characteristics as they would have been had the holistic model become a reality. Model Parameters and Logic The logic of the holistic model is relatively straightforward. Demand factors generate workload activities across inspector staff. This activity workload and its relationship to demand factors are documented through administrative and survey data. An average time to perform an activity is calculated. Staffing demands for inspectors are then estimated across field offices and regions based on anticipated activity workload. A sampling approach is taken with the data. Parameters for model or typical field office types are calculated and then applied to all similar office types. Administrative and support staff are estimated based on staffing to support the historical workload. Ratios of support to inspector staff are used to estimate needed changes in support/administrative staff when inspector staff changes. Key input data include activities by workload type (required, planned, and demand) by type of office. In addition, time required to accomplish these activities is critical. From these, the key set of model parameters— average times to complete the activities—is generated. The key model parameters are then used to project inspector staffing demand as a function of demand factors. Administrative and support
OCR for page 59
Staffing Standards for Aviation Safety Inspectors staffing are added, based on historically determined ratios of support to inspector staff. Key outputs of the model, then, are estimates of inspector staffing demand as a function of workload-generating demand factors across regions and offices. Actual staffing can be compared with staffing demand to project shortages or surpluses. Furthermore, the model enables “what-if” analyses to determine the effect of changes in the total demand or mix of demand factors on staffing shortages or surpluses. The model does not directly predict the consequences of shortages (or surpluses) in terms of the effect of outputs, although it is capable of estimating the expected number of required, planned, and demand activities that may not be accomplished for a given set of staffing shortages. Model Data Sources The holistic model relies on a combination of administrative databases and survey data collected from the regions and field offices. The administrative databases include the PTRS, which tracks and documents much of the workload, including the required and planned items, performed by the regions and field offices. It includes data regarding the activities performed, the type of staff performing the activities, and the time necessary to perform each activity. The Consolidated Personnel Management Information System (CPMIS) (no longer available since the beginning of FY 2006) was used to document on-board staff at all levels, including administrative and support staff. The National Vital Information Subsystem (NVIS), a headquarters-level database, documents the environment, including the number, kind, and distribution of offices. Field Office Self-Reporting Surveys are administered at sampled field offices. The purpose is to document the workload activities and time required for these activities, based on the responses of the field offices. The surveys both complement the PTRS data, because some activities are not recorded in it, and serve as a second source for the PTRS data. The surveys provide additional information on a key input of the model— frequencies of various activities and time required to perform various activities. Note that the survey is administered to only a sample of AFS offices. The sampling unit is the field office, and data at the activity level are collected from the field office. A survey was designed to be administered to staff offices and division managers at the headquarters level. The purpose of this survey is to obtain information on the major tasks performed, the job series performing each task, and documentation regarding programs, databases, handbooks, and other information. This information is generally to provide background, to identify surveillance tasks performed, and to
OCR for page 60
Staffing Standards for Aviation Safety Inspectors provide information to form a sampling frame for the Field Office Self-Reporting Survey, rather than to provide data that are directly used in model calculations. The proposed data collection for the holistic model has two desirable features. First, much of the critical information is routinely collected through PTRS, an administrative database that is maintained for operational purposes, not simply for the model. In principle, the PTRS data could be improved over time and, if necessary, expanded to support particular aspects of the modeling, at relatively low additional cost. The quality of the data can be tested and improved systematically, improving not only the estimates of the model but also the original application of PTRS. Second, the survey collection of self-reported data is based on sampling, targeted to information most needed for the model. This lowers the overall cost of data collection. Moreover, targeting of particular types of information has the potential for improving data quality. The holistic model’s use of data also has some undesirable qualities. First, the level of detail required poses a difficult challenge to the FAA’s data recording and analytic capacities, and it was one of the contributors to the rejection of the model. Second, we were told that the model does not account for variations in complexity among some work drivers;6 it treats them as equivalent, thus failing to accurately represent real workload. Basis for the Relationship Between Workload and Staffing Workload is defined in terms of activities. The time to perform the activities is provided through PTRS and through the field office survey. Based on estimates of average time to perform an activity, staffing demand is estimated by dividing the total demand for activities by the average yearly hours available for work per inspector. As factors generating demand increase (for example, certificates), staffing demand increases through the relationship between activities generated by demand factors and estimates of time to accomplish an activity. The simple averages used to generate parameters to provide the link between workload and staffing are less accurate than estimates derived from regression and maximum likelihood methods that take into account other factors that moderate the relationship between staffing and activities. 6 An example given by an FAA staff member was the average age of the aircraft fleets at different carriers. Older fleets require more intensive oversight than newer ones, but the model does not recognize this difference.
OCR for page 61
Staffing Standards for Aviation Safety Inspectors Summary Evaluation of the ASAM and Holistic Models The ASAM and holistic models each have systemic deficiencies that would not be overcome by patching new software into old. It is likely that development of a new model, if done well, would produce a much better product than such a remodeling effort. We do want to make clear that these models both have features that could be profitably incorporated into a new model. The development efforts for the two models have made AFS staff aware of many useful techniques and data sources and have exposed potential pitfalls in developing a staffing model. This experience and knowledge should be used in the process of developing any new model. Analysis of Potential Alternatives Adapted from Other Organizations In addition to the ASI-specific holistic and ASAM models that the FAA presented for review, the committee examined staffing models from other organizations for their potential relevance to our task. Because of the proprietary nature of staffing for any business, we had a very limited opportunity to review and document staffing models from private industry. However, we were able to review a number of public-sector manpower planning models, tools, and processes that resemble the ASI staffing situation in at least some respects. These include: airport security staffing (Atkins, Begen, Kluczny, Parkinson, and Puterman, 2003); distributed service networks (Palekar, Delli, and Rajagopalan, 2000); Air Force manpower assessments (U.S. General Accounting Office, 2002); Army manpower analysis (U.S. Army FORSCOM, 2005); Army manpower modeling (Hawley, Lockett, and Allender, 2005); Navy manpower modeling (Bowen and Wetteland, 2003); U.S. Environmental Protection Agency (U.S. General Accounting Office, 2000); and state courts (Fautsko, Hall, and Ryan, 2001). These organizations employ systematic approaches to satisfy their manpower and staffing needs. Their staffing situations are similar to that faced by the FAA, in that fairly substantial pools of diverse human resources are required on a sustained basis, but with continuously changing characteristics and levels. However, each organization has evolved a solution that is unique to its situation, and none seems to have generalized to the others. Simply put, we were unable to find anything approaching a
OCR for page 62
Staffing Standards for Aviation Safety Inspectors generic manpower/staffing model. What we concluded from our review, then, was that the unique features of any organization’s staffing requirements dominate the generic ones so that it is virtually impossible to successfully adapt a systematic approach from one to another. Since staffing models per se apparently do not generalize, our focus shifted from a search for proven alternatives to a consideration of generic model characteristics with reference to the ASI staffing situation. This refocused review, coupled with the committee’s experience, revealed only two important, broadly shared characteristics. First, all models must address the issue of factors driving the demand for manpower. Whether it involves court caseload, component failure rates, or number of calls per minute, staffing models must be able to predict or otherwise explicitly reflect the demand for human services as a function of the factors that drive that demand. Second, most models address the consequences of manpower supply-demand imbalance. In other words, they address not only how many staff members of each type are needed, but also the implications of having less staffing than is recommended by the model’s output. Viewed from a characteristics as well as an overall perspective, therefore, it is clear that unique considerations dominate, but it is equally clear that specification of both the factors driving demand (demand drivers) and the consequences of staffing deficiencies is essential to any viable staffing model. Rather than continuing the search for promising models external to the FAA or advising the FAA to do so, we decided that effort would be better spent considering the factors that are very specific to the ASI situation (e.g., aircraft types, travel time between locations, qualifications of different inspector types, designee oversight, ATOS transition, performance criteria, etc.) and the model characteristics that appear most salient to this unique set of requirements. In other words, the committee deemed adaptation of general modeling principles to the specific characteristics of the ASI staffing situation the most promising approach to both evaluating current practices and seeking improvements. The committee also reviewed the staffing standards used by the FAA’s air traffic organization for air traffic control specialists (ATCSs). A National Research Council committee reviewed the ATCS standards in 1997 and made recommendations for improvement (National Research Council, 1997). Some of those recommendations may have been implemented, but a review of recent FAA documents reveals that air traffic staffing still uses multilevel “engineered” standards, with a strong emphasis on detailed task analysis and on quantifying ATCS workload and activities (Mills, Pfleiderer, and Manning, 2002; Federal Aviation Administration, 2004d). The data from these analyses are used in combination with targets
OCR for page 63
Staffing Standards for Aviation Safety Inspectors for maximizing staffing efficiency and reducing staffing costs over time to determine staffing standards. The 2004 document describes a reassessment of the air traffic staffing standards to be performed starting in FY 2005 but, as they stand now, the ATCS staffing methods appear not to meet several of the model criteria described in Chapter 2. Most importantly, both the composition of the ATCS jobs and the context in which they are carried out are considerably more homogeneous than the wide array of ASI jobs and work settings. In sum, the committee believes that the current ATCS staffing standards are not a useful source of improvements to the AFS staffing methodology. Summary This chapter presents an in-depth analysis of staffing models developed inside and outside the FAA organization from the perspective of current and projected ASI staffing needs. In particular, it addresses the core question posed in the committee’s charge: Are current approaches to ASI staffing sufficient to cope with the growing demand and, if not, could upgrading or adapting other models in current use satisfy the need? To answer this question, the committee first reviewed past and present approaches developed by the FAA to guide staffing decisions, focusing particular attention on the two most comprehensive such efforts: the ASAM model currently in use by the AFS organization and the holistic model, which AFS conceived but never implemented, due primarily to cost considerations. We found both seriously deficient in a number of respects, most importantly, in their inability to predict the consequences of understaffing at either the local or the system level. The holistic approach did incorporate a number of essential features, but since it never materialized and—like ASAM—would have proven difficult to validate had it done so, we conclude that neither model represents a particularly promising point of departure for system improvement. However, analysis of the approach in use by the considerably different, substantially smaller AIR organization led us to conclude that it is sufficient to support the current AIR staffing requirements, subject only to improved recording systems. Next we explored staffing approaches in use by a sample of large organizations whose situations appear comparable in certain respects to that facing the FAA. But even a cursory analysis revealed that none would be adaptable, in whole or in part, to the ASI staffing situation. The reason, very simply, is that the unique features of each clearly dominate the shared ones to the extent that there is little to be gained in attempting to transport elements from one to another. We therefore conclude that the ASI demand structure is sufficiently unique to rule out either substituting
OCR for page 64
Staffing Standards for Aviation Safety Inspectors or adapting a model in use outside the FAA. Even the FAA’s ATCS staffing model was deemed unsuitable for ASI purposes. On the basis of these analyses, we therefore conclude that neither modifying current FAA models nor adapting those from the outside represents the most cost-effective strategy for the much-needed upgrading of the ASI staffing process. Although there is much to be gained from the ASAM and holistic efforts, we think that the present and anticipated future ASI staffing situation calls for development of an entirely new model. In the next chapter, the focus therefore shifts to identification of the specific facets of the ASI situation that, based on our multi-source investigation, we think must be considered in developing an effective staffing model.
Representative terms from entire chapter: