Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 128
128 Safety Management Systems for Airports
agreement between the airport and the stakeholder to permit a safety assessment of the stake-
holder unit at the airport. Some questions to consider might include the following:
· Are there provisions in leases/operational agreements to ensure the management of safety issues?
· Do stakeholders follow airport rules, regulations, and procedures?
· Do they keep safety records? Are they complete, accurate, and used to track progress? How
well are they performing?
· Do they participate in airport safety meetings? Do they have their own safety meetings?
· Do their service providers have specific safety training?
· Do they report accidents and incidents?
· Are their employees aware of how to report safety issues?
· Do they have appropriate resources to operate?
· Is their equipment well maintained?
Interfaces
In this context, an interface is the boundary between two (or more) of the functional areas
where they interact. Some of these interfaces are readily apparent in some of the previous ques-
tions, for example:
· Can your staff use your assets while wearing PPE? This is an example of the interface between
occupational safety and health, assets, and people.
· Do the procedures that they use apply to all weather conditions?
· Is their training program aligned with the department needs?
· Do their current schedules provide for supervision during all shift work?
6.7 Measuring SMS Performance--Trend Analysis
Importance of Trend Analysis
The causal factors of accidents and incidents, and the effectiveness of corrective actions at an
airport can be fully understood and evaluated only if their frequency and distribution are exam-
ined in terms of type of event, location, and conditions. Trend analysis is one leg of this analytic
triangle. It is used for hazard surveillance and monitoring, forecasting, program evaluation, pol-
icy analysis, and the investigation of potential causal relationships between risk factors and out-
comes. Trend analysis is used for the following objectives:
· To identify the overall pattern of change in a safety performance indicator over time (increase
or decrease, rate of change)
· To compare one time period with another (effectiveness of operations before versus after a
risk control action or the implementation of new regulations)
· To compare different airport areas or seasonal differences (level of safety for Apron A and
Apron B; accident rates in summer versus winter)
· To compare two or more groups (trained versus untrained, different service providers)
· To make future projections (monitor progress toward a safety objective; provide an estimate
of the rate of future occurrence)
What Is Trend Analysis?
Trend analysis looks for changes in safety levels over a given time period (e.g., the last 12 months).
The safety level is usually measured using key performance indicators selected by the airport.
OCR for page 129
SMS Operation 129
If the airport wants to measure the level of safety at the ramp, a simple safety
performance indicator in this case may be the number of accidents that have
occurred on the ramp in any given month. Checking if the number of accidents is
increasing or decreasing over the months is trend analysis.
The objective of trend analysis is to
· Determine the "safety health" state of the airport, Analyze trends to determine if things are
· Identify trends in safety levels, and getting better, getting worse, or staying
· Identify needs and determine actions required the same.
to maintain and/or improve safety.
Management could have set a goal to reduce ramp
incidents by 20%. This statement is appropriate at the organizational level, but before ramp oper-
ations can act on it in a meaningful way, it needs to identify at least two things: (1) the baseline (i.e.,
how many incidents are occurring now) and (2) which activities are having an effect on apron
incidents (passenger bridge, service equipment-aircraft, vehicle-vehicle, etc.). This knowledge
comes from trend analysis.
If you do not have the necessary data, set up a method (as part of your reporting system) that
will allow you to collect data relevant to those activities. Once you start collecting the data, you
will know how many incidents are occurring. You can then set a baseline against which you can
compare future performance.
You should be careful when using the raw number of accidents or incidents to verify trends.
The number of undesirable events is related to the number of operations and a drop in the num-
ber of incidents may be a consequence of a decrease in activity level. You may prefer to use acci-
dent rates instead of the number of accidents. An accident rate is the number of accidents divided
by a fixed number of operations over a period (e.g., 10,000 operations in 1 year). A decrease in
the rate of accidents is a clear and reliable picture of safety improvements at your airport.
Analysis of the data collected should allow you to identify which activities have the most
impact in the incident count. This information should help you focus on the major problems
and develop action plans to reduce them. At the same time, it will allow you to track whether
these plans are effective.
Assume that the data collected show that, after 6 months, bridge-aircraft incidents add up to
eight events. After analysis and investigation, you decide that implementation of a stricter train-
ing program and spot checks should improve airline employees' skills and compliance with the
rules, thus reducing the number of bridge-aircraft incidents. Six months later, when you do your
next review, you should be able to see if the number of incidents has gone up or down. If the
number of incidents has not decreased, additional actions will be required (e.g., intensifying
supervision and enforcement of SOPs) to mitigate the likelihood of more incidents.
Measurements must focus on factors that are related to overall safety objectives. Most impor-
tant, measurements must be related to the most significant risk contributors. Although per-
formance measurements include incident statistics, they are primarily intended to be proactive
(i.e., identify problems before an accident occurs).
OCR for page 130
130 Safety Management Systems for Airports
Some representative performance indicators can be as follows:
· Number of airside vehicle operation infractions per month
· Number of FOD reports
· Number of bird strikes
· Number of runway incursions
· Percentage of employees with basic SMS training
Collecting information on the number of violations should be done carefully to
avoid bias. If you do not have enforcement officers on the ramp, the numbers
may actually go down, even when more incidents occur.
For larger airports, performance indicators may be related to specific areas of the
airport, for example:
· Number of airside vehicle operations infractions per month on Apron C
· Number of runway incursions at a specific "hot spot"
Data Considerations
For SMS, most trend analysis data are related to the key safety performance indicators selected
for monitoring. The airport needs to make sure the data quality objectives are met. Trend analy-
sis requires strict monitoring protocols. If the airport wants to determine the trends for apron
accidents, it should make sure reliable data are gathered every month over a significant period
(e.g., 3 to 5 years).
It is essential to identify the audience and the type of analysis or presentation appropriate for
that audience. For example, if it is necessary to report the trends to raise safety awareness at the
ramp, the audience will be every person working on the ramp. In this case, the message should
be very simple.
Determining the cause of a trend is more difficult than determining the trend, but it is the crit-
ical element for defining effective corrective actions. To identify causal factors for certain trends,
it is necessary to consider all the exogenous parameters that can influence those trends (volume
of traffic, environmental conditions, time of day, etc.).
The following are some important issues to consider when developing a trend analysis:
· Consistent data quality--Trend analysis assumes that the same or equivalent methods and
protocols are used for all the monitoring.
· Time frame and number of samples--5 years of monthly data for accident and incident trend
analysis; for step trends, at least 2 years of monthly data before and after a major change (e.g.,
new SOP, change in personnel, application of a corrective action, change in organization).
· Seasonality--Parameters that vary naturally in different seasons of the year may require spe-
cial statistics (e.g., certain types of birds can be more frequent in specific months of the year;
apron accidents are more prone to occur under low visibility conditions).
OCR for page 131
SMS Operation 131
· Statistical significance of a trend--It is common to observe upward or downward trends that
simply occur by chance (see an example in Figure 22). This is particularly frequent for per-
formance measurements over small periods or associated with low incidence events (e.g.,
number of runway incursions). The best way to check if the trend is real or is simply random
variation is to set longer periods for evaluation to collect more data and use statistical tech-
niques to check if the differences are statistically significant.
Gathering Data
After the airport safety objectives and goals are set, it is necessary to define the safety perfor-
mance indicators that will be used to measure performance and check if the objectives are being
met. Trend analysis is based on quantitative information collected for these performance indi-
cators (e.g., number of incidents at the ramp, number of bird strikes). The airport operator who
records each of these accidents will have a number for each month over a period of a few years.
Other sources of data available for trend analysis are conclusions from accident/incident
reports. The root causes identified in the investigations can provide valuable information on spe-
cific safety issues that should be addressed by the airport.
Data collection is a critical activity for trend analysis. It is recommended that a written proce-
dure be established to define who will collect the data, the means for collection, specific proce-
dures, and who will receive the information.
Table 25 depicts an example of information on safety performance indicators over a 6-year
period, and Table 26 shows an example of data gathered on the number of bird strikes in each
month. Table 27 contains the number of job-related injuries at the ramp and the number of
operations at the airport during 9 months.
Very small operators may not generate enough data to allow for the monitoring
of any trends. In these cases, a possible alternative is to look at the industry as a
whole to help identify possible trends. Reviews of accident reports, articles in
trade magazines, discussions with industry groups, participation in industry
associations, and other operators can provide useful data.
Table 25. Number of ramp accidents (example).
Year Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Total
2003 3 0 5 4 1 2 7 1 2 0 3 1 29
2004 1 4 2 2 1 5 6 4 0 2 1 4 32
2005 2 0 1 1 0 2 4 1 2 0 2 1 16
2006 0 1 2 1 0 0 3 1 0 2 0 1 11
2007 1 0 0 1 0 1 2 0 0 1 1 1 8
2008 0 0 1 0 1 0 2 0 0 0 1 0 5
Table 26. Number of bird strikes (example).
Year Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Total
2003 0 1 2 0 3 0 2 2 1 3 1 2 17
2004 1 0 3 0 0 1 1 2 0 2 1 1 12
2005 0 0 1 0 2 0 0 0 1 0 1 0 5
2006 1 0 0 2 1 1 0 1 0 1 0 0 7
2007 0 2 0 1 2 0 2 0 1 0 1 1 10
2008 1 0 2 0 1 1 0 0 3 1 2 0 11
OCR for page 132
132 Safety Management Systems for Airports
Table 27. Number of job-related injuries at the ramp.
Month Total Number of Total Number of Rate of Injuries per
People Injured Aircraft Operations 10,000 operations
Jan 6 6648 9.03
Feb 5 5180 9.65
Mar 4 4895 8.17
Apr 3 3650 8.22
May 4 5055 7.91
Jun 5 6230 8.03
Jul 6 8350 7.19
Aug 7 10640 6.58
Sep 8 10910 7.33
Measuring Trends
There are two methods to determine trends: visual and statistical. As shown in Figure 20,
graphing or mapping data for people to see is the easiest way to communicate trends, especially
to a non-technical crowd; however, this method does not allow one to "measure" that there is a
trend or how big it is. Even when a statistical analysis is performed, it should start with the visual
assessment of trends.
Another example is depicted in Figure 21. In this example, the root causes of runway incur-
sions by type of vehicle over a period of 1 year are counted. It can be noted that the operation of
emergency/snow removal vehicles was the main activity associated with runway incursions, and
the airport should take actions to provide additional training to those drivers. In addition, based
on the high frequency of incidents associated with communication procedures, the airport may
decide to provide additional training on radio communication, replace existing equipment, and
establish procedures for escorting contractors.
The statistical method, on the other hand, can identify hard-to-see trends and can give a num-
ber that is defensible and repeatable; however, such methods can be difficult and challenging to
apply. A statistical analysis is best performed by a qualified person.
Trends for Safety Performance Indicators
35
32
30 29
Number of Occurrences
Ramp Accidents
25 Bird Strikes
20
17 16
15
12 11 11
10 10
7 8
5 5 5
0
2003 2004 2005 2006 2007 2008
Year
Figure 20. Plotting for visual analysis (example).
OCR for page 133
SMS Operation 133
Root Causes of Runway Incursions by Vehicle Drivers in 2007
9
8
Number of Occurrences
7
6
5
4
3
2
1
0
Operation of Unfamiliarity with Lack of proper Communication Driver was
emergency/snow airport phraseology or equipment failure distracted
removal vehicles (contractor) equipment
Figure 21. Runway incursions in 2007 (example).
For most cases, a visual analysis with some very simple statistical measuring will be sufficient
for evaluating most safety performance indicators used in airport SMS.
Plotting and Smoothing
Spreadsheet software like Microsoft® Excel can be used to plot information on safety perform-
ance gathered by the airport. Usually, the information is plotted over the period that the data
were collected, sometimes called time series. When evaluating plots, care must be exercised
regarding the following issues:
· People tend to focus on the extreme values and not on more subtle changes
· Gradual trends are hard to detect by visual analysis
· Seasonal variation and exogenous variables can mask trends in a parameter
· Viewers can "see what they want to see" sometimes
· Finding no trend may only mean the data were insufficient or the type of plot was inappropriate
Using the example data for bird strikes in Table 26, the plot in Figure 22 can be generated.
Obviously, it is very difficult to state any conclusions from this plot. However, by using the
average number of bird strikes for the last 12 months instead of using the raw number of bird
strikes in any given month, a trend may become more evident.
Moving averages can be used to smooth out short-term fluctuations, thus highlighting longer-
term trends or cycles. The threshold between short term and long term depends on the factor
being analyzed, and the parameters of the moving average should be set accordingly.
For the previous example, the average number of bird strikes for Jan 2003 is computed by
averaging the results from Feb 2002 to Jan 2003. For Feb 2003, the value is the average from Mar
2002 to Feb 2003, and so on. Note that there are no data for the first 11 months simply because
there is no information available to compute the average for the past 12 months. The resulting
OCR for page 134
134 Safety Management Systems for Airports
Number of Bird Strikes per Month
3.5
3
Number of Accidents
2.5
2
1.5
1
0.5
0
Se 3
Se 4
Se 5
Se 6
Se 7
Se 8
M 3
M 4
M 5
M 6
M 7
M 8
03
Ja 4
Ja 5
Ja 6
Ja 7
08
-0
-0
-0
-0
-0
-0
0
0
0
0
0
0
0
0
0
0
n-
n-
n-
n-
n-
n-
p-
p-
p-
p-
p-
p-
ay
ay
ay
ay
ay
ay
Ja
Ja
Month
Figure 22. Number of bird strikes per month.
trend for bird strike data is depicted in Figure 23. In this case, the frequency of bird strikes was
decreasing until the beginning of 2006, when it started to increase.
Using Accident and Incident Rates
The examples presented so far have used raw data only. However raw data can sometimes be
misleading because the number of accidents and incidents tends to have a relationship with the
volume of operations or the intensity of activities.
Average Number of Bird Strikes for Previous 12 Months
Average Number of Bird Strikes
1.6
1.4
1.2
1
0.8
0.6
0.4
0.2
0
Se 3
Se 4
Se 5
Se 6
Se 7
Se 8
M 3
M 4
M 5
M 6
M 7
M 8
Ja 3
Ja 4
Ja 5
Ja 6
Ja 7
08
-0
-0
-0
-0
-0
-0
0
0
0
0
0
0
0
0
0
0
0
n-
n-
n-
n-
n-
n-
p-
p-
p-
p-
p-
p-
ay
ay
ay
ay
ay
ay
Ja
Month
Figure 23. Bird strikes trend using a 12-month moving average.
OCR for page 135
SMS Operation 135
The example data given in Table 27 puts the problem in perspective and helps your under-
standing. As noted in that table, the number of job-related injuries at the ramp dropped from
January to May; and after June the numbers started to rise. The trend is shown in Figure 24.
However, the volume of operations changed significantly during this period. One can expect
to have a higher number of accidents if more operations are carried out at the airport. If, instead
of raw numbers, the rate of accidents per 10,000 operations is used, a different trend can be
observed.
The rate can be calculated using the number of accidents in a given month and the number of
aircraft operations for that same period, using the following equation:
AccRate = TNAcc × 10,000/TNOps
Where:
AccRate is the rate of accidents per 10,000 operations
TNAcc is the total number of accidents in a given period
TNOps is the total number of aircraft operations during the same period
For example, using the data from Table 27, in January:
AccRate = 6 × 10,000/6,648 = 9.03 injuries per 10,000 operations
The fourth column in Table 27 depicts the accidents rates for the example data provided. When
plotting the accident rates, instead of the raw data, the trend shown in Figure 25 is observed; the
rate is clearly decreasing.
Trend Analysis Reports
As a minimum, the following information should be reported for each trend analysis:
· Display plots of the observed data over time
· Comments in narrative form on the stability of the rates and approaches used to improve it
· Report average percent change for periods when the rate is fairly constant
· Interpret in narrative form the trend and how it relates to achieving the safety objectives of the
airport
Number of Job-Related Injuries
9
8
Number Of Accidents
7
6
5
4
3
2
1
0
1 2 3 4 5 6 7 8 9
Month
Figure 24. Trend for job-related injuries at the ramp.
