Saturday, October 7, 2017

Discussing Numbers

Analysis of data in aviation is an application of historical data to predict future events. This data is applied to predict hazards, but is not available as a tool to predict incidents or accidents. Incidents or accidents cannot be predicted since there is no capability within a data analysis to assign time and location of a future incident or accident.

Reporting hazards is SMS painting a picture of the operations.
The concept of predicting any hazard is the same concept as predicting the hazard of an inflight icing conditions. When an airplane is flying into icing conditions, the hazard of icing is predictable based on area weather forecast and knowledge, but it cannot be pre- assigned to one specific future incident or accident since variations are unpredictable.

A high volume of reports for one operator does not necessary conclude that this operator is a higher risk than another operator.  Raw data of reports are often assumed to be a higher risk and assigned a value without further analysis of the data itself. SMS has become a tool where competitors could apply this as their competitor advantage in a contract bid that they have fewer reports than their competition. Anyone who is unfamiliar with the function of an effective SMS could easily buy into this scam that fewer reports equal lower operational risk. This is backwards of what SMS is. Without data, there is a zero-confidence level of the safety culture, or how healthy operational safety is. On the other hand, with collected data the confidence level of how safe operations is can be established by a statistical process control analysis. However, a high volume of reports does not automatically ensure safe operations, but provides more data for analysis of an SMS to implement processes for continuous safety improvements.

Let’s take a moment and analyze reports from three small airports. These airports are similar in size, operations and movements and are within 200 NM of each other. The first airport reported 7 events the last 15 years, but did not make any report submissions the last 5 years. The second airport reported 66 events during the same period, with last reported event this year. The third airport reported 264 events during the same timeframe as the other two. At the first glance, it appears that airport #3 is a high-risk airport compared to the other two. This raw data does not tell a story or provide any valuable information to make statements of operational confidence level. Only by analyzing the data is it possible to paint a picture of operational safety and make statements related to a confidence level of safety in operations.

Analyzing an SMS more than discussing numbers. 
An initial analysis of the operations shows that airport #1 quit reporting 5 years ago. Based on trends this imply that the airport stopped all reporting and not that the events stopped happening. The reason is not known until a further inquiry into airport operations is conducted.
Airport #2 is continuing to report and the numbers of reports are steadily decreasing in numbers these last 15 years. Airport #3 shows a steady reporting structure chart where the numbers of annual reports are variable, but are moving above and below the average line in the chart. Airport #3 has therefore a healthier reporting culture than the other two airports.

When analyzing facts, opposed to opinions, airport #3 is an airport with accountable process for airport users to rely on information and data from this airport in their decision-making processes. Airport #3 can with a high level of confidence state that they have in place operational processes that paints an accurate picture of their airport operations. The other two airports have no supporting documentation to support their opinions that the SMS pictures painted are accurate pictures of their operations. 

Data, information, knowledge and comprehension of operational processes are vital components for continuous safety improvements. Analyzing SMS is more than discussing numbers, where the group or person with a louder voice and better vocabulary wins the argument. When applying strategies and solutions to airport safety it is not the numbers of events that becomes the issue, but the comprehension of airport operations.


CatalinaNJB