Findings Without Indicators

Why small-operator safety audits so often change nothing — and what to do before May 2027

Under the FAA’s 2024 Safety Management Systems final rule, Part 135 operators and commercial air tour operators under § 91.147 must have an SMS in place and submit a declaration of compliance by May 28, 2027. For several thousand operators this will be the first formal safety management programme they have ever run.

The risk I want to name is not that these operators will fail to build something. Most will. The risk is that they will build two things that never meet: an audit file and a set of safety performance indicators, each complete on its own terms, neither informing the other.

I have seen this at an airline with a mature programme and ample staff. A small operator building from zero in eighteen months is more exposed to it, not less.

The NTSB has already described the gap

In its July 2024 special investigation report on Part 135 operations, the NTSB examined 116 fatal and 460 nonfatal accidents between 2010 and 2022. Among the three systemic issues it identified was the absence of proactive safety management. In one investigation, the president of an air tour operator told investigators he had believed the operation was too small to maintain an SMS.

That belief is the first thing the rule removes. The second thing — which no rule can remove — is the assumption that having the programme is the same as using it.

Where indicators should come from

The common practice is to adopt indicators from an industry template or a parent organisation. Borrowed indicators have a shared defect: they measure somebody else’s risk distribution.

The practice I ran worked on an annual cycle. At the end of each year I drafted the company-level safety performance indicators for the year that followed; those were issued to the operating departments as the basis for their own departmental indicators; and performance against them was monitored month by month through the year. I did that for six years. The part of the cycle that did the work was not the year-end assessment. It was the monthly review — the mechanism by which a deviation became visible before it became an event.

A useful diagnostic: if an indicator has been met every month for a year, it is either set too loosely or it is not measuring what actually threatens this operation. An indicator exists to raise an alarm early, not to be awarded at year end.

Audit findings are an input to indicators, not a substitute for them

An audit produces twenty findings. The usual next step is to sort them by department and distribute them for correction. That sort is the point at which most of the information is lost.

Sort them instead by contributing factor. Of those twenty, how many point to the same class of procedural defect? How many to the same gap in training coverage? How many to the same break in a cross-department handover?

Sorting by department tells you who must act. Sorting by contributing factor tells you which part of the system keeps failing — and only the second can feed an indicator, because only the second tells you what to monitor next year. Of twenty findings, the valuable content is usually the three that recur.

Investigation: stopping at the deviation

Whether an unsafe-event investigation is organised by the operator itself, or by the regulatory bureau within its jurisdiction, the method is the same: interview the personnel involved, review operational and technical documentation, identify contributing factors, develop risk-control and corrective measures, and track them to closure. I have done this work from both sides.

The failure mode is concentrated in the third step, and it is almost always the same failure: treating “the individual did not follow the procedure” as the conclusion.

That is an observation, not a cause. The questions that follow it are the investigation. Was the procedure executable under the conditions that applied at the time? Does it conflict with another procedure? Was the information needed to execute it available at that moment? Did training cover this scenario, or only the nominal one?

Stop at the deviation and the corrective action can only be “reinforce training and ensure compliance.” That phrase has appeared in a great many corrective-action reports and has changed very little. Carry the analysis through to a procedural defect or an information gap, and the action becomes specific, and therefore verifiable.

What process variation tells you that event counts do not

I hold a Lean Six Sigma Green Belt and completed Black Belt training with GE Aviation. The value of that training in safety management is not better statistical reporting. It is a way of looking at variation.

A single serious event commands attention, but it is rare and gives you one data point. The variation in routine minor deviations — how differently the same task is performed across crews, shifts and conditions — is a continuous signal about the current state of control.

Variation widening means control is loosening. Variation narrowing means a measure is working. Either reading is more informative than “no events this month,” which an operator with few flights will report most months regardless of its actual condition. For a small operator, this is the difference between an indicator set that works at low volume and one that does not.

What to do before May 2027

A declaration of compliance is a point measurement. It records that the required elements existed on a date. Whether the programme works is a different question, answered over time, by whether deviations are seen earlier this quarter than last.

Three connections are worth building while the programme is still being designed, because they are far harder to retrofit. Classify audit findings by contributing factor as well as by department. Let each month’s indicator deviations become next cycle’s audit scope. And require every corrective action to name its verification method, its verifier and its verification date at the moment it is written.

Build those three and the audit file and the indicator set stop being two ledgers. They become the two ends of one loop.