29th January 2006
FOQA - Flight Data Analysis
of Aircraft for Flight Safetyby Captain Mike Holtom, British Airways -- Source: PIA Air Safety Publication (Abridged from the presentation at FSF 52nd annual international Air Safety Seminar)
Introduction
British Airways (BA) started using flight data recorders 40 years ago and for the last 30 years all its pilots have taken for granted that their operations have been recorded and analysed by an exceedence program. Operational Flight Data Monitoring, known more recently as Flight Operational Quality Assurance (FOQA), is probably the most important safety tool available to aviation, yet it is fully operational in only a few airlines. Properly managed, the capital investment and running costs are recovered many times over. Pilot associations embrace its benefits, as do maintenance managers, accountants and insurers. Each year the industry identifies world accident causes, many of which would be even more predictable if airlines had the detailed knowledge provided by FOQA of how their aircraft are actually being operated. Data buses in modern aircraft make FOQA data readily accessible for analysis. Without the best information gleaned from that analysis we cannot make the best decisions regarding the performance of flight crew or aircraft systems or aviation's infrastructure.
In practice FOQA is the routine downloading and systematic analysis of aircraft parameters that were recorded during flight either by the crash-protected recorder or the Quick Access Recorder (QAR). The latter is easier to access and usually records far more parameters. Both are fed by one or more Flight Data Acquisition Units. Analysis of the data usually takes three forms:
Continuous comparison of flight profile, engine and systems operation with a set of defined parameters in order to detect exceedences.
Compilation of data to obtain an accurate overall picture of the operation and the condition of engines and systems.
Diagnostics, research and incident investigation.
From a Flight Operations perspective, a FOQA programme should identify:
Non-compliance and divergence from Standard Operating Procedures (SOPs).
Inadequate SOPs and inadequate published procedures.
Ineffective training and briefing, and inadequate handling or command skills.
Fuel inefficiencies and environmental un-friendliness.
From a Maintenance perspective, the programme should identify:
Aerodynamic inefficiency.
Powerplant deterioration.
System deficiencies (including those due to maintenance and aircraft design).
Aviation's Most Important Safety Tool?The Flight Safety Foundation has been vigorously supporting FOQA for ten years. Sound statistical evidence has helped other influential aviation bodies to proclaim FOQA's safety benefits.
UK CAA: "Such systems allow an airline to identify and address specific operational risks and are strongly encouraged as part of a Safety Management System."
FAA: "Because of its capacity to provide early objective identification of safety shortcomings, the routine analysis of digital flight data offers significant additional potential for accident avoidance. It is potentially the best safety tool of the 21st century."
Royal Aeronautical Society: "It is the most important way to dramatically improve flight safety."
Flight International: "Knowledge of risk is the key to flight safety. Until recently that knowledge had been almost entirely confined to that gained retrospectively from the study of accidents and serious incidents. A far better system, involving a diagnostic, preventive approach, has been available since the mid-1970s."
Over the last three years, there were 2,403 fatalities from the 65 accidents in which CFIT, landing short/long, and loss of control were deemed causal factors. The Flight Safety Foundation ALAR study recommended using FOQA to "identify performance trends that can be used to improve approach and landing safety". It also identified the following as the most frequent operational shortfalls:
Establishing and adhering to SOPs.
Recognising the need for and executing a missed approach when appropriate.
Unstable and rushed approaches.
Scandia Insurance has recently overlaid FAA data with that of non-USA airlines. This shows that airlines which have been using FOQA data for 7-14 years now have a lower accident rate than USA airlines, and those airlines which have used FOQA for more than 14 years have an accident rate under half that experienced by the US carriers.
Further protection is gained from combining FOQA with other safety management tools such as risk assessments, culture measurement, audits and periodic checks.
Some of The Difficulties Surrounding FOQA ImplementationIn this highly technical age and approaching the 21 st century it is curious why so few airlines have fully implemented a FOQA programme. Perhaps it is even more odd that such systems are not yet mandatory in at least some countries. In considering some of the difficulties which may explain this reluctance, it is worth remembering that they have nearly all been overcome in some parts of the world. However, FOQA will always be difficult to implement in societies where data is easily accessible to parties who would use it for purposes other than flight safety, i.e. unenlightened regulators, litigants, criminals and the media.
a) Motivation
Lack of motivation at a high level in an airline stems from not understanding the overwhelming benefits of a FOQA programme compared with the costs. This may be compounded by a mistaken belief that not knowing about problems means they do not exist.
b) Cost
The business difficulties associated with FOQA are mostly cost-based but many costs have reduced dramatically in the last decade. Data-buses in modern aircraft generally preclude expensive modification and re-certification of aircraft wiring that was once necessary. At least one Flight Data Acquisition Unit is normally as standard to feed the crash-protected recorder. Conventional desktop computers and printers are sufficiently powerful for data replay, and cost a fraction of their predecessors. The QAR however is normally an additional requirement.
A method of transferring data from the aircraft to the replay system is also necessary and this requires a set of media such as tape cassettes, optical disks or PCMCIA cards, or portable computers to "milk" the recorders. Early tests using infrared AR1NC Gatelink were considered to be unsuccessful but a new high bandwidth wireless Gatelink with greater range (more than one mile) seems to be more promising and does not need computer networks linking the gates. ACARS and Satcom are currently too expensive and the bandwidth is too narrow for bulk data download. Although broadband systems are planned, routine bulk transmission of encrypted data via satellite is unlikely for some years.
Replay and analysis software is complex and therefore tends to be rather expensive to licence and maintain. However, competition is bringing the price down.
Manpower is required for data retrieval, replay, analysis and system maintenance. The latter may extend from on-board equipment maintenance to software configuration and parameter adjustment for those airlines wishing to squeeze the maximum from their data.
One area of cost reduction to offset investment and running costs, is that a well-run FOQA programme should bring a reduction in insurance premiums.
A high level of technical expertise is required to obtain the best benefits. A service provider with that expertise could process raw data and supply a number of customers with the results of analysis. The chief barrier at present is the cost of transporting large volumes of data. In due course technical solutions are likely to make such enterprises cost-effective.
The last business difficulty to be mentioned is corporate structure. Vertical smoke stacks of large departments in airlines can make it difficult to produce a business case where benefits are spread across departments in a different ratio from costs. This is exacerbated when costs are more easily quantifiable than benefits.
c) Technical Difficulties and Standards
Many technical difficulties have been overcome during three decades of evolution. Inordinate advances in computing power, software development and digital avionics have made FOQA easier and cheaper. However, data volumes have increased significantly. The Boeing 777 continuously processes around 60,000 parameters and even recording 2,000 of them can produce 40 or 50 Megabytes of compressed data per day for each aircraft.
Evolution has not been accompanied by standardisation, but certain elements appear to be stabilising. AR1NC appears to be the most common data format and QARs generally record 12-bit compressed data words. "Compressed" means that the other 4 bits of the two-byte (16-bit) word are also used to store data. Some systems record uncompressed data; others use 11-bit words. Data frame formats vary and so does the number of words per frame (64 for older aircraft. 1024 for the latest). Synchronisation words are sometimes used for data storage instead of their intended purpose. Different equipment, such as radio altimeters, may be fitted to the same aircraft type but may send data in differing formats to the Flight Data Acquisition Unit (FDAU). These problems, however, are all relatively easy areas to handle.
It is more difficult for a FOQA system to synchronise data from multiple sources. Sensors work at different rates, control units have different internal processing speeds, and the FDAU samples the results at different rates. Consider also the processing times of flight instrument displays and the result is small, but potentially significant differences between sensing, recording and displaying to the pilot. Flight operational analysis should take account of these factors.
The sampling-rate problem is worth further explanation. For example, modern aircraft record normal acceleration up to 16 times per second to be sure of capturing short-duration spikes. For most purposes it does not matter whether the value obtained for each sixteenth of a second actually occurred at the beginning, the middle or the end of the sixteenth; or indeed if it were the maximum achieved during that period. Synchronisation of multiple parameters sampled at lower and different rates can be difficult. This has a bearing on event detection and it is therefore common for events to consider more than one data sample for each parameter.
Bad data can occur fairly frequently for various reasons. The effects on FOQA can be minimised by identifying data that is out of a defined range, or that has made a physically impossible step-change between consecutive samples. Synchronisation bytes when available can also be used to identify suspect data. Rules may optionally be applied during processing so that maximum benefit can be obtained regardless of bad data. Similarly, algorithms can be applied to achieve sensible interpolation. In both cases though, it is important to know that certain data was either calculated or estimated (not measured) and to understand how the values were derived.
d) Accessibility and Potential for Misuse
The possibility of an external party using voluntarily recorded data for litigation or for enforcement proceedings has been well publicised. However it is now being suggested that the risk from having the data is less than that of not having a FOQA programme.
Pilot associations are understandably concerned that data may be misused. Even if legal protection is given from freedom of information laws and regulatory enforcement, there is still a potential threat from an unenlightened pilot manager.
BA's FOQA Processes and BenefitsCurrently BA's Engineering department analyses 5 Gigabytes of flight data each day, taken from 8 aircraft types; more than 5 million flights have been analysed since 1966. All aircraft are fitted with QARs and approximately 94% of all flight operations are successfully analysed (nearly 100% for newer aircraft types). Data volume will increase to around 10 Gigabytes each day during the next few years of fleet replacement.
Some engine data is transmitted by ACARS. But all data is recorded on optical discs or mylar tape cassettes (on older aircraft). These are removed from the aircraft during overnight maintenance and loaded into hoppers. Robots automatically feed multiple replay units from the hoppers to achieve continuous analysis with minimum human involvement.
Routine analysis is carried out in three main areas: Flight Operations, Engine Health and Aircraft Performance.
1. Flight Operational Monitoring
Two types of data are retrieved during this analysis. In the first, which is event detection. The data for each flight is scanned for any exceedence of a defined parameter or any other special event. The software is called SESMA (Special Event Search and Master Analysis) and was originally developed inhouse in the 1970s. It identifies around 65 different events, many of which are common to different aircraft types and allow cross-fleet comparisons. For each event the detected values of the relevant parameters are recorded in a database and are used to calculate a severity index. The second data-type registered during the scan is the maximum or minimum value of certain parameters at particular phases of flight, for every flight. These are stored in another database, called MaxVals in BA, and its use is described later.
a) Event Detection
This is a selection event types detected by BAs SESMA program:
Abandoned takeoff
Altitude deviation
Abnormal pitch landing (high)
Approach speed high within 90 secs of T/D
Climb out speed low 400 ft to 1500 ft AAL
High rate of descent below 2000 ft AGL
Deep landing
Land flap not in position below 500 ft AAL
Deviation above glidepath below 600 ft AAL
Mmo exceedence
Early flap change after T/0
Pitch rate high on takeoff
Exceedence of flap/slat altitude
Reduced flap landing
Excessive bank above 500 ft AAL
Reduced tail clearance
Excessive pitch attitude
Speedbrake on approach below 1000 ft AAL
Flap placard speed exceedence
Stick shake
Go around from below 1000 ft
Tail strike GPWS windshear warning
WAS resolution advisory
High energy at 1000 ft
Unstick speed low
High normal acceleration at landing
Vmo exceedence
Some of Concorde's additional events:
Droop nose speed exceedence (all angles)
Reheat applied above Mach 1.75
Pitch attitude low above Mach 1.0
Reverse Thrust above 30,000 ft
Radiation - instantaneous
Reverse Thrust above 375 kts
Reheat applied above 46,000 ft
Tyre Limit Speed High at Take-Off
Examples of alert values:
Deviation above glidepath below 600 ft AAL: 1.5 dots fly-down for 3 seconds.
Excessive bank above 500 ft AAL : +/- 35 degrees for 2 seconds.
In BA we discover between 400 and 500 events per month; most of these are very minor. The number of events detected is influenced by the tightness of parameter setting as well as crew performance. Parameter setting is a balance to achieve the desired knowledge without overloading analysis resources or losing credibility (with crew, association or management). An interesting comparison can be made regarding events that have been detected for which the captain should file an air safety report. The level of reporting is an indication of the culture within the fleet. Cross-fleet differences may pose questions within management circles.
Although some events can be detected by one discrete (e.g. stick shake, GPWS), most require several parameters to be tested (e.g. high pitch attitude on landing). Complex algorithms are needed to detect deep landings and altitude deviations.
For each event a severity index is automatically calculated using the parameter values. For example deviation below the glideslope would take account of the degree of deviation as well as the height above the terrain. Numeric and severity-weighted trends are produced from the database of events. The latter more accurately reflects the safety of the operation.
Severity index algorithms were obtained for each event and each aircraft type using a technique called "Optimal Decision Maker". It was developed for the European Space Agency and is a hypothetical expert which comprises the best relevant expetise and weights each perspective according to importance. The technique is also used in the nuclear industry to give a measure of overall safety. In BA's case, the experts included flight ops managers, standardisation captains, fleet association representatives, aerodynamicists, performance, propulsion and avionics engineers and CAA's Airworthiness Division.
The severity index is based on a scale of 0 to 100 where 0 represents no risk to normal aircraft operation; 100 represents imminent Jeopardy for which immediate action is required to preserve the safety of the aircraft. A reference event that applies to all aircraft types is the onset of a hard GPWS warning: this is considered to have a value of 100. Severity may be greater than 100. It is more important for the severity levels to reduce than the number of events.
Also for each event a graph or trace of relevant parameters can be either printed or viewed on-screen. For the more serious events a subset of the data is archived to allow simulation of flight deck instruments and graphic reproduction of flight path. In these cases detailed feedback is sent to the crew (via the association) in the form of a few minutes of relevant data and an animation program on floppy disk for replay on a home computer. We are keen to understand why events occur and the crew's honest and detailed account is valuable in helping others to avoid the same event.
The pilot association provides a sound interface between management and crew. In BA the following arrangement has worked for many years:
There is a long-standing formal agreement between airline and association in which the critical words are: "Evidence from a Flight Data Recorder alone will not constitute a basis for any disciplinary hearing or action." In addition, if there is an internal investigation, the data is made available to the association immediately and it is classified as either raw, partially refined, or refined data (in accordance with department of transport accident investigation standards).
Pilot managers do not have the ability to determine the identity of crew who experienced events. However there may have been a written air safety report as well.
If the airline wishes to obtain further information or provide feedback regarding an event an association representative (a pilot on the same aircraft type, usually a training captain) makes the contact. If appropriate, the representative may assist in the understanding of SOPs. (Peer pressure is powerful, particularly when it comes from a body of professionals demanding high standards from it members)
Within the system, the frequency and severity of events is known for all pilots. Their identity is coded but the association has the "key". If any pilot gives cause for concern, whether detected by the airline or by the association, the latter will make its view known to the individual. Training is provided where command skill or handling technique should be improved.
Pilots in airlines without FOQA programmes often express concern that the data may contradict their version of the event. However, the opposite is nearly always true.
b) MaxVals
This data is used to produce trends and often provides answers to questions posed by analysing the event database. Consider rotation-rate for example, if an exceedence is detected and it is determined that technique was at fault, it is valuable to know whether the fundamental problem lies with the individual or with the training system. The distribution can be determined by MaxVals for almost any filtered subset of data, e.g. by period, location, aircraft variant, etc. The figure (not shown in this article) is a MaxVals chart showing maximum bank angle below 500 ft. There are two peaks, the main one at 3° and a secondary at 20°. Analysis shows that all bank angles above 11 ° occurred during a low level procedural turn approaching runway 13 at Hong Kong's Kai Tak airport. Even so, 18 of these 1079 approaches exceeded 30° of bank (the event limit, and therefore details are recorded in the event database) but none exceeded 35°.
Benefits derived from analysis can only be obtained by making changes. This could be within the airline through training, producing better procedures, or improved maintenance for example it might also be, and often is, desirable to improve the operating environment. BA has used its data to get runways resurfaced, aircraft systems modified and improvements to ATC procedures. In order to persuade other parties to make changes it is vital to have sufficient supporting data. In the future it is hoped that airlines will pool appropriate data to persuade the more intransigent third parties to improve our environment and infrastructure.
Change Management should be an integral part of any organisation's safety management programme. Processes to ensure that changes (to any aspect of the operation including people, equipment or environment) have a positive effect on safety. This is very difficult without an effective monitoring tool such as FOQA, and the staff to carry out analysis.
Benefits Could be Grouped into Four Areas:a) Non-Compliance and Divergence from Standard Operating Procedures (SOPs)
This is probably the most critical and useful part of FOQA and is a continuous audit of pilot performance. SOP compliance is strongly encouraged by the mere existence of the QAR and awareness that every input to the aircraft is recorded. FOQA analysis and feedback enhance that compliance. Using MaxVals and event detection, BA analysts look at the following SOP areas (among others):
Adherence to noise procedures
Approach stability and accuracy
Bank angles on approach and landing
Flap/gear selection speeds and exceedences
GPWS and WAS response and technique
Normal acceleration (airborne and landing)
Rates of descent close to the ground
Rotation rates and climb-out accuracy
Touchdown points
Use of reverse thrust and braking
b) Inadequate SOPs and Inadequate Published Procedures
If pilots do not consistently comply with SOPs it is perhaps sensible to consider first that the SOPs could be improved: For example, one FOQA airline discovered an error in its interpretation of the manufacturer's manual: the speed increment for final approach was derived from the full wind-speed rather than the headwind component. In strong crosswinds there was excessive speed during the flare resulting in either low pitch on touchdown or an extended flare.
Following a few incidents and analysing tail clearances, BA has reduced its tail-strike risk by reducing the target rotation rate on the B767. It has also extended flap life by altering flap selection procedures.
It is assumed, not always correctly, that published airport departure and approach procedures are all viable. It is surprisingly common for simulator refresher checks to reveal that some are almost impossible in certain wind conditions and aircraft configurations. BA is starting a project to compare published profiles with all its departures and arrivals.
c) Ineffective Training and Briefing, and Inadequate Handling or Command Skills
It is relatively straight forward to use FOQA to assess effectiveness of training, communication with crew, and briefing systems. An extreme example of this occurred in 1991 when BA was experiencing many false GPWS warnings. Analysis of 300 warnings revealed that only 13% were genuine. Flight crew's perception of the system was poor and only 40% of the warnings received the correct response. However 20% of the genuine warnings received no response. This unacceptable situation was handled by changing policy, introducing GPWS simulator training and helping the manufacturer to improve its system. Subsequent analysis of 120 warnings confirmed a successful programme (all received the correct response) and more recently EGPWS has been developed.
Events sometimes result from inadequate command skill. Decision-making and assertiveness should ensure that ATC clearances are rejected when not SOP-compliant or potentially unsafe. Similarly, if an approach is not stable, an early decision to go-around is company policy - with no recrimination.
FOQA also identifies pilots who need help in re-learning handling skills. In BA this is achieved through the pilot association; some airlines handle this through their independent safety department.
d) Fuel Inefficiencies and Environmental Infringements
Statistical analysis of route fuel and taxi fuel assists in refining flight planning. Early descents and long approaches are inefficient and environmentally unfriendly. They sometimes occur through controller incompetence or convenience. It is hoped that airlines using FOQA will pool data to identify airports and ATC areas that consistently cause these problems. For each aircraft type, such statistics could include these average fuel burns: 200 air miles to landing, engine-start to takeoff, to top-of-climb, top of descent to landing, and landing to engines-off.
FOQA is also used to identify non-compliance with noise procedures.
2. Engine Health Monitoring (EHM)( Omitted in This Article )
3. Aircraft Performance MonitoringThis covers a variety of areas: one of the most important is validating aircraft performance against manufacturer's specification, including analysis by manufacturer-supplied software.
For example, fuel-burn is compared across a fleet to identify aerodynamic inefficiencies that may be caused by inexact door alignment or control rigging. Statistical data is compiled from analysis of all autolands to ensure no degradation after certification. It is also used for diagnostics and to resolve design and maintenance problems; for example, wheel and brake reports. Data is collated to support cases for a change (e.g. runway resurfacing) and development (e.g. EGPWS).
Two other FOQA benefits should be mentioned. Firstly, monitoring the safety of franchise operations is far more reliable if the franchise agreement insists on a properly run FOQA programme in the franchise, with access to the data by the franchiser. Secondly, FOQA contributes to accident and serious incident investigation if the QAR data survives. There have been incidents where the more extensive QAR data has been invaluable in determining cause, because protected recorders often had a limited set of parameters. Use of data for accident investigation is well documented and beyond the scope of this paper.
The Future
From the text above, the authors wish-list is obvious, and hopefully will be achieved within the first 25 years of the 21st Century:
FOQA is mandatory (for the few airlines that still have not understood the cost/ benefit ratio).
DFDAUs and QARs are fitted to new aircraft as standard.
Data formats are standardised and well documented.
Legislation protects data against misuse.
Appropriate data is pooled to support arguments for improvement (by manufacturers, ATC, regulators and airports).
Broadband satellite communications allow cost-effective transmission and analysis in real-time.
The views expressed in this paper are the author's own and are not necessarily the views of British Airways.
