Airline safety management: enhance your SMS with IOblend
Today we are looking at the data aspect of flight safety management in the aviation industry.
Flight safety management is a very complex operation, involving multiple people, systems, data and in-depth analytics. The pieces must come together in a seamless process to ensure that the airline not only complies with the safety regulations but also stays ahead of the “curve”. This process is called Safety Management System (SMS).
SMS is a systematic and proactive approach to managing safety risks. Risk management is at the heart of SMS. Key areas are identification of safety issues (e.g. number of unstable approaches), risk assessments (severity of unstable approaches) and mitigation (SOP to avoid and deal with the unstable approach events). SMS requires a strong assurance function that monitors compliance and performance.
The principle of the SMS in an airline is to:
- Collect occurrence data
- Identify hazards
- Assess the risks (by combining the likelihood of occurrence and the possible consequences of each hazard)
- Identify and put mitigation measures in place
- Monitor the efficiency of the mitigation
Data is at the heart of SMS. There are multiple data sources involved in SMS, with flight data being one of the key ones.
Background
Airlines apply Flight Operational Quality Assurance (FOQA) methodology for the routing capturing, monitoring and analysis of flight operational data to provide more information about, and greater insight into, the total flight operations environment. Alternatively, this methodology is also known as Flight Data Monitoring (FDM). FDM data is often collected by a Quick Access Recorder (QAR), a system installed aboard an aircraft. The FDM process inherently belongs to the Safety Management System (SMS) of an airline. FDM is an efficient input to SMS for flight operations.
FDM data is generally used in the following analyses:
Continuous comparison of flight profile, engine and systems operation to detect exceedances
- Compilation of data to obtain an accurate overall picture of the operation and the condition of engines and systems
- Diagnostics, research and incident investigations
Flight Operations
- Non-compliance and divergence from SOPs
- Inadequate SOPs and inadequate published procedures
- Ineffective training and briefing, and inadequate handling or command skills
- Fuel usage
Maintenance
- Aerodynamic inefficiency
- Powerplant deterioration
- System deficiencies
FDM
FDM covers a set of core events established in the main areas of interest for the different flight phases (Taxi, Take-off, Cruise, Descent, Approach and Landing). Many aircraft flight parameters can be collected (such as IAS, AOA, accelerations, heading, etc). The data from the flight systems is stored in the flight data recorders and QAR that are installed onboard the aircraft.
The data is downloaded periodically when the aircraft reaches a suitable station or maintenance base. Data transferring from the aircraft to the storage system normally requires a set of media, such as tape cassettes, optical disks or PCMCIA cards, or portable computers. This data can be quite large. Boeing 777 processes around 60,000 parameters continuously. Recording 2,000 of these produces 50Mb of compressed data per day for each aircraft.
The resulting data is stored in a large database with the airline (operator). The data is then processed by the airline to provide a corresponding series of flights and flight events to the end user.
Analysis is the core of the FDM process, such as monitoring the occurrence rates and trends of the various events (e.g. number of unstable approaches in a period). FDM can be assisted by a software suite to make the analysis of the data easier.
FDM provides the capacity to analyse a wide range of parameters and to identify contributing factors that will help to assess and understand the root causes of incidents. Since FDM gathers the data of the complete airline or fleet, the analysis provided in a weekly or monthly report allows one event to be analysed in a wider context instead of being focused on that single event.
FDM sits at the heart of the airline’s safety management. While the FDM data is recorded automatically, processing, cleaning and enriching it for the use in the SMS (or other functions) is still largely a manual process. In addition, FDM data requires very strict access security and guardrails to ensure only intended users can work with the data and only for specific purposes.
An airline’s SMS incorporates a variety of data inputs, FDM being central. Events are analysed in a broader scope of information that better describes the occurrence.
For example, the FDM will record an unstable approach at a set altitude. It will show what the aircraft was doing at the time (airspeed, AoA, descent rate, equipment status, etc). However, it would not be able to show what the crew were going through during the event, what their training/experience/fatigue levels were, what the weather conditions were like, whether an SOP existed for such an event, etc. Many factors must be considered against a set of thresholds, scaled across thousands of flights for a larger airline.
Some events must be dealt with right away, while others are observed over time to assess whether a pattern is developing. Good quality data is essential for SMS.
SMS data processing with IOblend
Our technology and experience working on flight safety data from FDM and other inputs allows us to enhance an airline’s analytical capabilities significantly, especially where a lot of manual data processing is still involved.
We work closely with the customer to understand their data requirements and develop a bespoke solution using IOblend DataOps.
- FDM data is often protected and of a sensitive nature. IOblend easily complies with the set policies.
- All data sources are accessed by any of the following means:
- API
- JDBC
- File systems (e.g. Linux, iOS, Windows)
- Flat files (e.g. CSV)
- Blobs
If required, IOblend can setup a secure private, managed cloud or on-prem data infrastructure (or assist the airline to set it up themselves). The infrastructure can be used to store and compute the data from various data sources and subsequently channel it to its ultimate destinations.
Once IOblend is connected to the data sources:
We will evaluate the original QAR/FDM data and assess it with respect to data quality attributes: completeness, consistency, accuracy and timeliness. An example of the completeness evaluation follows below. Similar checks will be conducted for all other data quality attributes.
- The original QAR/FDM data is assessed against the airline’s requirements
- If the required QAR/FDM data fields are missing or incomplete, a solution will be defined with the airline, for example:
Data owner updates their processes and SOPs for QAR/FDM to reflect the required fields, and/or
We work with the data owner to determines what missing QAR/FDM data fields can be accessed from other supplementary sources (such as flight designation, origin/destination, equipment types, weather, etc from Flight Ops, ACARS, M/X, FMS, Scheduling).
Typically, the unique records are identified on primary keys. In the case of the airline operational data relating to a flight (and are stored in several other systems such as ACARS, FMS, AMOS, Flight Ops, Flight Planning, Scheduling) these usually are:
- Flight designation
- Datetime
- Equipment
- Orig/Dest
- Crew ID
- Engine type and ID
These mappings will be identified and implemented into an IOblend data pipeline to effectively tie together the data.
- Duplicate data will be automatically removed from the end product (but can be retained in the archives if needed)
- We will work with the data owner to define appropriate rules for consistency and accuracy that will be implemented into the IOblend data pipeline
All data quality rules are added as transformations in an IOblend data pipeline. These are defined as either SQL or Python rules that are applied to the incoming data. Any data that fails to meet any one of these rules is automatically moved to a holding table that can be inspected by the data owner and fixed and re-integrated into the data pipeline later. Users have the option either to allow the successfully cleaned data to be processed (and “farm” out the erroneous data) or stop the whole process entirely until a manual intervention is applied.
We will use IOblend advanced DataOps solution to develop one or more fully productionised data pipelines to extract, clean, transform, monitor, archive, and make available for onward consumption the QAR/FDM data. These pipelines will address the business logic of any complexity (i.e. any integrations are possible). We approach the pipeline design in a modular way and can easily switch on/off various elements, define execution sequences and dependencies, add/remove data sources, and modify transformations. The QAR/FDM data can also be further enriched with additional internal and external data, computed fields and/or reference data as part of the main design and ad hoc projects.
A brief explanation of how we easily create data pipeline templates and how they are reused follows below (watch our videos for more info):
- Transformations are one of three types of components in an IOblend data pipeline, the others being source (defines where source data comes from) and sink (defines where the result of a data pipeline goes to). There can be multiple unlimited sources, transformations and sinks in a data pipeline.
- Each component is stored as a JSON file. Each of these files is stored in a project folder. To re-use a data pipeline project, all one has to do is copy the project folder, re-name it and open the data pipeline project folder in IOblend.
- Likewise, to re-use a component from another data pipeline project you can copy the component’s JSON file into the IOblend project folder where you want to use it. Then when you open the project in IOblend Designer, the component will appear and be ready to use. This means generating templates and changing them with IOblend is very easy.
The pipeline’s output will be made available for consumption on demand.
IOblend data platform will automatically monitor data quality issues in real time. If any system fails to provide the data on time or the data is corrupted, IOblend will highlight the missing records and attempt to automatically reacquire them from the main source within the event window or the specified secondary sources (and log the data record origins).
As required, the QAR/FDM data can form a part of additional data products, where the flight data from the system enriches a complex output using multiple other inputs (e.g. M/X, ACARS, Flight Planning, etc). Relevant fields will be integrated into the new products.
The data products can then be easily integrated into other projects, shared for wider studies, safety audit reports, etc. IOblend DataOps supports it all, giving the airline time to focus on deep analytics instead of worrying about data issues.
IOblend is a highly scalable platform, able to handle large amounts of data and lots of small data, batch and streaming. It has been developed on top of Apache Spark, the world’s most powerful data framework, and uses a Kappa architecture.
IOblend has been designed as a no-code/low code solution to greatly accelerate pipeline development. Let your analysts and SMEs take charge of their complex data.
All created assets and templates are stored in JSON files and can be easily searched and re-used to further streamline development. Once the solution is running, very little supervision is required due to the high degree of system automation. You manage your data issues “by exception”. We strongly believe in the principle of “build once, build to last”.
Step-change your development effort and cost. What took months now takes days. Totally platform-agnostic. The advanced data capabilities you wished you could have in your airline, are now within easy reach.
Get in touch today. Let’s discuss how we can assist you with the SMS.
Data Lineage: A Data Governance Must Have
Data lineage is the backbone of reliable data systems. As businesses transition into data-driven entities, the significance of data lineage cannot be overlooked
IOblend: Simplifying SCD for Real-Time Analytics
Businesses rely on accurate, up-to-date data to make informed decisions, which is why understanding and managing slowly changing dimensions (SCDs) is crucial.
Metadata Management Made Simple with IOblend
Metadata In today’s data-driven world, information reigns supreme. Businesses and organizations are constantly seeking ways to extract valuable insights from their data to make informed decisions. One often overlooked but essential aspect of this process is metadata. Metadata is the unsung hero that empowers data management, analytics, and decision-making. In this blog, we will delve
Change Data Capture: IOblend’s Seamless Approach
Change Data Capture In the fast-paced world of data management, staying ahead of the curve is not an option, it’s a necessity. Change Data Capture (CDC) is the secret weapon that allows businesses to keep pace with the constant flux of data. In this blog, we will delve into the world of CDC, explore different
Data Schema Management with IOblend
Data Schema Management In today’s data-driven world, managing data effectively is crucial for businesses seeking to gain insights and make informed decisions. Data schema management is a fundamental aspect of this process, ensuring that data is organized, structured, and compatible with various applications and systems. In this blog post, we’ll explore the significance of data
Smarter office management with real-time analytics
Commercial property Welcome to the next issue of our real-time analytics blog. This time we are taking a detour from the aviation analytics to the world of commercial property management. The topic arose from a use case we are working on now at IOblend. It just shows how broad a scope is for real-time data
