What makes a flight
Welcome to the next leg of our airline data blog journey. In this article, we will be looking at what happens behind the scenes to make a single commercial flight, well, take flight. We will again consider how processes and data come together in (somewhat of a) harmony to bring your flight to reality.
Once the schedule development of the following season has kicked off, the airline starts preparations to deliver it. Almost all airline functions get involved in this phase, leading all the way to the departure. It gets very busy indeed.
Once our flight is scheduled, it is given an origin/destination, flight number, dates of operation, departure/arrival times and aircraft type(s). Scheduling share their data with the rest of the business via SSIM extracts, an industry standard schedule issue format.
The SSIM is then translated/sliced into relevant cuts for other systems or dashboards to use. It can get quite technical to make it usable for other systems, especially if the data does not cover the entire schedule program, but only certain sections and time periods.The SSIM extract is ancient, designed when the mainframes still ruled the data world, so working with it requires particular finesse.
Revenue and Sales
One of the recipients of the SSIM is Revenue management. They will manually ingest this data, check for quality and completeness, adjust as needed, and prepare the system for a run. The system will then price our flight using proprietary algorithms that take into account multiple data points, like date, time, flight profile (business/leisure/vfr/etc), size of the aircraft, any historical information from past or similar flights, connecting traffic, booking classes, competitor capacity, seasonality, ancillary sales (bags, priority seating, etc), events, and other secret ingredients (e.g. manual adjustments).
The result will be a daily demand profile usually in a shape of an upward curve and the associated total expected revenue. Some airlines manage for load factor, others for revenue – different approaches. I’ll do a deep dive into the Revman function in a separate blog.
Depending on an airline, the schedule can go on sale around this time, so we can see and book our flight for the first time. Some airlines sell well in advance of ever completing operational preparations (12 months+). Putting a schedule on sale is an arduous process, requiring many hours of extreme effort on the part of the IT. Most airlines have legacy booking engines, so ensuring the schedules are released robustly takes lots of coding and testing. Any inadvertent changes to the core tables and code can break the system. Integrations with other systems and GDS providers further complicate the task.
A nightmare scenario for the business is the booking engine falling over. A half-day service disruption would cost several million in lost sales for a larger airline. The systems are legacy (decades old), very fragile and lack the new generation of smart features to improve sales and customer experience.
Yet hardly any airline dares upgrade or replace them with something more robust and suitable for the modern day and age. The original developers have long left, along with their knowledge of the mysterious spaghetti of data pipelines and obscure business logic. Documentation, you say. What documentation? Just barebones if you are lucky. As long as the system ticks over, no one wants to risk meddling with it. So, these systems persist with patches and workarounds until the day they inevitably fail.
Fleet planning
After a herculean effort, our flight is on sale and the bookings are starting to come in. We are still long away from departure, but the various functions within Ops have already kicked off their activities in preparation.
First, Ops planning must make sure they have the aircraft available to operate our flight on the day. Scheduling used a working plan, but now it needs to be firmed up.
Fleet planning oversees the numbers in a fleet plan document. It covers the number of aircraft in the feet at any given point, showing active equipment (available to fly), exits and deliveries, ownership status, aircraft in maintenance, and on standby. Lots of manually maintained spreadsheets, point systems and external data dumps thrive here. None of them are interconnected and only the fleet managers can make good sense of this data.
The fleet plan constantly changes, each day different from the next, so Ops planning need to ensure the aircraft is indeed available when it’s required. Otherwise, our flight may need to be cancelled, re-scheduled or a substitute aircraft procured (i.e. wet lease) – all three actions consume time and money.
Maintenance
Engineering constantly liaises with the fleet team to update the plan on the exact availability of the aircraft. Maintenance is generally a scheduled event, but the jobs can overrun, and unexpected part failures do occur. While predictive engine monitoring is now becoming prevalent, other components follow prescribed wear and tear patterns, making it harder to pinpoint exactly when the failures may occur.
This necessitates swapping parts before the end of their useful life (wastage) or after they failed (can cause AOG events). In addition, new aircraft deliveries have a thirty-day window, so it’s often impossible to pinpoint the exact date until the last moment, making the availability planning even more challenging. Better forecasting and real-time data integration with the production line and wider component monitoring would go a long way to help.
Crewing
Once you confirmed the aircraft availability, you need to crew it. Crewing makes sure our flight has sufficient personnel available, trained to the required standards and rostered to operate it. Crewing is normally split in two categories – flight deck and cabin. The flight deck (pilots) must have a captain and a first officer, having different levels of experience. You need to find, train and roster them appropriately. You are not allowed to fly with just two first officers, but flying with two captain-level pilots is considered wasteful.
Cabin crew (flight attendants) numbers are just as vital to get right. Regulations specify how many must be onboard to look after the passengers. They must be hired, trained and rostered for the flight in the locality of operation – bringing the crew in from the far-flung places is expensive.
Both these functions are related but require separate management. Flight crew rostering is distinct from the cabin crew as they follow different regulations, contracts and shift patterns. It is a delicate combination of resource and people management.
Each crewing function uses their own point systems and spreadsheets. In more progressive airlines, some form of an optimisation software may be used to better match availability with rosters, but mainly it is done in a spreadsheet by a team of experts. Mistakes happen, usually driven by poor forecasting and late management decisions, causing serious cost and morale issues.
Crew shortages are just as bad as having unproductive staff. You must have enough crew for every flight, covering peak seasons, rest, sickness, leave, emergencies, standby, overnight stays in a hotel, but not too many so that they are grounded in the off-season. The airlines can also easily get into trouble if they fail to spot dropping morale, higher attrition, excessive sick leave, and fatigue issues – all of which increase staff shortages and safety risk.
Crewing is one area that would greatly benefit from better data sharing and analytics. As quality data may not be available at the right time, Crewing make working assumptions, which can and do go wrong, sometime spectacularly. In just about every airline, this is a legacy function with legacy systems, ripe for innovation (hope you are reading this, airlines!)
Ground Ops
Next, we need to make sure our flight can be handled successfully by both the departure and arrival airports. This is where Ground Ops (GOps) step in. Again, driven by the inputs from the Network plan and Scheduling, GOps prepare to dispatch and receive the flight.
They must ensure the contracts are in place with the third-party service providers (e.g. Swissport or Menzies) at each end, self-check in kiosks are updated with the flight information, the correct flight handling equipment is in place, the check-in staff are available to attend the desks, bags can be loaded and unloaded, fuel supplied to the aircraft, catering provisioned, and the cleaning services arranged.
GOps data analysts use their own “master” spreadsheet to provide an indication of the resource requirements. This spreadsheet is often the main repository of historical data for the airline. The team would share its plans and forecasts via emails or text messages with the external suppliers and finance.
This is done manually and never in real-time. The data cannot be accessed automatically by the external systems and teams that require it. Ground service providers on the other end of this chain face a myriad of disparate data integrations from multiple airlines and at different times, trying to make sense of all the inputs and do their own staff and equipment planning.
Airport Procurement
Related to the GOps function is Airport Procurement. Their job is to get the best commercial deal from the airports that our flight will connect. This function plays a much more prominent role in the low-cost carriers, where the airport costs make up a higher relative percentage of the total flight costs. Procurement use planned passenger throughput to negotiate better airport rates.
Calculations vary widely by station and often rely on future traffic projections, type of customer (are they likely to spend money at the airport’s shops, parking and restaurants?), seasons (summer can be more expensive due to peak demand), time of day (morning and early evening departures/arrivals cost more at some airports due to congestion), aircraft size, noise levels, turnaround times, whether we’re using a gate or a remote stand and busses, and even aircraft weight. Data needs to be supplied for each of these variables to perform complex calculations.
Procurement do it all manually, each contract is a standalone model that doesn’t always use the same data sources. The contract’s logic is then rolled up into the finance models, but the legacy system often struggles to understand the complexity of the algorithms, so the numbers end up averaged. A better data integration solution that would sit in between Procurement and Finance would go a long way towards improving the accuracy of the cost accounting.
Regulatory and safety checks
While the ground handling and airport preparations are taking place, in parallel, safety and regulatory checks are carried out to ensure the full compliance before our flight can take place.
Airports that are considered “challenging”, like ones with short runways or those that are awkwardly located (e.g. in narrow valleys, middle of a bustling city or next to a beach) require special pilot training. The airline must also obtain in advance all overflight and landing permits if you want to avoid an international incident.
The airline shares its flight information with the relevant authorities via custom formats or docs, as not many understand the SSIM schedule format outside dedicated systems and SMEs.
Slot management
In addition to the regulatory compliance, some airports are also coordinated, meaning that an airline must have a permission (slot) to take off, land or use the gates there. Larger, congested airports like London Heathrow, Amsterdam, Paris or New York are coordinated, and the slots are very valuable (hard to obtain). Peak slots at Heathrow can be worth millions of pounds and become available once in a blue moon. SSIM is the primary data format for slot coordination.
Slot management is a core function of Scheduling and requires extremely high experienced SMEs to work on them, predominantly manually and withing a short time space. It is a complex task to say the least, requiring a careful balancing of the existing portfolio (poor flight performance can lead to the loss of the slot), trading and negotiating with other airlines (some slots can be worth more than a small airline!), plus lobbying the coordinators to get new or better-quality slots.
Scheduling extract and share multiple cuts of the schedule data with coordinators other airlines and internally. If you ever got a message that your flight was re-timed by 5-10min, this is highly likely due to the slot shifts.
Financial performance check
As we get closer to the departure date, Ops gradually firm up the operational parameters. Finance then calculate the expected profitability of our flight, using the live booking information for revenue and approximated, average operating costs – they still cannot attribute real costs to a flight, neither as part of the plan nor the actuals. The sums are rolled up into a financial plan, which the business then uses to gauge the expected profitability of the flying program.
If the flight appears to be underperforming commercially, Airport Procurement, Marketing, PR and Sales teams get activated to make the additional push. The airports are generally supportive of additional flights because they bring more revenue, so they are keen on the route’s success and can offer marketing assistance. Network and Scheduling may also get involved to alter the product – i.e. find stronger days and times of operation or shelve the flight altogether if it proves a basket case.
Flight Ops Control
Finally, if all looks good commercially and operationally, the flight gets handed over to the OCC, who will manage it all they through to completion. The OCC is a beating heart of the airline’s operations – everything is happening in real time. They get data feeds from the aircraft (ACARS/satcom), ATC, external traffic data (e.g. Flightradar), pilot comms, ground ops, maintenance, airports, weather, and make on-the-spot decisions to manage the flights.
The schedule that was planned and sold would always look different to the one actually flown. There will be re-times, crew and aircraft swaps, delays, diversions, disruptions, cancellations – all kinds of events that necessitate the changes. A robust feed of real-time data, automation and dynamic analytics capabilities could be of significant help here, as the job currently relies on very experienced SMEs working long shifts. If our flight leaves on time and with our baggage onboard, it means all the pieces fell into place successfully and the hard work has paid off.
Airlines have to cope with data challenges
As you now realise, the airline business is insanely complex. Considering how much of it depends on manual data analytics, legacy tech and archaic processes, makes you admire them even more. Just to make a single commercial flight happen, it takes a year of hard effort. I have lived this life for over a decade and gained a huge appreciation for the folks who work in the airline business.
Here is just a short a list of the data challenges that the airlines face today:
- Data integration across business functions is poor – most of the effort is done in-house by the engineering teams, who are not given the time and resources to build robust, modern ETL around the legacy infra and systems.
- On-prem or cloud warehouses only cater to a narrow set of the business data due to the immense coding/testing/deploying effort required to add new data sources, maintain/upgrade the existing ones, deliver data migration projects, and perform cross-system data integration.
- Most systems are legacy point-solutions that are hard to integrate.
- Data updates for such systems are manual and arduous, taking most of the analyst’s time.
- There is no master data management of any noteworthy significance in place.
- IT budgets are small in comparison to other businesses and face constant pressure, meaning that advanced technical solutions are often out of reach for many airlines.
- The analytics are overwhelmingly done in spreadsheets by SMEs.
- The top management only see periodic static summary KPIs in power-points and dashboards (great-looking ones!).
- Integrated analytics (especially real-time) that are crucial to this industry are still considered science-fiction.
- Coordination is done via endless meetings that progressively roll up the food chain and dilute the information, leading to late decisions and unclear direction.
- People do a lot of “firefighting” due to suboptimal forecasting, miscommunication, and a lack of dynamic analytics capabilities.
- Data science teams sit on the periphery instead of playing the central role in integrating all data systems and analytics under one umbrella.
When we speak with the airlines, they all agree with our assessment. But the task is usually seen an insurmountable – everything needs to change. Where do we start? There will never be enough money and resources to do it. Yet standing still and hoping things will magically improve will not work either. It is costing airlines millions in lost revenue and operational issues every year.
If ever there were a perfect case study on company silos, airlines would be it. Everyone beavers away in isolation. The policies for proper knowledge retention are weak and not supported by tech. In many departments, the entire knowledge base can walk out of the door when a person quits. It takes years to rebuild that knowledge from scratch, often costing the business hundreds of thousands in bad decisions in the process. But these costs are not surfaced, so they get ignored or attributed to other causes.
I am not suggesting a big bang change here. Start in the areas with the biggest ROI and rollout the improvements across the rest of the business. Invest in the newest technology – you’ll be surprised to see how much more cost-efficient it is compared to the dominant players. By doing things smarter, you will see drastic improvements to the bottom line over time.
You will make better decisions on fleet, staffing, ground handling, training, safety, disruption, CRM, sales, network, scheduling – you name it. Some airlines have realised the potential already (e.g. JetBlue) and use their data for real-time analytics and dynamic decisioning. Make sure you make your data team an integral part of planning and delivery. They must work side by side with the SMEs to support them with new technologies, process improvements and advanced analytics.
We have lived and breathed airlines for many years. At IOblend, we combine industry expertise and cutting-edge real-time data integration technology to help you improve your airline’s data estate efficiently and save you big money in the process. We help you make the most of the data capabilities no matter how difficult your user case may be. Especially so if it is real-time integration. Each airline has their own quirks, we know.
We have developed strong partnerships across the technology sector to bring you the best of what is available. No matter what your data challenges are, we will help you tackle them. There world of modern data integration is rich, catering for all sorts of use cases. See some listed here (compiled by our friends at Portable).
You have everything to gain here. Reach out to us – let’s start the conversation.
Improving airport operations with insights from IOblend involves a multifaceted approach where data-driven decision-making plays a crucial role. The complexities of commercial flight operations, from schedule development to crew management, necessitate precise coordination and efficient resource utilization. Utilizing real-time analytics, as exemplified by IOblend, can transform various operational aspects such as fleet planning, maintenance, and crewing. This approach enables better forecasting, enhanced data integration, and more effective crew management. By leveraging advanced data analytics and real-time information processing, airports and airlines can optimize operations, reduce costs, and improve overall efficiency, leading to smoother, safer, and more reliable air travel experiences.
IOblend Data Mesh
IOblend Data Mesh – power to the data people! Analyst engineering made simple Hello folks, IOblend here. Hope you are all keeping well. Companies are increasingly leaning towards self-service data authoring. Why, you ask? It is because the prevailing monolithic data architecture (no matter how advanced) does not condone an easy way to manage the
Data lineage is a “must have”, not “nice to have”
Hello folks, IOblend here. Hope you are all keeping well. There is one thing that has been bugging us recently, which led to the writing of this blog. While working on several data projects with some of our clients, we observed instances when data lineage had not been implemented as part of the solutions. In
Welcome to the IOblend blog
Welcome to the IOblend blog page. We are the creators of the IOblend real-time data integration and advanced DataOps solution. Over the many (many!) years, we have gained experience and insight from the world of data, especially in the data engineering and data management areas. Data challenges are everywhere and happen daily. We are sure,
