Building a flying program
In the last blog, I have covered how airlines plan their route networks using various strategies, data sources and analytical tools. Today, we will be covering how the network plan comes to life.
Once the plans are developed, they are handed over to “production”. Putting a network plan into production is quite an involving effort. It requires a rather large number of people, technologies, systems, procedures and processes to come in concert to deliver what will be eventually called a “flying program”. A flying program is what the airline will (aim to) fly and what we, as passengers, are going to experience. Let’s have a closer look at what is involved.
The network plan sign-off
Before a network plan makes it to production, however, it requires a formal sign off from the senior management. The plan informs an initial view of the expected financial performance and resource requirements, which determines the shape of things for at least the next year or two. The airline must get very comfortable with what they are committing to, because getting the network wrong will make a material negative impact on the profitability. So naturally, the network plan gets a lot of scrutiny from all sides of the business, voicing concerns left, right and centre.
“We don’t have enough crew to deliver this plan.” “The fleet deliveries may be slipping again.” “Are you saying we need to open a new crew base in X country in less than a year?” “Impossible, there are not enough stands at this airport to park all the planes!” “I’m not sure we can get a favourable deal from the Y supplier”. The list of concerns goes on and on.
Data is at the heart of this process. It is essential to get good quality data and associated analytics in front of the debate. Otherwise, opinions and anecdotes drive the conversations. Airlines can be very opinionated places. A growing airline will put significant pressure on the departments, getting them out of their comfort zones. It feels like a never-ending firefighting exercise on a company-wide scale. It’s tough. There is a lot of pushback. This is why it’s crucial to make decisions based on facts, or plans will not progress.
After countless reviews and deliberations, the network plan gets the final sign-off. With the management’s blessings, the plan is handed over to the production departments. Production is normally divided between Scheduling and Ops planning, where the former develops a flying program, and the latter ensures it can be delivered. It is a collaborative effort.
Role of data in schedule development
Depending on an airline, this process can be on-going (rolling monthly schedule development), seasonal (build separate seasons), annual or some other cuts of the same pie. But the schedules are hardly ever a “copy-paste” from the previous seasons. Building schedules is just as much a creative art as it is science, and the development window is never big enough. Schedule development and production are some of the most stressful aspects of the airlines.
Scheduling will use a dedicated point system to create and manipulate schedules. Systems from Sabre or LH Systems are commonly used. The system requires a large number of data inputs before it can be used productively. It takes multiple data inputs, like a detailed fleet plan, flight numbers, slot availability, crew rules (each airline will have multiple flavours), airport curfew restrictions, firebreaks, flight revenue performance, flight banks, transfer windows, etc. The more parameters an airline can digest during the schedule development phase, the more informed the program will be as a result. Some airlines have started using schedule optimisers and simulators to help them find novel solutions.
That sounds like a prudent approach, using data to inform the decisions. The issue is that the data inputs come from mainly manually maintained datasets and various siloed systems. Data quality issues are rife. The Scheduling teams spend significant amounts of time gathering, cleansing and manipulating the data into a usable shape. Poor data quality will delay schedule development significantly, so it is a constant struggle balancing data issues with the pressing timescales (art & science, remember?). The teams must make the correct calls as much as possible – schedule reworking is a major drain on time and resources that few airlines can spare.
If the airline is using optimisation tools, those will require many months of data cleansing and system calibration before it can produce usable results. But if the data is even slightly off, it will create truly bizarre scenarios that are of no use to anyone.
Operations planning
In parallel, Ops planning are busy developing their own solutions to the new and existing challenges. They work on ensuring there is sufficient crew available (and at the right places) to fly the program, that the training pipeline can handle both new joiners and recurring training. The airline must have sufficient crew experience available to fly the program safely, regardless of its complexity.
They also coordinate fleet planning, maintenance and standby aircraft requirements to decide how many airplanes and what types are available on any given day. The task is a lot harder than you may think and the exact availability may not be decided until the very last moment. An aircraft may be late coming out of maintenance, new aircraft deliveries can slip, it could have been damaged, etc. The airline can dispatch a standby aircraft in many of such cases, so the Ops team puts some aircraft aside as a safety buffer. But each aircraft on standby is an aircraft that makes no revenue, so the airline carefully balances this trade-off. If they manage to predict the issues arising far in advance, they can request a reduction in the flying program (not ideal due to the foregone revenue), procure additional “bridging” aircraft (usually as an expensive wet lease) or do magic.
Then the airlines must overlay all kinds of other operational restrictions. These include ground handling constraints, safety and security policies, flight paths, fuel contracts, de-icing, regulatory requirements, political landscapes, marketing events, base openings/closings, night stops, etc.
There are a lot of people involved in the schedule production phase. The process gets quite messy, but over time it gradually funnels towards a “flyable” program. Depending on an airline, the schedule can be further broken out by sub-seasons (monthly, holiday seasons, peak/off-peak, etc) and the portions firmed up on a rolling basis. This is done because of the uncertainty to either complete the full production phase on time, slot availability, tactical moves, or uncertainty to deliver the full program from the operational side. When you get flight change or cancellation alerts weeks or months in advance, this is why.
The flying program gets a formal sign off from the senior management before it both goes on sale and is then operated. Yet the process does not stop there. Schedule adjustments go on until the flight is completed. This part mainly occurs within the Ops control (flight management departments), which handles the schedules in its last stages before the flights and through the actual flights – fully dynamic flying program management. This is a whole other topic for later, however.
Data quests
Having gone through multiple iterations of network development and production cycles at easyJet (and having experienced them in various other airlines), one thing is obvious – airlines are a perfect setting for an internal federated data exchange/mesh type of data architecture.
Airlines are very siloed places when it comes to data. Each department will have their own custom-cooked solutions (spreadsheets), manually feeding off a point system or physical documents. They are the custodians of their data and would (or would not!) share it with the other teams for analytics. Data sharing is highly dependent on the technical ability of the custodian to provide it. Some SMEs have very little data training beyond copy-pasting a set of data from their core spreadsheet. You often had to do it for them to ensure the data you needed was at least of the desired shape. You’d sift through the quality of it later.
There is no catalogue or even localised metadata anywhere in sight. You go on a companywide quest of data discovery if you need a new data set. It’s often a “not what you know, it’s who you know” approach and can take quite a bit of time to locate the right person in a large organisation and then convince them to provide the data to you – remember, it’s extra work for them, so they’d be doing you a favour. It will cost you a cup of coffee and several meetings (or a formal request from the line manager and weeks of wait…).
But if you don’t know what you are looking for, you may never discover what data there is out there to enrich your analysis. It can be very frustrating. Airlines are gold mines of data, but they only ever use fractions of it, predominantly in departmental siloes built on legacy tech. Every department is generally highly protective of their datasets, often for no other reason than distrust of the other teams.
Data federation is the way to improve airline analytics
I’ve seen multiple attempts to organise the data centrally and they all failed miserably (and will continue to do so). The complexity is just too big to sort it out with a big bang. But a federated data exchange will work well if properly implemented. It will not even have to put undue pressure on the data owners to manage the “data products” beyond what they already do today. Their spreadsheets are already their data products. They may not always be of solid quality (that can be addressed over time with feedback and automation), but they are at least maintained on a somewhat regular basis and can be consumed on demand.
A data mesh proficient system or toolset will be able to handle data cataloguing and seamless exchange of production grade data. The setup will require collaboration from all parties involved. My experience shows that people want to make their lives simpler by allowing clever automation to take over their daily data struggles. Everyone hates manually updating their datasets every day/week/month. And they often want other data but don’t even know where to begin looking for it. The ground is fertile.
Data engineering should drive the delivery but the buy-in must come from the data owners. It can be done. We have already done it in localised settings. Implementation just needs to be done in a manner that does not add extra workload and stress on the data owners but makes it easy for them to participate in the new ways or working.
Start with a light-touch governance policy to make it quicker to implement. From the wider business side of things, you don’t need to add many data sources at first – start with the least challenging ones and grow from there. Once people see how beneficial it is to them, you can accelerate the rollout. Just make sure whatever methodology you apply can be easily scaled.
Scheduling, for example, can use IOblend to access the data from specified on-prem or cloud locations (systems and files) with a click of a button. You can control permissions who can access what data. Fully production pipelines will process the data automatically and deliver quality data for their analytics, not three weeks from now but instantly. Any issues will be flagged up right away and allow for easy audits to trace the causes. This is done “by exception”, so the team no longer has to process reams of data manually to check quality, completeness, etc.
The Ops planning teams can discover and readily share information among their functions in the same manner. Problems can be addressed well in advance of them becoming material. And with that kind of level of data maturity in your organization, you can start to implement MLops and run predictive analytics cases to prevent the issues from ever occurring. If you can better identify the causes of flight disruption at an airport, you can develop a solution to address it without sacrificing the program density, for example. I will even mention the NLP and LLM capabilities within your reach – very hot topics today.
IOblend can easily scale to cover use cases of any complexity.
The benefits are worth it
If you give your people weeks and months of their productive lives back, the quality of your analytics and production will improve immensely. The costs will come down too. If you can give your planners an extra month to work on the schedules, they will find new ways to make it more efficient, resilient and profitable.
If you give your Ops teams more time to source crew, negotiate new contracts or develop new predictive maintenance techniques, your bottom line will thank you. Airlines operate on very tight margins. You need to be able to constantly find efficiencies to stay competitive. A fully productionised data mesh will provide a material improvement to your data capabilities.
I strongly recommend you find a partner to help you put in place a solid data exchange architecture cost effectively. If you are exploring how to make your data work harder for you, get in touch. We can help you.
Improving airline analytics with a data mesh represents a transformative approach to airline data management. In the complex world of airlines, where data is often siloed and fragmented across various departments, a federated data exchange or data mesh architecture can be revolutionary. This method allows for seamless data cataloguing and exchange of high-quality data, facilitating collaborative efforts across departments without overwhelming data custodians. By implementing a data mesh proficient system, airlines can effectively harness their rich data reserves, simplify data maintenance, and improve decision-making processes. This approach not only streamlines operations but also enhances analytics, leading to more informed and agile responses to operational challenges.
Automate airline fuel data collection & management
Collecting and managing airline fuel data is complex and time consuming. IOblend can greatly streamline the process and enable real-time decisioning.
The Data Mesh Gotchas!
I think most practitioners in the data world would agree that the core data mesh principles of decentralisation to improve data enablement are sound. Originally penned by Zhamak Dehghani, Data Mesh architecture is attracting a lot of attention, and rightly so. However, there is a growing concern in the data industry regarding how the data
IOblend Data Mesh
IOblend Data Mesh – power to the data people! Analyst engineering made simple Hello folks, IOblend here. Hope you are all keeping well. Companies are increasingly leaning towards self-service data authoring. Why, you ask? It is because the prevailing monolithic data architecture (no matter how advanced) does not condone an easy way to manage the
Data lineage is a “must have”, not “nice to have”
Hello folks, IOblend here. Hope you are all keeping well. There is one thing that has been bugging us recently, which led to the writing of this blog. While working on several data projects with some of our clients, we observed instances when data lineage had not been implemented as part of the solutions. In
Welcome to the IOblend blog
Welcome to the IOblend blog page. We are the creators of the IOblend real-time data integration and advanced DataOps solution. Over the many (many!) years, we have gained experience and insight from the world of data, especially in the data engineering and data management areas. Data challenges are everywhere and happen daily. We are sure,