Data Strategy: Taking a Business View
In the data industry, we talk a lot about the importance of having a coherent data strategy. And for good reasons. Data strategy aligns data-related activities with the strategic goals of an organisation. It’s about turning data into value. Those businesses that implement a data strategy make better informed decisions, perform better commercially, and have more satisfied customers. Every organisation should be putting in place a data strategy. It’s a no-brainer, right?
Then why do we constantly bang on about the value of a data strategy?
It turns out plenty organisations still don’t have one in place. For all sorts of reasons. Having spent significant time in business and understanding the value data brings, I really think they are missing out.
What is data strategy?
This subject has been beaten to death. But since it’s a standalone blog, I’ll summarise the key principles to save you time on research.
Data strategy is essentially a roadmap for acquiring, storing, managing, sharing, and using data for the benefit of the business.
At its core, it ensures data is given the same treatment as other valuable business assets. We have an HR strategy, commercial strategy, etc., to give us a structured path to better results. It’s the same with a data strategy.
Data strategy outlines clear objectives for data usage, governance, data quality, access policies, and data privacy. And it suggests a technology stack that supports the business goals.
Not really rocket science. If you ignore all the tech jargon associated with data, it’s not that hard. Its main aim is to set a formal framework to how the business maximises its value with data.
You are losing time and money without a data strategy
Data is everywhere in organisations. It’s your invoices, HR records, sales, inventory, finance, resource utilisation, you name it. It is structured and unstructured. It is periodic and live. It resides in various systems, documents and even people’s heads.
Without a data strategy, you are highly likely missing opportunities to apply that data to improve your core business. And more than likely paying for the things you don’t need or use.
Missing opportunities: In most organisations, data tends to sit in silos, often going underutilised or unused entirely. Most people in the company aren’t even aware of what data they have, let alone how it can help them make better decisions. You don’t know what you don’t know. They work on subsets of the “common” data (e.g. finance numbers, customer, sales, HR, resource planning, etc), mainly within their own work function. This data is rarely shared with other departments for more in-depth analysis, so your business does not benefit from better insights.
Wasting resources: Then, you have some seemingly random data that is being automatically collected and stored. There is no official owner. It was originally collected for a purpose (e.g. experiment or project) but has since been forgotten. No one seems to be using it, but there is ten years’ worth of it in the warehouse. It could be valuable, but the business users are unaware it even exists. Having data that goes unused is a waste. And you are paying for it: the devs are still maintaining the pipelines and infra.
Creating complexity: Another common and expensive issue is having myriads of systems that were added over time but never retired. There are “miles” of pipelines, obscure business logic, redundant functionality, and unclear ownership. Usually accompanied by eye-watering license costs that the business pays out every year (from different departmental budgets). There is a small army of devs dutifully maintaining this maze. It’s like mould growing in the bathrooms of your house if left unchecked. I met companies with hundreds of disparate systems, overlapping in all sorts of ways.
Why is a data strategy necessary?
The value of the data strategy lies in the right data being used effectively improve business activities.
I do occasionally hear “our business is already successful without a data strategy”. Our subject matter experts (SMEs) have years of experience and make good decisions. It’s just not worth the expense to develop a data strategy when we already doing well.
Then why is your data-savvy competitor outperforming you year after year? Why are your profits stagnating? Why are your warehouses brimming with unsold inventory? Why is your airport resource utilisation so low compared to the peers?
As a business, your goal it to make money. Data has proven critical to achieving that goal. Without data, you are deriving your insights from the “gut feel”, anecdotal evidence and past experiences. Nothing is wrong with that. It’s just not enough. Enriching your current decision making with data gives you a much more grounded analysis.
When we apply data, we see that the low airport resource utilisation is driven by the shape of the flight schedule. The schedule is driven by the crew roster design implemented years ago to save on the headcount at the time. This led to an inefficient schedule design that drove peakiness on the ground. The data shows that the cost saving today is more than offset by the inefficiency on the ground (three-fold). They were losing big $$ for the past several years and didn’t even know it.
See now why it’s important to put in place a data strategy?
Implementing a data strategy: avoiding the pitfalls
OK, you have decided that data strategy is indeed important to you and will bring benefits to the bottom line. How do you go about implementing one?
It can be a daunting exercise indeed. But mainly if you make it so. Many companies fail to implement a coherent data strategy because they try to do too much all at once.
“Let’s aim for the stars! We got tons of data. We’ll scrap all our legacy systems, migrate to the cloud, and embed AI into everything. We will build a large data team, give them top tools, and let them do their thing. They know data better than any of us. They will tell us how to make more money. We’ll be the best data-driven organisation ever!”
Get ready to fail.
This is a typical, top-down managerial misconception of a data strategy.
- You don’t uproot your entire commercial department when the sales are down. You set a vision and do incremental improvements.
- You don’t scrap your product delivery schedule when you find inefficiencies in it. You optimise what you have and gradually roll out improvements.
- You don’t re-fleet an airline overnight. You phase in new aircraft over time and retain flexibility to modify deliveries and spec.
Why should data be any different? Data is an integral part of your business. It lives and breathes with the business activities. Set an overarching framework, but start phasing the use of data incrementally, where it makes the biggest impact.
Set a global vision but break the implementation into manageable chunks
Focus your effort: You need to be realistic about what data can do for you. Data isn’t a magic bullet. It is a tool to help you do whatever it is you do now but in a more informed manner. If you are an airline, your business is to fly people around. Data can help you do it more efficiently, safely, and attract more passengers. So, focus your data strategy on improving that first.
Don’t over-engineer: Another way to fail miserably at delivering a data strategy is by over-committing to a specific architecture or tooling. Business evolves. Technology evolves. Nothing is static. The deeper you integrate with a tech stack of today, the harder (read costlier) it will be to unpick it three years from now when your needs change.
Do the math: Never spend more than you need to deliver your data strategy. If the tangible business benefit is $2m pa from using better data analytics, don’t spend $5m pa on your data platform and tools. This also goes for doing everything “open source”. You can easily spend just as much money on developer time to make it work for you. There is never a free lunch, remember that.
Keep it simple: Keep your data architecture and tooling as simple as possible. Build it modular for ease of management. Aim to have tools that perform multiple functions (such as end-to-end ETL, full-suite DW, feature-rich BI tools, etc) that are easy to use and maintain. Always keep in mind the total cost of ownership. It’s so easy to spend a fortune on data and analytics suites with shiny bells and whistles that are rarely used in practice. If Excel is what your SMEs prefer, let them use it.
Speed is key: Drastically shorten delivery timescales. The business needed the data yesterday, not eighteen months from now. Build for MVP, demonstrate value asap. Then you will get the buy-in and quick adoption. I’ve seen costly data projects fail to deliver value because they were “a day late and a buck short”.
So, to wrap it up, we can hopefully see now that a data strategy is essential for modern business operations. By understanding and implementing a sound data strategy, organisations can unlock make more informed decisions. These lead to growth, innovation, and a competitive advantage, which, in turn, translate to better operations and a stronger bottom line.
We always encourage implementing a data strategy in your organisation sooner than later. At IOblend, we are in the business of making data strategy implementation much easier than it has to be. We can help you design your strategy, deliver it fast and avoid costly pitfalls.
Get in touch and let’s see how we can make your data work for your business.
IOblend presents a ground-breaking approach to IoT and data integration, revolutionizing the way businesses handle their data. It’s an all-in-one data integration accelerator, boasting real-time, production-grade, managed Apache Spark™ data pipelines that can be set up in mere minutes. This facilitates a massive acceleration in data migration projects, whether from on-prem to cloud or between clouds, thanks to its low code/no code development and automated data management and governance.
IOblend also simplifies the integration of streaming and batch data through Kappa architecture, significantly boosting the efficiency of operational analytics and MLOps. Its system enables the robust and cost-effective delivery of both centralized and federated data architectures, with low latency and massively parallelized data processing, capable of handling over 10 million transactions per second. Additionally, IOblend integrates seamlessly with leading cloud services like Snowflake and Microsoft Azure, underscoring its versatility and broad applicability in various data environments.
At its core, IOblend is an end-to-end enterprise data integration solution built with DataOps capability. It stands out as a versatile ETL product for building and managing data estates with high-grade data flows. The platform powers operational analytics and AI initiatives, drastically reducing the costs and development efforts associated with data projects and data science ventures. It’s engineered to connect to any source, perform in-memory transformations of streaming and batch data, and direct the results to any destination with minimal effort.
IOblend’s use cases are diverse and impactful. It streams live data from factories to automated forecasting models and channels data from IoT sensors to real-time monitoring applications, enabling automated decision-making based on live inputs and historical statistics. Additionally, it handles the movement of production-grade streaming and batch data to and from cloud data warehouses and lakes, powers data exchanges, and feeds applications with data that adheres to complex business rules and governance policies.
The platform comprises two core components: the IOblend Designer and the IOblend Engine. The IOblend Designer is a desktop GUI used for designing, building, and testing data pipeline DAGs, producing metadata that describes the data pipelines. The IOblend Engine, the heart of the system, converts this metadata into Spark streaming jobs executed on any Spark cluster. Available in Developer and Enterprise suites, IOblend supports both local and remote engine operations, catering to a wide range of development and operational needs. It also facilitates collaborative development and pipeline versioning, making it a robust tool for modern data management and analytics
IOblend: Simplifying SCD for Real-Time Analytics
Businesses rely on accurate, up-to-date data to make informed decisions, which is why understanding and managing slowly changing dimensions (SCDs) is crucial.
Metadata Management Made Simple with IOblend
MetadataIn today’s data-driven world, information reigns supreme. Businesses and organizations are constantly seeking ways to extract valuable insights from their data to make informed decisions. One often overlooked but essential aspect of this process is metadata. Metadata is the unsung hero that empowers data management, analytics, and decision-making.In this blog, we will delve into the
Change Data Capture: IOblend’s Seamless Approach
Change Data Capture In the fast-paced world of data management, staying ahead of the curve is not an option, it’s a necessity. Change Data Capture (CDC) is the secret weapon that allows businesses to keep pace with the constant flux of data. In this blog, we will delve into the world of CDC, explore different
Data Schema Management with IOblend
Data Schema Management In today’s data-driven world, managing data effectively is crucial for businesses seeking to gain insights and make informed decisions. Data schema management is a fundamental aspect of this process, ensuring that data is organized, structured, and compatible with various applications and systems. In this blog post, we’ll explore the significance of data
Smarter office management with real-time analytics
Commercial property Welcome to the next issue of our real-time analytics blog. This time we are taking a detour from the aviation analytics to the world of commercial property management. The topic arose from a use case we are working on now at IOblend. It just shows how broad a scope is for real-time data
Better airport operations with real-time analytics
Good and bad Welcome to the next issue of our real-time analytics blog. Now that the summer holiday season is upon us, many of us will be using air travel to get to their destinations of choice. This means, we will be going through the airports. As passengers, we have love-hate relationships with airports. Some