The Data Revolution: AI Takes the Wheel

The way companies leverage data is undergoing a radical transformation. OpenAI's recent unveiling of GPT-4 represents a quantum leap, accelerating the shift from static dashboards to dynamic, AI-powered decision-making. This revolution promises to reshape entire industries, and companies that fail to adapt risk getting left behind.

The New Drivers of Success

In this AI-driven future, a company's success will hinge on two critical factors:

1. Data Volume and Insights: The more high-quality data a company can access and analyze, the better. Data is the fuel that powers intelligent decision-making.
2. AI and Machine Learning Expertise: Companies that can leverage AI and machine learning to extract actionable insights from their data will gain a significant competitive edge.

Data is the new (may some say old) oil - the raw fuel that will power automated decision-making systems. For years, the data teams have struggled to make dashboards a useful tool for decision-makers, but everyone in this ecosystem invested way too much effort in manual work and not enough in simplifying the way decisions are being made.

The companies that can amass the largest data assortment and wield state-of-the-art ML, not even AI, to extract maximum value from that data will gain an immense competitive edge. Collecting large datasets is just the first step; the real differentiator will be the computing power and technical capabilities required to transform that data into meaningful decisions rapidly and at scale.

We are looking at a future where the ability of the company and the data team to build a model and deploy it will determine the success of converting traffic into revenues, and the expectations from the data team are that this time will be shortened.

Rethinking Data Governance

As AI takes center stage, data governance frameworks will need to evolve beyond their current scope of ensuring data quality for dashboards and reports. The new priority will be enabling the secure storage, reliability, and stability of data pipelines to fuel AI and machine learning models effectively.

Rather than excessive manual processes and validations, governance efforts should focus on bringing raw data into centralized data warehouses/lakes while maintaining its usability and fitness for AI model consumption. This involves robust metadata tagging, data lineage tracking, and observability to detect anomalies that could derail models.

One of the biggest challenges the Governance team tries to solve is the data mesh decentralization methodology, which moved the responsibility for the data from the centralized BI team to the producers of the data. This is not a wrong thing, but the Governance team needs now to ensure that this team knows what data they need to produce, they know how to produce it and correct errors in it, and maybe become the data product managers for those teams who do not have someone in this role today.

While governance gives the work frames of the data contract and the ownership clearance, they also need to create centralized tools to collect the data, ingest, and validate their quality before they arrive in use. They need to create centralized tools to document what data arrives, where it’s being used, and what it’s impact, and I will go one step forward, they will have also to set what is the ROI of the data and make it visible for the organization.

Redefining Growth Targets

To thrive in this new model, companies must redefine how they set and measure growth targets. Success can no longer be judged solely by conventional metrics like website traffic, conversion rates, or revenue. Instead, the growth levers of the future are:

• The volume of data being utilized and its associated costs
• Performance and accuracy of AI/ML models in production
• The extent to which manual processes are automated via these models
• Data-Driven ROI the return on investment from data and AI initiatives.pen_spark

Companies should establish clear numerical targets for transforming human-driven work to AI-driven automation while quantifying the ROI from their data and model investments.

I am not calling here to kill jobs, but yes, if a computer can do a job better and faster, let it do it. In the beginning, it will use the multi-arm-bendate and at some point, you will still need to have the human experiment out of the box ideas or even be there to disrupt the system

The Evolving Data Team

The shift towards AI/ML-powered decision-making means we won't see large data engineering and analyst teams anymore; the focus should no longer be on building dashboards, analysis, and reports. Instead, data teams should shrink into lean "mechanic" crews tasked with ensuring all components of the AI engine are running smoothly. Their responsibilities will include ingesting new data sources, monitoring model performance, tuning algorithms, and maintaining system uptime. Just like the transition to self-driving cars, humans will still be indispensable - but in more specialized maintenance and oversight roles rather than operating the vehicle directly.

This democratization of data through intelligent models means that managers can gain insights in real-time without having to rely on data analytics teams.

The rise of AI doesn't mean the end of data teams. Instead, their roles will transform. Data teams will become lean "pit crews" focused on:

• Data Acquisition: Ensuring a steady flow of new data sources for models.
• Model Monitoring: Continuously tracking and evaluating model performance.
• Algorithm Tuning: Fine-tuning algorithms to optimize model accuracy.
• System Maintenance: Ensuring all components of the AI engine run smoothly.

The Path Forward

If what I describe seems like a distant futuristic vision, let me share a real-world precedent. In 2016, as part of Zalando's mobile team, we built one of the company's first centralized data warehouses to consolidate data from multiple sources. We used this data to create a performance dashboard and - crucially - a model to forecast ROI 360 days into the future for each user session.

This ROI projection became the north star for optimizing marketing spend across channels. Using the forecast we built machine to automate the campaign decisions from increasing budget, or reducing it, creating new creatives and uploading it to our ad partners, all in an automated way we supervise of the team on the operation and it’s success.

While groundbreaking at the time, I now realize that this was just an early glimpse of the AI-driven future. Today's models can go far beyond the simplistic forecasting we had, which was very manual-heavy, to enable truly automated decision-making with minimal human involvement.

The time is now for companies to start planning their AI data strategies or risk falling behind. The game has changed from investing in armies of analysts to maximizing insights from stale dashboards. Instead, the new imperative is optimizing every dollar invested into acquiring quality data and developing high-performance AI models that can autonomously turn that data into profitable actions. If the basic raw data is correct and consumable from the warehouse,

Data consumers won’t really need to know SQL or build tables. Tools like Copilot will be able to help them build the tables they need to consume the data instead of waiting for an analyst. Even Tableau online gives tools to self-create the data, and the only thing you need to ensure is that the data is structured in a normalized way so it can be ready and the user will be able to take it from there with native language without the need to understand SQL.

In this future, KPIs like "data ROI" - the return generated for each dollar spent on data acquisition, storage, modeling, and infrastructure - will become crucial indicators of success. CEOs must start viewing AI models not just as decision support tools but as core revenue generators that disrupt markets by massively reducing human labor costs.

Those who still view AI as an abstract future trend are at risk of being blindsided. Pioneers like OpenAI and Anthropic are giving the world a glimpse of the coming AI revolution that will upend entire industries. For companies, clinging to legacy manual processes and traditional data practices will increasingly become a vulnerability.

The victors of this new age will be the ones who can assemble market-leading data assets and develop AI/ML capabilities to extract maximum value from that data better than competitors. However, you need to remember that if you build on top of someone’s tool, you should build it in a way that allows you to move it from one tool to another without creating much damage for you and the business. Just like previous technological transformations, there will be winners and losers - and the defining factor will be which companies can most nimbly adapt their resources and culture to this AI data paradigm.

The AI reckoning is no longer a distant prospect - it's happening now. Companies must take decisive action to skill up their workforces, redefine processes, and place intelligent models at the heart of their businesses. The time to plan for this seismic shift is now before nimble startups and AI rookies run circles around cumbersome incumbents. Data may be the oil, but AI will be the combustion engine accelerating leaders...and leaving laggards choking on fumes of the past.

The AI revolution is here. Companies that can harness its power will be the ones disrupting markets, optimizing operations, and ultimately achieving long-term success. Want to learn more? I'm creating a whitepaper outlining three actionable steps companies can take to prepare their AI data strategy. Follow me here or on Substack to get early access to the latest news!