For more than a decade, cloud architectures have been built around a deliberate separation of storage and compute. Under this model, storage became a place to simply hold data while intelligence lived entirely in the compute tier.
This design worked well for traditional analytics jobs operating on structured, table-based data. These workloads are predictable, often run on a set schedule, and involve a smaller number of compute engines operating over the datasets. But as AI reshapes enterprise infrastructure and workload demands, shifting data processing toward massive volumes of unstructured data, this model is breaking down.
What was once an efficiency advantage is increasingly becoming a structural cost.
Why AI exposes the cost of separation
AI introduces fundamentally different demands than the analytics workloads businesses have grown accustomed to. Instead of tables and rows processed in batch jobs by an engine, modern AI pipelines now process large amounts of unstructured and multimodal data, while also generating large volumes of embeddings, vectors, and metadata. At the same time, processing is increasingly continuous, with many compute engines touching the same data repeatedly—each pulling the data out of storage and reshaping it for its own needs.
The result isn’t just more data movement between storage and compute, but more redundant work. The same dataset might be read from storage, transformed for model training, then read again and reshaped for inference, and again for testing and validation—each time incurring the full cost of data transfer and transformation. Given this, it’s no surprise that data scientists spend up to 80% of their time just on data preparation and wrangling, rather than building models or improving performance.
While these inefficiencies can be easy to overlook at a smaller scale, they quickly become a primary economic constraint as AI workloads grow, translating not only into wasted hours but real infrastructure cost. For example, 93% of organizations today say their GPUs are underutilized. With top-shelf GPUs costing several dollars per hour across major cloud platforms, this underutilization can quickly compound into tens of millions of dollars of paid-for compute going to waste. As GPUs increasingly dominate infrastructure budgets, architectures that leave them waiting on I/O become increasingly difficult to justify.
From passive storage to smart storage
The inefficiencies exposed by AI workloads point to a fundamental shift in how storage and compute must interact. Storage can no longer exist solely as a passive system of record. To support modern AI workloads efficiently and get the most value out of the data that companies have at their disposal, compute must move closer to where data already lives.
Industry economics make this clear. A terabyte of data sitting in traditional storage is largely a cost center. When that same data is moved into a platform with an integrated compute layer, its economic value increases by multiples. The data itself hasn’t changed; the only difference is the presence of compute that can transform that data and serve it in useful forms.
Rather than continuing to move data to capture that value, the answer is to bring compute to the data. Data preparation should happen once, where the data lives, and be reused across pipelines. Under this model, storage becomes an active layer where data is transformed, organized, and served in forms optimized for downstream systems.
This shift changes both performance and economics. Pipelines move faster because data is pre-prepared. Hardware stays more productive because GPUs spend less time waiting on redundant I/O. The costs of repeated data preparation begin to disappear.
Under this new model, “smart storage” changes data from something that is merely stored to a resource that is continuously understood, enriched, and made ready for use across AI systems. Rather than leaving raw data locked in passive repositories and relying on external pipelines to interpret it, smart storage applies compute directly within the data layer to generate persistent transformations, metadata, and optimized representations as data arrives.
By preparing data once and reusing it across workflows, organizations allow storage to become an active platform instead of a bottleneck. Without this shift, organizations remain trapped in cycles of redundant data processing, constant reshaping, and compounding infrastructure cost.
Preparing for AI-era infrastructure
The cloud’s separation of storage and compute was the right architectural decision for its time. But AI workloads have fundamentally changed the economics of data and exposed the limits of this approach—a constraint I’ve watched kill numerous enterprise AI initiatives, and a core reason I founded DataPelago.
While the industry has begun focusing on accelerating individual steps in the data pipeline, efficiency is no longer determined by squeezing marginal gains from existing architectures. It is now determined by building new architectures that make data usable without repeated preparation, excessive movement, or wasted compute. As AI’s demands continue to crystallize, it is becoming increasingly clear that the next generation of infrastructure will be defined by how intelligently storage and compute are brought together.
The companies that succeed will be the ones that make smart storage a foundation of their AI strategy.
—
New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to doug_dineley@foundryco.com.
