Over the past several months, I’ve found myself engaged in some rather spirited conversations about artificial intelligence, data centres, and the extraordinary amount of capital being invested in GPU infrastructure.
The prevailing narrative is familiar. Artificial intelligence requires enormous computing power. GPUs are the engines driving the AI revolution. And the companies building massive GPU data centres are positioning themselves at the centre of what could become one of the largest technology transformations in history.
I don’t disagree with any of that.
GPUs are remarkable pieces of technology. Without them, many of the breakthroughs we’ve witnessed in modern artificial intelligence would not have been possible.
But I keep coming back to a question that I believe deserves considerably more attention: are we so focused on the infrastructure required to create intelligence that we’re overlooking the infrastructure required to put that intelligence to work?
I believe that distinction could become increasingly important as AI moves beyond research laboratories and technology companies and becomes embedded in the daily operations of millions of ordinary businesses and it leads me to a somewhat contrarian view.
The CPU, a technology piece that rarely receives the same headlines as the GPU, may become one of the most strategically important and underappreciated components of the next phase of the AI revolution.
But there’s an even bigger idea behind that argument. What if the future of AI isn’t simply about building increasingly powerful models? What if it’s also about making intelligence smaller, more efficient, more specialized, and ultimately inexpensive enough to become a routine component of everyday business operations?
Technology Has a History of Getting Smaller, Not Just Bigger
Early computers occupied entire rooms. They required enormous amounts of electricity, specialized facilities, and teams of people to operate them. Over time, computers became smaller, faster, more efficient, and dramatically less expensive. Eventually, computing power moved from mainframes to desktops, laptops, smartphones, and embedded devices.
The same pattern is observed across countless technologies.
Televisions became thinner and more energy-efficient. Storage devices that once occupied entire cabinets can now hold enormous amounts of information on something smaller than a fingernail.
As technology becomes more powerful, it is also made more efficient, more accessible, and increasingly specialized. So why should artificial intelligence be any different? So why should artificial intelligence be any different?
Much of the industry’s attention today is directed toward building ever-larger language models with increasingly sophisticated reasoning capabilities and there will undoubtedly continue to be a need for these powerful models.
But I question whether the majority of everyday business applications will ultimately require that level of intelligence.
Consider a small business that wants an AI agent to reconcile invoices, manage inventory, process customer orders, or schedule deliveries.Does that business really need one of the world’s most sophisticated AI models to perform those tasks? In many cases, probably not.
A smaller, specialized model is more than capable of handling the intelligence required, and here’s where the discussion becomes particularly interesting.
Once an AI agent understands what needs to be done, how much artificial intelligence does it actually need to complete the task?
An AI Agent Doesn’t Need to Think Every Time It Acts
Picture a small manufacturing company that deploys an AI agent to manage incoming supplier invoices.The agent receives an invoice, extracts the relevant information, identifies the supplier, compares the invoice against an existing purchase order, verifies pricing, checks for discrepancies, and updates the accounting system.
At certain points in this process, artificial intelligence may be extremely useful.
An AI model might interpret an unfamiliar document, identify relevant information, or determine how to handle an unusual exception. But does every subsequent action require another call to a large language model? Of course not.
Once the information has been interpreted, much of the remaining work can be handled by conventional software – database queries, numerical comparison, business rules, API calls, validation, record updates, transaction processing, etc. In fact, using a sophisticated language model to repeatedly execute predictable, deterministic tasks could be unnecessarily expensive and, in some cases, less reliable than conventional software.
This leads to an architecture that I believe will become increasingly common: an AI agent uses intelligence when intelligence is required.
It uses conventional computing when conventional computing is sufficient. It escalates to more sophisticated models, or human intervention, when it encounters something outside its normal operating parameters.
The future of agentic AI may involve relatively small amounts of sophisticated reasoning surrounded by enormous volumes of traditional computing activity. If that proves correct, the infrastructure implications could be substantial.
The SME Market Could Fundamentally Change the Equation
Much of today’s AI infrastructure conversation revolves around hyperscalers, frontier models, and enormous GPU clusters but I believe one of the largest long-term opportunities for artificial intelligence lies within small and medium-sized enterprises.
Most SMEs have no interest in building large language models. They aren’t planning to hire teams of AI researchers or operate their own GPU data centres. They want practical solutions to everyday business problems. They want to automate repetitive processes, improve productivity, reduce administrative overhead, and make better use of the information already contained within their businesses.
Think about the thousands of routine activities that take place inside an ordinary company: processing invoices, updating customer records, reconciling accounts, monitoring inventory, preparing reports, scheduling employees, managing purchase orders, responding to routine customer inquiries, etc. Many of these tasks require relatively modest computing resources.
Now imagine millions of businesses deploying dozens, and eventually hundreds, of specialized AI agents to handle this work. A single agent might execute hundreds or thousands of operations during a typical workday. Multiply that activity across millions of businesses, and the aggregate computing demand becomes enormous.
Crucially, many of those operations aren’t necessarily GPU-intensive. They’re conventional computing activities involving CPUs, memory, databases, storage, networking, and software execution.
This is why I believe the widespread adoption of AI agents could fundamentally change how we think about AI infrastructure. The next major wave of demand may not be driven exclusively by the computational requirements of increasingly powerful models. It may also be driven by the enormous volume of relatively simple tasks that AI agents are asked to perform.
The Intelligence May Become Cheaper. The Work Still Needs to Get Done.
There’s another dimension to this discussion that I find particularly compelling. As artificial intelligence evolves, we’re already seeing improvements in model efficiency through techniques such as distillation, quantization, and specialized model architectures. These developments make it possible to run increasingly capable models with fewer computational resources.
I suspect this trend will continue. Tasks that require large, expensive models today may eventually be handled by much smaller models running on less expensive infrastructure. Some inference workloads may move to CPUs, specialized AI processors, or edge devices. And in many business workflows, AI may only be required occasionally, rather than continuously throughout the entire process.
But here’s the important distinction: even if the cost of intelligence declines dramatically, the work that intelligence enables still has to be done.
Customer records still need to be retrieved. Databases still need to be updated. Transactions still need to be processed. Applications still need to communicate with one another. Business rules still need to be enforced.
And all of that runs on computing infrastructure. Cheaper doesn’t eliminate the need for computation. It may actually create demand for much more of it.
We’ve seen this pattern again and again in the history of technology. When computing becomes cheaper and more accessible, we tend to find exponentially more ways to use it. The same could prove true for AI. As the cost of deploying intelligent agents declines, businesses may automate processes that were previously too expensive or complicated to justify. That could create an entirely new category of computing demand.
Data: The Other Side of the AI Infrastructure Story
Another element that often gets overlooked in the GPU discussion is data.
An AI agent is only as useful as the information it can access and the actions it is authorized to perform. In most businesses, that information is typically distributed across accounting systems, customer relationship management platforms, inventory databases, document repositories, and numerous other applications. AI agents need secure, reliable ways to retrieve and process this information. Sometimes they need to transform it. Sometimes they need to combine information from multiple sources. Sometimes they need to store new information or update existing records.
Not every agent needs to download or permanently store the data it uses. In many cases, accessing existing systems on demand will be more efficient. Nevertheless, large-scale agent adoption could generate substantial demand for data processing, database operations, storage, memory, networking, and the CPU resources required to coordinate these activities.
And as businesses become increasingly dependent on AI agents, reliability, security, and observability become essential. It’s one thing for an AI model to generate an impressive response. It’s quite another for an autonomous agent to reliably execute a business-critical process thousands of times without making an expensive mistake.
That brings me to what I believe may be one of the most important developments in the next phase of AI.
What If the Real Value Shifts Toward Execution?
Today, much of the perceived value in artificial intelligence is concentrated in the models themselves, a nd understandably so. The capabilities are extraordinary.
But as models become more efficient, less expensive, and increasingly interchangeable for certain applications, I wonder whether the economic value will begin shifting toward the infrastructure that makes those models useful.
Consider the difference between having access to an intelligent model and having an AI agent that can reliably operate within a business. The model provides reasoning. But something still needs to connect that reasoning to business data. Something needs to determine which tools the agent can access. Something needs to execute the required actions. Something needs to verify the results, manage permissions, monitor performance, and ensure that the agent behaves reliably.
That operational layer is where AI becomes more than an interesting technology. It’s where AI becomes a productive business tool.
And I believe the platforms that successfully manage this transition could become increasingly important from AI as an interesting technology. The companies that build it well stand to become increasingly vital.
This is the problem we’ve put front and centre at Railtown AI, as we build the technology to develop agents, connect them to business data, deploy them into production, and monitor how they perform. Our experience keeps reinforcing a simple lesson: building an intelligent agent is only part of the challenge. Making that agent useful, reliable, secure, and economically viable inside a real business is an entirely different undertaking.
And much of the infrastructure required to accomplish that has very little to do with training enormous AI models.
The Future Isn’t GPU vs CPU
Let me be clear. I’m not suggesting that GPUs are going away, or that demand for GPU computing will decline. If anything, it will keep growing as AI adoption expands. More agents mean more total inference requests, even if each request needs less computation.
But I believe it’s a mistake to assume that the future of AI infrastructure will be defined exclusively by GPU capacity.
GPUs will remain essential for training and many forms of inference.CPUs will remain essential for application execution, orchestration, data processing, and the operational activities surrounding AI workloads.Specialized processors will increasingly handle workloads for which they are best suited. Memory, storage, and networking will continue to play critical roles.
The most effective AI infrastructure will likely be heterogeneous, combining different types of computing resources to deliver the best performance, reliability, and economics for each workload. And this is where I think the data centre conversation needs to evolve.
Rather than focusing almost exclusively on how many GPUs a facility can accommodate, perhaps we should be asking how efficiently that infrastructure can support the complete lifecycle of millions of AI-powered applications.
The Future Isn’t GPU vs CPU
I often think about how technology markets tend to concentrate enormous attention on the component driving the current wave of innovation. Today, that component is unquestionably the GPU, and deservedly so.
But technological revolutions rarely stop at the breakthrough that made them possible, they evolve, become more efficient, more accessible. And eventually, they become so deeply embedded in everyday life that we stop thinking about the underlying technology altogether.
Most people don’t spend much time thinking about the processors, databases, storage systems, and networking infrastructure required to operate the applications on their smartphones. They simply expect apps to work.
I believe artificial intelligence is heading in a similar direction. There may come a time when businesses no longer think of themselves as adopting AI. They will simply operate with dozens or hundreds of intelligent agents quietly performing routine tasks in the background. Some of those agents will rely on sophisticated language models. Others will use small, specialized models and many of their actions will be executed through conventional software running on conventional computing infrastructure.
If that future materializes, the demand for CPUs, memory, storage, data infrastructure, and orchestration could become far more significant than today’s GPU-centric conversation suggests.
The most interesting question may not be whether CPUs will become more important than GPUs. It may be whether we’re underestimating the total computing infrastructure required when intelligence becomes inexpensive enough to be used everywhere.
We’ve spent the past several years talking about the computing power required to create intelligence. I believe the next great infrastructure conversation will be about the computing power required to put that intelligence to work.
And I suspect the CPU will have a much bigger role in that conversation than many people currently appreciate.
