Companies are becoming increasingly aware of the cost of launching an AI pilot. They are far less likely to know how much it costs to run it in production.
At first glance, this difference seems technical. In practice, it is one of the most important financial decisions that boards of directors face today. A pilot project demonstrates that the model can answer a question, summarise a document, assist a programmer or speed up customer service. Production, however, reveals something far more challenging: whether it can be done on a daily basis, safely, predictably and in a way that makes economic sense.
That is where the true cost of AI begins.
Computing power remains the most visible cost item. GPUs, specialised cloud instances, inference fees, training time, token limits, storage and data transfer. These are elements that are easy to enter into a spreadsheet, as they have price lists, billing units and invoices. No wonder they quickly become the focus of discussions between IT and finance.
The problem is that AI infrastructure does not end with compute. Increasingly, it does not even begin there.
According to the International Energy Agency, data centres consumed around 415 TWh of electricity in 2024, accounting for approximately 1.5 per cent of global electricity demand. By 2030, this consumption could rise to around 945 TWh, with AI being one of the main drivers of this trend. The scale of investment also shows that we are no longer talking about a marginal cost for IT departments. Goldman Sachs estimates that between 2026 and 2031, global investment in AI infrastructure – including computing power, data centres and energy – could reach $7.6 trillion.
For an individual company, this does not, of course, mean it has to build its own data centre. It does, however, mean something else: the cost of AI increasingly depends on infrastructure that is not visible in a product demonstration. On data quality. On the stability of integration. On security. On monitoring. On the people who can keep the model running. On energy, power availability, networks and cloud architecture.
This is why pilot projects are often cheap, whilst production is surprisingly expensive.
In initial calculations, data is the first thing to be overlooked. Not the storage itself, but all the work required to turn it into useful fuel for AI. Data must be cleaned, combined, annotated, secured, versioned and maintained to a high standard. You need to know where it comes from, who has access to it, whether it can be used in a given process, and whether it misleads the model. Without this, even the best model remains an impressive tool built on shaky foundations.
The second hidden aspect is logs, monitoring and observability. Production AI cannot operate like a black box. A company should know who is using the model, what queries are being sent to the system, what responses are being generated, where errors occur, when quality drops, and whether the model is starting to behave differently than intended. This requires tools, data retention, procedures, alerts and people capable of interpreting the signals. In pilot projects, monitoring is often an afterthought. In production, it becomes a prerequisite for control.
The third cost is integration. AI rarely creates value in isolation. It must connect to a CRM, ERP, data warehouse, customer service system, document repository, employee app or development environment. Each such integration involves testing, access rights, data mapping, API maintenance, liability for errors and the risk of system failures at the interfaces. The model may be ready sooner than the organisation that is to adopt it.
The fourth element is security. Often postponed during the pilot phase, it no longer offers the same leeway once the system goes live. AI that has access to an organisation’s knowledge, customer documents, code, financial data or operational information requires access control, encryption, auditability, secret management, retention policies and protection against data breaches. Added to this is regulatory compliance, which in Europe will increasingly tie AI projects to responsibility for data, risk and the way the system is used.
The fifth, and most underestimated, cost is people. The model does not run on its own, even if it is often marketed that way. Someone has to prepare the data, design the process, maintain the environment, check the quality of responses, respond to incidents, update policies and explain to the business where AI actually helps and where it merely creates the illusion of automation. In this sense, AI does not eliminate operational costs. Rather, it shifts them elsewhere.
In the background, another dimension of the cost equation is growing: energy. AI is changing the nature of digital infrastructure, as it requires not only more computing power, but also a stable power supply, cooling and increasingly advanced data centre architecture. Research into the concentration of new data centres indicates that the heaviest loads are concentrated in North America, Western Europe and the Asia-Pacific region, which may increase local pressure on electricity grids. In Europe, scenario analyses show that, after 2030, the development of AI may make the location of infrastructure more dependent on the availability of stable power and the flexibility of the energy system than on the mere presence of green energy.
For management boards, the conclusion is simple but uncomfortable. AI cannot be accounted for solely as the cost of the cloud, licences or GPUs. It must be accounted for as the cost of a process that is intended to operate within the organisation on a permanent basis. CFOs and CIOs should discuss not only the monthly invoice from the technology provider, but also the cost of a single use case, a single model version, a thousand inferences, integration with a key system, maintaining data quality, meeting security requirements and managing risk.
Only then does AI cease to be a technological experiment and become an integral part of the operational model.
The biggest mistake companies make today is not investing in AI itself. The mistake is treating it as if it were an off-the-shelf tool. In reality, AI is closer to a system that needs to be fed with data, embedded in processes, secured, monitored and constantly refined.
A pilot project answers the question of whether AI works. Roll-out answers the question of how much it costs when it actually works.
And it is precisely this second answer that should be of most interest to the board. Because the cost of AI doesn’t stop at the GPU. It is not the model itself that costs money, but the entire system that enables the model to operate securely, stably and with real business value.
