The algorithm rejected the loan application, even though the customer met all the criteria. The system misjudged the risk of the transaction and blocked an important business partner. The AI model recommended a decision that cost the company millions of zlotys. In such situations, the board of directors does not usually ask why the artificial intelligence made a mistake. It asks a much simpler question: who is responsible for this decision?
This question will come up more and more often. Companies are implementing AI in sales, finance, customer service, HR and logistics, whilst at the same time only just learning to manage accountability for decisions made with the involvement of algorithms. The paradox is that the greatest risk today is not the technology itself. The greatest risk is the lack of a clear decision-maker.
Artificial intelligence bears no responsibility. It has no legal personality, cannot be held accountable in court, and cannot explain its actions to a supervisory board. The organisation is accountable. It was the organisation that decided to implement the system, defined its role in the business process, and approved the way it is used.
This is an important distinction, because in the debate on AI we often focus on the technology’s capabilities, forgetting that responsibility for its use remains with people.
The problem is that, in many organisations, responsibility for AI is fragmented. The business defines the project’s objectives and expected outcomes. The IT department selects the solution and is responsible for its integration. The data team prepares the information on which the model operates. The supplier develops the technology, whilst the legal or compliance department assesses regulatory compliance. Everyone is responsible for a part of the process, but when an error occurs, it very quickly becomes apparent that no one feels they own the decision as a whole.
This is precisely where the problem of governance begins, rather than that of technology.
A wrong decision by an AI system is almost never the result of a single event. It is usually the consequence of many previous decisions. Perhaps the model was trained on out-of-date data. Perhaps the company defined the system’s operating rules incorrectly. Sometimes integration with other applications fails, and sometimes an employee thoughtlessly accepts a recommendation generated by the algorithm.
That is why it is increasingly said that AI does not make a single decision. It flows through the entire organisation. Every stage of this process can become a source of error and, at the same time, a point where responsibility lies.
This also explains why the AI Act is so often misinterpreted. Many managers expect the new regulations to clearly indicate who is responsible for the decisions made by artificial intelligence systems. However, this is not the case. The regulation sets out the obligations of providers and entities using AI, and imposes requirements regarding risk management, documentation, oversight and data quality, but it does not create a simple framework of liability for every business situation.
In other words, the law sets out a framework for action, but does not replace the board in organising accountability.
This means that every company must answer several fundamental questions for itself. Who approves the use of AI in a business process? Who monitors the model’s performance after implementation? Who has the right to suspend its operation if irregularities arise? And finally, who takes responsibility for the consequences of decisions affecting a customer or business partner?
In practice, the answer is very rarely: the supplier.
This is one of the most common myths surrounding artificial intelligence. The supplier is responsible for the quality of its own solution, compliance with regulatory requirements and contractual obligations. However, it is not responsible for the way in which an organisation uses its product. If a company implements a model in the wrong process, uses incorrect data, ignores warning signs or fails to provide adequate human oversight, the responsibility remains with the company.
That is why not only the technologies themselves, but also the principles of cooperation with suppliers, are becoming increasingly important. Contracts for AI solutions are no longer standard software purchase agreements. Increasingly, they cover issues such as data quality, liability for model errors, rules governing the use of customer data, incident response procedures and the scope of post-implementation support. It is precisely in these areas that many disputes are resolved in practice, even before the first problem arises.
The most mature organisations, however, take a broader view. They do not focus on apportioning blame after an error has occurred, but rather define responsibilities even before the system goes live. They identify the business process owner, assign roles to individual teams, establish procedures for monitoring model quality, and clearly define the point at which a human should take control of the decision.
This approach is increasingly referred to as AI governance. It is not just another layer of bureaucracy, nor is it merely a regulatory requirement. It is a way of managing technology that is increasingly influencing key business decisions.
In the coming years, it will not only be the companies that implement artificial intelligence the quickest that gain the upper hand. It will be equally important which organisations are quickest to establish transparent rules regarding accountability for decisions made with the involvement of AI.
Because in the world of AI, the most important question is: has the organisation established in advance who will take responsibility for an AI error?
