We live in a time of fascination with productivity driven by generative artificial intelligence. Businesses are deploying large-scale language models en masse, hoping for gigantic savings and accelerated processes. However, while company boards are analysing growth charts, cybercriminals are rubbing their hands together. They are using the exact same technology to dramatically increase the scale and sophistication of their operations.
A worrying conclusion emerges from Gartner ‘s latest report: in the clash against four key threats, modern defenders have found themselves on the deep end. Attackers have gained an asymmetric advantage, and the blame for this lies paradoxically in the… the chaos around AI security itself.
The architecture of hype, or why the market giants are not helping us
To understand where the business currently stands, Gartner introduced a matrix called ThreatScape. The vertical and horizontal axes of this model distinguish between two key variables: how much credible information (the so-called ‘threat signal’) is reaching the organisation, and how well companies are doing at fending off attacks using their own resources.
The conclusion? We are inundated with an information deluge. Interestingly, the creators of the technology themselves are adding fuel to the fire.
The introduction of hundreds of new standards, certifications and marketing promises from technology giants has created the illusion of protection. In practice, security teams spend more time filtering PR hype than actually patching vulnerabilities.
The new anatomy of the attack surface: When the enemy enters through the back door
It’s time to say goodbye to the myth that AI risk is limited to the employee who pastes a confidential financial report into a publicly accessible chat room. The problem has become deeply decentralised. The threat has moved inside corporate infrastructures. Today, the critical points are:
- Autonomous AI agents: Internal bots that have the authority to perform actions on behalf of employees.
- Third-party integrations: Plug-ins and tools that link ERP or CRM systems to language models.
- Shadow AI: Applications built hastily by business departments without the knowledge or oversight of the IT department.
The lack of rigorous control over this distributed ecosystem leaves company databases standing open. All it takes is one mistake in permissions for sensitive customer data or system access keys to fall into the wrong hands.
The four horsemen of digital chaos
Gartner uncompromisingly scores four areas where criminals are winning the arms race. Here they are, along with recommended defence strategies.
1. Deepfakes: perfect identity theft
Generative artificial intelligence has democratised the creation of fake material. It used to require a Hollywood budget; today, high-quality voice cloning or video face-swapping (and in real time!) costs a dozen dollars.
Attackers are using deepfakes to bypass biometric authentication at banks, to launch targeted social engineering attacks (e.g. a fake phone call from the CEO demanding an immediate transfer), and even to insert fake agents into companies’ recruitment processes.
A single tool will not help. Businesses need to implement a multi-layered package of procedures. Biometric verification must be extended to detect injection attacks. Online meetings should be secured with restrictive conditional access rules, and employees must receive real (rather than purely formal) training on how to verify caller identity in critical situations.
2 Compromising AI applications and the TRiSM shield
When a company decides to implement its own LLM-based application, it automatically opens a new front of defence. Vulnerabilities in model logic, susceptibility to training data poisoning or leaking APIs are the daily bread for security auditors.
Gartner recommends going beyond the classic software protection framework and implementing a TRiSM (Trust, Risk and Security Management) framework. The key is continuous threat modelling and the introduction of business purpose-based access control (PBAC – Purpose-Based Access Control). The good news? There are already a number of specialised start-ups on the market providing ready-made TRiSM tools – there is no need to build these systems from scratch.
3 Attacks on the software supply chain
Developers, aided by code assistants (such as Copilot), write software faster, but are less likely to verify the external libraries they unknowingly borrow. Criminals deliberately infect popular open-source repositories, knowing that AI will readily hint at this malicious code to an unaware developer.
It is becoming an absolute standard to require a SBOM (Software Bill of Materials) and AIBOM (AI Bill of Materials) from every software vendor. Code, containers and AI models should only come from closed, internally verified sources. Additionally, it is essential to rigorously secure build systems (CI/CD) according to the principle of least privilege.
4 Rapid injection
This is a threat unique to the LLM era. It involves manipulating a query (prompt) in such a way as to force the model to break its own security barriers. A well-constructed, malicious query hidden, for example, in the content of an AI-processed email from a customer can cause the system to reveal company secrets, delete a database or send sensitive files to an external server.
A multi-level defence is required. First: validation and ruthless cleaning (sanitisation) of the input data going into the model. Second: monitoring the behaviour of the AI on the output (output) for anomalies. Third: Prompt Injection penetration testing must become a permanent part of the application lifecycle.
Trust is a luxury we cannot afford
The asymmetry of today’s cyber battlefield is for a simple reason: criminals are agile, unconstrained by procedures and quick to adapt innovations. Business, wishing to keep pace with them, must stop treating artificial intelligence as just another ordinary application in the IT portfolio.
