Imagine you are driving a Formula 1 car, you are pushing the limits of speed, the mechanics are grabbing their heads in the corners and you have just gained an advantage over your biggest rival. At the same moment, however, you let go of the steering wheel, open the window and shout through the megaphone to the judges and the public: “Stop this race, because we’re all about to die!”.
This is exactly the attitude of Anthropic today. One of the most important players in the artificial intelligence market, the creator of the Claude family of models, recently published a manifesto on its blog that has caused a powerful resonance in the technology world. The company is calling for a temporary delay or even a complete halt to the development of the most advanced AI systems. The reason? Technology has begun to evolve faster than society and regulatory structures can digest.
However, we are not dealing here with the voice of external sceptics or philosophers. It is the alarm raised by the people who are building these machines. And that is what makes this message so striking.
Controlled hypocrisy or belated responsibility?
From the perspective of a business market observer, Anthropic’s appeal carries a huge dose of irony. The company is, after all, in a permanent clinch with OpenAI and Google. It is a classic arms race, with capital and investor attention flowing only to those who deliver models that are ‘faster, bigger and smarter’.
What’s more, Anthropic is not slowing down. Recently, the Claude Mythos model hit the market, causing a stir at the highest levels of government. The system proved so effective at autonomously detecting ICT vulnerabilities that the US government had to step in, directly interfering with the technology’s access policy. When code becomes a potential cyberweapon, the jokes stop.
Anthropic co-founder Jack Clark and Marina Favaro, principal researcher at The Anthropic Institute, write plainly:
“How we secure, monitor and manage artificial intelligence systems is becoming increasingly important.”
This is a partial mea culpa for the pace they have self-imposed. In just five years, the industry has gone from simple text generators to autonomous software-writing agents. The question is: is this sudden bump in alarmist tones pure responsibility, or an attempt to gain time at a time when competitors are treading on their toes?
Factory without people: Eight times more code, zero programmers
The strongest point of Anthropic’s manifesto, however, is not general warnings, but hard data from their own backyard. The company has let out the secret and shown what the day-to-day work of developing artificial intelligence looks like. The conclusions are sobering.
- 80% of the internal code at Anthropic in the last month has been written by the Claude Code system.
- The company’s engineers admit on the blog that they haven’t written code themselves in months – they have merely become curators and reviewers of the work done by the machine.
- Productivity, as measured by the number of lines of code per employee, has increased eightfold (8x) since the beginning of the year.
For every manager and chief technology officer (CTO), these figures are the financial holy grail. They represent a giant leap in productivity while radically cutting operating costs and the time it takes to implement new features.
However, this stick has two ends. As AI developers optimise their own product with the same product, the development loop begins to tighten dangerously. The developer’s job ceases to be about creating the architecture and begins to resemble merely supervising a production line that is moving ever faster.
From assistant to architect: the self-improvement phase is coming
At Computex in Taiwan, one of the most talked-about topics was the evolution of AI from ‘assistants’ to ‘autonomous agents’. This is a fundamental paradigm shift. Today’s systems no longer wait passively for a single prompt from a human. They can define tasks themselves, divide them into sub-projects and even control and correct each other’s work.
The next step on this evolutionary ladder is what scares Anthropic’s leaders the most. It concerns the point at which artificial intelligence systems will be able to independently develop and improve their own subsystems and underlying architectures (referred to as ‘sinsectors’).
Until six months ago, the code generated by artificial intelligence required deep revision by experienced programmers (senior developers). However, this quality gap is disappearing before our eyes. Anthropic estimates that, later this year, autonomous agents will write code better, cleaner and safer than any human.
And this is where the logic trap comes in. If we get to the point where AI models are the only instances capable of evaluating and improving the code of the next generation of models, humans will be pushed out of the process altogether. We will not be able to control improvements because we will not understand them. Then, as Jack Clark warns, “humans will surely lose the reins”.
Geopolitical and business stalemate
Does Anthropic’s call for a work stoppage stand any chance of being implemented? The short answer is no. In the current market and geopolitical realities, this is a completely utopian proposition.
We live in a reality where AI technology has become the equivalent of nuclear power during the Cold War. If Anthropic were to unilaterally slow down, their place would be taken within weeks by OpenAI, Google, Meta or the booming labs from China. There is too much money at stake. Venture capital funds have pumped hundreds of billions of dollars into the generative AI sector and expect hard returns on investment, not philosophical treatises on security.
On the other hand, we have regulators. The fact that US government agencies have begun to control access to the Claude Mythos model shows that politicians are beginning to recognise the threat. States, however, do not see the problem in ‘humanity losing control’, but in the risk of a hostile geopolitical camp taking control. This in turn fuels the pressure to develop even faster, just to be first.
The real spectre of institutional inertia
The Anthropic Manifesto should be read not as a prediction of a revolt of the machines in the style of Hollywood sci-fi films, but as a sober diagnosis of institutional helplessness. The real problem is not that AI will gain consciousness and turn against us. The threat is our own human organisational inertia.
We create systems that optimise at an exponential rate, while our laws, education systems, labour markets and security procedures evolve at a linear rate. The gap between these two worlds grows wider every day.
Moving to a level where 80% of technology is created without human hands is a business masterstroke, but also a powerful test for market stability. Unless Silicon Valley and global decision-makers find a way to develop a common, hard framework for overseeing autonomous code, we are in for a hard landing. And this in a world where humans – from their position as proud creators – can very quickly fall into the role of passive and somewhat confused passengers.
