Artificial intelligence company Anthropic has called for the world’s leading AI developers to establish a coordinated mechanism that would allow the temporary slowing or pausing of advanced AI development if safety risks begin to outpace society’s ability to manage them.

The company, known for developing the Claude chatbot, warned that the pace of AI progress is accelerating rapidly and could soon reach a point where systems are capable of improving themselves faster than humans can oversee or regulate them. In a blog post published on Thursday, Anthropic argued that humanity should have the option to deliberately slow AI advancement if necessary to ensure adequate safeguards are in place.

The proposal comes at a time when AI capabilities are advancing at an unprecedented rate, fuelling both optimism about transformative benefits and concerns over potential risks.

Concerns Over Recursive Self-Improvement

At the heart of Anthropic’s warning is the possibility of “recursive self-improvement”, a scenario in which an AI system becomes capable of designing and developing more advanced versions of itself without direct human intervention.

According to the company, current trends in AI development suggest that increasingly powerful models are becoming significantly faster at performing complex software engineering tasks, including coding and system design. Given sufficient computing resources, future AI systems could potentially contribute to the creation of their own successors.

Anthropic acknowledged that such a breakthrough could deliver major benefits across scientific research, healthcare, engineering and other sectors. However, it cautioned that self-improving systems could also increase the likelihood of humans losing meaningful control over AI behaviour and decision-making processes.

The company said any future slowdown mechanism would not be intended to halt innovation permanently. Instead, it would provide time for alignment research and societal institutions to catch up with technological progress. Alignment refers to efforts aimed at ensuring AI systems act in accordance with human values, intentions and safety requirements.

Anthropic’s proposal also highlights the challenge of international competition. The company argued that any pause would need to be globally coordinated and independently verifiable to ensure that competitors do not secretly continue development while others slow down.

Industry Divided On Governance Approach

The proposal has already exposed differing views within the AI industry regarding governance and oversight.

OpenAI, the creator of ChatGPT, offered a contrasting perspective in a report released this week. The company argued that decisions regarding the pace of AI development should ultimately be determined by democratically elected governments rather than individual corporations.

OpenAI said no single company, laboratory or special interest group should be responsible for setting the rules governing AI advancement. Instead, governments should establish the safeguards, accountability frameworks and regulatory structures needed to manage increasingly powerful technologies.

The debate reflects a broader struggle within the AI sector as companies race to develop more capable systems while policymakers attempt to craft appropriate regulations. Despite growing calls for oversight, regulatory efforts have lagged behind technological progress, particularly in the United States, where many of the world’s leading AI firms are headquartered.

Earlier this week, the administration of US President Donald Trump issued an executive order encouraging AI companies to voluntarily submit their most advanced models for government cybersecurity testing before public release. However, critics argue that voluntary measures may not be sufficient as AI capabilities continue to accelerate.

This is not the first time prominent figures have advocated a pause in AI development. In 2023, technology entrepreneur Elon Musk joined other industry leaders in supporting a proposal from the Future of Life Institute calling for a six-month halt in advanced AI development to allow additional safety measures to be implemented.

Cybersecurity Threats Add To Growing Alarm

Anthropic’s warning coincides with fresh concerns emerging from the cybersecurity community.

Researchers at the University of Toronto recently demonstrated how AI tools could be used to develop a new form of adaptive computer worm capable of modifying its attack strategies as it spreads across networks. The research showed how an AI-powered worm could move from device to device, identifying vulnerabilities and altering its behaviour to maximise effectiveness.

Lead researcher Nicolas Papernot warned that cyber threats are no longer limited to the most sophisticated AI systems. He noted that open-source AI models, which are widely accessible and inexpensive to modify, could also be weaponised by malicious actors.

Papernot said that in the past, attackers typically focused on high-value targets such as hospitals, banks, power grids and water treatment facilities. However, AI-driven cyber tools could dramatically reduce the cost and complexity of launching attacks, making even neglected personal devices potential entry points for larger operations.

The findings have intensified calls for closer cooperation between governments, academic researchers and technology companies to develop effective defences against AI-enabled cyber threats.

Anthropic has long positioned itself as a safety-focused AI company, frequently advocating stronger safeguards and responsible development practices. Yet the company’s latest proposal arrives amid intense commercial competition, with both Anthropic and OpenAI pursuing public listings that could value them among the most influential technology firms in the world.

As AI systems continue to grow more capable, the debate over whether humanity needs a global emergency brake on development is likely to become one of the defining technology policy discussions of the decade.

Long or Short, get news the way you like. No ads. No redirections. Download Newspin and Stay Alert, The CSR Journal Mobile app, for fast, crisp, clean updates!

App Store –  https://apps.apple.com/in/app/newspin/id6746449540 

Google Play Store – https://play.google.com/store/apps/details?id=com.inventifweb.newspin&pcampaignid=web_share