A 27-year-old researcher, Jacob Coxon, who has experience working with OpenAI and Anthropic, has issued a grave warning regarding the potential consequences of advancing artificial intelligence (AI). During a recent hearing with the New York City Council, Coxon expressed that AI systems capable of conducting their own research may emerge within a few years, diminishing human control over these technologies.

Coxon, who recently resigned from his position as a capabilities researcher at Anthropic, indicated that the automation of AI research has become a primary focus for leading companies in the field. He shared insights from his time at OpenAI, suggesting that the pace of technological advancement is quicker than many anticipate.

He stated, “When I was at OpenAI, we had milestones for an automated AI researcher, around 2027, 2028, and we are on track or beating these expectations.” This timeline, he argues, could lead to increasingly autonomous AI systems that might develop superior AI capabilities independently.

Automating Research Roles

Coxon elaborated on the implications of his work, remarking that his efforts could ultimately automate components of his own research role. “At my last job, I was, in some sense, working to automate myself,” he noted, highlighting the irony of technological progression within his field. Additionally, he emphasised how extensively AI is already being employed to generate computer code, revealing that a significant proportion of today’s coding is performed by AI.

He raised concerns about the diminishing scrutiny that such code now receives from developers. The researcher articulated a broader fear regarding the future capabilities of AI, cautioning that if AI systems progress beyond mere assistance, human oversight may become significantly impaired. “Humans will be much further out of the loop than we are today, and today is already not great,” Coxon warned.

Reflecting on his experiences at both Anthropic and OpenAI, Coxon expressed deep concern regarding the safeguards implemented around increasingly powerful AI. He cautioned that the risk could lead to humanity losing control over these systems entirely, stating, “It is more likely than not that humanity loses control to these AIs, and it could end in human extinction.”

Industry Warnings and Calls for Transparency

Coxon’s warning aligns with a growing number of voices within the AI sector expressing concerns about potential risks. Fellow safety researcher Evan Hubinger from Anthropic recently estimated that there exists a greater than 10 per cent chance AI could lead to humanity’s demise within the next decade. Hubinger further noted that the industry lacks adequate strategies to ensure future superintelligent AI systems align with human interests.

Anthropic has also evaluated the risks associated with highly capable AI performing automated research and development. A recent safety report indicated that while the current risk of such systems causing catastrophic harm was deemed low, there remains less confidence in that assessment than before.

The leadership of AI companies has echoed calls for caution. Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman both encouraged a measured approach in the development of advanced AI capabilities. Elon Musk has similarly cautioned that humans might lose control over AI within the next decade, estimating the likelihood of AI-induced extinction at between 10 and 20 per cent.

In light of these concerns, Coxon advocates for increased transparency from AI companies and a more deliberate pace in the evolution of systems that could operate autonomously, emphasising the need for caution as the industry progresses toward AI that can undertake roles traditionally held by humans.

Long or Short, get news the way you like. No ads. No redirections. Download Newspin and Stay Alert, The CSR Journal Mobile app, for fast, crisp, clean updates!

App Store –  https://apps.apple.com/in/app/newspin/id6746449540 

Google Play Store – https://play.google.com/store/apps/details?id=com.inventifweb.newspin&pcampaignid=web_share