Anthropic urges AI development pause

Anthropic, the creator of the Claude AI, has called on major artificial intelligence labs to consider a coordinated and verifiable pause in development. The company warns that rapid advances in AI, particularly the potential for 'recursive self-improvement' where AI systems improve themselves without human intervention, could lead to humans losing control. Anthropic's co-founder Jack Clark and Anthropic Institute lead Marina Favaro highlighted that AI's ability to complete tasks has been doubling every four months. They argue that a pause would allow society to manage the immense implications and for societal structures and alignment research to catch up. This call comes amidst Anthropic's confidential filing for a US initial public offering, valuing it at $965 billion, and its ongoing competition with rival OpenAI. Other AI developers like Alphabet, Meta Platforms, and Mistral have not yet responded to the call. The United States government has been slow to regulate AI, with a previous Trump administration executive order placing the onus on labs for voluntary cybersecurity testing. Researchers from the University of Toronto also recently demonstrated new AI 'worm' capabilities, adding to security concerns.