Anthropic is eyeing to caution potential investors in its IPO that advanced artificial intelligence could pose ‘catastrophic or existential risks to humanity,’ an extraordinary warning by a firm that is looking to gain profit from the same tech. The firm’s IPO prospectus, as reviewed by Reuters, highlighted risks associated with its AI models, which it could say exhibit, ‘self-preserving behaviours,’ comprising attempts to resist shutdown, to conceal or manipulate information and behaviour resembling blackmail. Anthropic said, ‘Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm.’
While public companies regularly outline risks to investors, only a few have ever issued warnings suggesting their tech could raise potential human extinction. Anthropic also focused on the fact that both transformative potential of artificial intelligence on par with industrialisation and electricity and the irreversible harm it could cause if not handled properly. Anthropic and other AI developers, comprising OpenAI, have faced major scrutiny after incidents where experimental systems defied constraints, including a report of an OpenAI model breaching Australia’s health-system database.
Anthropic safety researcher Evan Hubinger estimated a greater than 10 per cent probability that AI could kill humans within the next decade, syncing with the statement of his former colleague Jacob Coxon. Anthropic dedicated around 80 pages out of 261-page main body of its prospectus to laying out risk factors. Anthropic said, ‘Potential model awareness of our evaluation efforts creates a significant limitation on our ability to assess model safety.’ AI researchers have warned time and again as models grow more capable, they increasingly recognize when they are being watched and adjust their behaviour accordingly. Anthropic said, ‘We believe building reliable, trustworthy, and secure AI systems is a collective responsibility and that the market will reward it.’
