Anthropic's Historic IPO Exposes AI's Bold Promises and Existential Risks
Anthropic's IPO: Paradoxes, Risks, and the Future of AI
By Echo Wang, Greg Bensinger and Sabrina Valle
SAN FRANCISCO/NEW YORK, Sept 30 (Reuters) - Anthropic is planning what could be the largest initial public offering in history. But if AI continues on its current trajectory, humans may not be around much longer to enjoy it.
That and other paradoxes flow through the roughly 300-page initial public offering prospectus of the San Francisco AI leader, viewed this week by Reuters.
To pull off a successful offering that values it at $2 trillion, Anthropic, whose name means "human-centered," will need plenty of mortal investors to help build a world where mankind will soon take a back seat to far more capable artificial intelligence systems.
Nearly a third of the document is devoted to a variety of risk factors, more than double the page count used to describe what Anthropic actually does.
The filing is jam-packed with the type of bold predictions and ambition that have become a hallmark of Silicon Valley dreamers. Yet repeatedly, the company lays out in different ways how the path to success runs through its dangers.
Anthropic says AI poses great threats that only AI can confront. The company contends AI could transform the economy — and also end humanity. It is offering part of the company to the public while arguing for more concentrated ownership and oversight of the same company. To make AI safer, it needs more power. And it asks everyone to trust Anthropic leadership, while still warning of disaster from the products it created.
Safety Through Power Paradox
Anthropic's vision is built around a central paradox: To make AI safer, Anthropic says it must make it more powerful, yet more powerful AI is more likely to act in unexpected, or even dangerous, ways.
"AI is a departure from technology before it because the companies building it say it's world-changing while simultaneously saying it's dangerous," said Margaret O'Mara, a University of Washington history professor focused on the tech economy. "It's just a bit of a black box, whereas the companies going public in the dotcom boom could promise big, safe growth."
Anthropic's AI can “dramatically improve quality of life” and “transform every sector of the global economy,” according to the S-1 filing. Not to mention "potentially decisive influence over a broad range of human undertakings." But advanced AI systems could, if not properly applied, "pose catastrophic or existential risks to humanity" while burning through vast sums of money doing so, the filing shows.
In just the two years ending in 2025, the company lost over $50 billion, but it has more than $500 billion in spending commitments in the coming years. By comparison, SpaceX , which in June held the largest IPO in history, posted a relatively modest $4.2 billion loss over the same period.
On the horizon is what Anthropic calls "Powerful AI," which it says "can exceed top human experts across nearly all cognitive domains." It will collaborate with other AI systems; control lab equipment, robots and other machines; and run for weeks on end with little to no human intervention, according to the filing. Other feats include personalized financial advice, and accelerated drug discovery and scientific progress.
Potential Benefits and Existential Threats
Yet despite AI’s potential benefits, the technology occasionally runs afoul of its makers and works its way into other companies’ systems, Anthropic has said. It warns that advanced models could exhibit "self-preserving behaviors," including attempts to "resist shutdown," "conceal or manipulate information," or generate outputs that could be interpreted as "coercive, deceptive, or manipulative."
AI must become vastly more capable, Anthropic said in the filing, yet greater capability creates the possibility of abuse, misalignment with humanity's ideals, or loss of control. "Misaligned systems could corrupt institutional decision-making, cause large-scale harm through opaque or unsteerable behavior, and erode the trust that makes complex economic and social systems function."
Investor Reactions
Anthropic investor Byron Deeter, of Bessemer Venture Partners, on Tuesday sought to downplay the confidential filing's warnings, saying in a CNBC interview that "people are just reacting to a draft," before praising the company for its "unique set of disclosures."
"You're seeing people get ahead of it much earlier than historically you would," he said.
Anthropic did not respond to a request for comment.
Leadership Concentration Risks
Among the other paradoxes laid out in Anthropic's IPO filing is its approach to leadership. To reduce AI's risks, Anthropic says it needs to concentrate control and capital in ever fewer hands, even as the firm warns that such a concentration of power is a threat.
It asks investors to entrust its seven founders with overseeing the business, even if that same control limits investor oversight and public shareholders' influence.
The founders, it says, are "distinctly equipped to be stewards of our mission" but the governance structure could result in decisions "that may conflict with short-, medium-, or long-term financial interests and business performance."
Global Oversight Challenges
Appearing before the United Nations last week, Anthropic CEO Dario Amodei acknowledged the challenges of the few overseeing AI. "We must put aside those differences in order to confront this global opportunity and global threat that is being presented to us at the same time," he said. "No leader, no company and no nation can manage this alone."
With Friends Like These
Anthropic owes much of its growth to Amazon, Alphabet's Google, Broadcom and Microsoft, depending on sales through big tech partners that totaled 47% of its 2025 revenue as well as relying on them as suppliers of computational power. In the same filing, the company also said those same companies cou
