Anthropic's IPO caps an explosive growth period for the AI startup, filled with contradictions
A mere five years before breaking the record for the biggest public stock offering ever, Anthropic began as a Silicon Valley startup when its founders broke away from OpenAI with their own ideas about how to develop artificial intelligence
A mere five years before breaking the record for the biggest public stock offering ever, Anthropic began as a Silicon Valley startup when its founders broke away from OpenAI with their own ideas about how to develop artificial intelligence.
Today, the company that dedicated itself to prioritizing safety, building the Claude chatbot along the way, has been one of the loudest voices raising warnings about how the same technology could bring catastrophe.
It's one of the contradictions that mark the brief, colorful history of Anthropic, which emerged Thursday from its first day of trading on Wall Street with a market value of $1.75 trillion.
The company, which still pledges to prioritize humanity's benefit, now faces pressure from shareholders to wring profits from the technology its most senior leaders have described as posing existential threats.
Anthropic CEO Dario Amodei described the push and pull between safety and AI advancement as one of the “central tensions at the heart of the company” on Thursday.
In an interview with CNBC, he described the rapid pace of AI advancements — due in large part to advanced models helping to build the next, improved version of themselves — as akin to building something that lets a driver go 200 miles per hour when they had previously been driving at 60. His company, he said, is working to develop it safely.
“What we’ve done with pacing is essentially said, ‘Look, you know, we could drive at 200 miles an hour. Maybe let’s only drive at 90 miles an hour, spend time figuring out how to do that safely,’" he said. "And the rest of that surplus, we spend building the model in the right way.”
Warning of potential calamity, the company forges ahead
The company's seven founders — Amodei, 43, his sister and Anthropic’s president Daniela Amodei, 39, and five other former OpenAI employees — left the rival company in pursuit of building one that more clearly prioritized safety in artificial intelligence development.
Anthropic publicly has been committed to safety and transparency around their AI models, but insiders have sounded alarm bells in recent months.
An Anthropic researcher resigned from the company in September, criticizing the safety practices of both Anthropic and OpenAI. Jacob Coxon, whose message spurred industry-wide conversations about the speed of AI's growth, wrote that both Anthropic and OpenAI are “gambling with our lives.” He warned that some working on AI development believe it could threaten human life by the end of the decade.
Dario Amodei responded with an essay calling for an industry-wide slowdown on the technology’s development. He enumerated detailed plans for slowing development and ushering in coordination across different companies and countries, and championed the idea of having independent evaluators embedded within Anthropic and other labs.
But the results from at least one of those independent evaluations underscored Coxon's concerns. A recent report from the nonprofit Model Evaluation and Threat Research found Anthropic is developing its systems around twice as quickly as it was a year ago, largely because of the help of AI itself.
Amodei said Thursday in comments at the Nasdaq opening bell that the company's “most difficult safety work lies ahead of us.”
“And to state it unequivocally, we will pace the training and deployment of AI systems, taking as long as is necessary to have confidence that they are safe and will remain under human control,” he said.
A company pledging to benefit humanity has enriched its founders
Anthropic is a public benefit corporation, meaning it has a legal obligation to balance shareholder interests with the public good. But these types of companies are rarely public, and of the few that are, they are significantly smaller than Anthropic, so the company is entering uncharted territory.
The company has said it is working towards a future where “AI’s benefits are shared by everyone," but the initial public offering has created a concentration of extreme wealth for its founders, employees and other investors.
Before the IPO, the two Amodei siblings each had a net worth of just over $14 billion, according to Forbes. By Thursday afternoon, Dario Amodei's net worth was $22.7 billion and Daniela Amodei's was $22.8 billion.
The Amodeis and the other co-founders have said they are committed to pledging 80% of their equity to philanthropy, although that commitment is voluntary. The company has an equity donation match program that allows employees to pledge equity, which Anthropic matches. Anthropic has also pledged at least 5% of its stock to a “public benefit” fund.
Speaking to CNBC Thursday, Daniela Amodei said the concept of a public benefit corporation “spoke” to the co-founders when they were initially creating the company.
“We’re really trying to do something that we do think can have this incredible economic value and impact, but we want it to be baked into essentially our structure that we also have to balance that against building things that are good for people,” she said.
Anthropic offers itself as a solution for AI doomsday fears
In a filing with the Securities and Exchange Commission last month, Anthropic devoted an unusually lengthy 93 pages to the risk factors they must spell out for investors. Among the concerns were that advanced AI systems could pose “catastrophic or existential risks to humanity.”
Some argue Anthropic has outlined the ways its technology and that of its competitors could have devastating consequences in the name of sheer transparency. But critics have questioned the emphasis the company has placed on these dystopian-like potentials while positioning itself as the logical solution given its safety-minded focus.
A “Saturday Night Live” sketch in September made this contradiction a punchline. Cast member Jane Wickline played Dario Amodei, saying: “Sam Altman, myself, all the major AI players, we have talked about this extensively, and we are all on the same page here: We do not condone what we are doing.”
The faux interview on "SNL" concluded with a line that seems to have resonated with skeptics, showing up on a protest sign outside of Anthropic's San Francisco headquarters on Thursday: “I urge you to urge me to stop.”
___
Associated Press business writer Wyatte Grantham-Philips contributed.