🎙️ Ben Mann - Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night
Benjamin Mann is a co-founder of Anthropic, an AI startup dedicated to building aligned, safety-first AI systems. Prior to Anthropic, Ben was one of the architects of GPT-3 at OpenAI. He left OpenAI driven by the mission to ensure that AI benefits humanity. In this episode, Ben opens up about the accelerating progress in AI and the urgent need to steer it responsibly. Below are the core themes of the podcast, including AI safety, the pace of AI progress, societal impacts, and personal insights, as drawn from the transcript.
🎧 Listen here.
1. The Genesis of Anthropic: Prioritizing Safety Over Speed
Benjamin Mann begins by recounting the origins of Anthropic, founded in 2020 after he and several colleagues left OpenAI. As one of the architects of GPT-3, Ben worked on both research and product aspects, including tech transfers to Microsoft. However, he grew frustrated with OpenAI’s internal dynamics. He recalls Sam Altman describing three competing “tribes”, safety, research, and startup, within the organization, saying, “Whenever I heard that, it just struck me as the wrong way to approach things.” Ben and his team, including leads from OpenAI’s safety groups, felt that safety was not the top priority, overshadowed by commercial and research goals.
This realization drove them to establish Anthropic, a company dedicated to building AI that is “helpful, harmless, and honest.” Ben emphasizes that Anthropic’s mission is to stay at the forefront of AI research while making safety the central focus.
He notes, “We felt like we wanted an organization where we could be on the frontier, we could be doing the fundamental research, but we could be prioritizing safety ahead of everything else.”
This section explores the tension between safety and progress at OpenAI, the founding vision of Anthropic, and the challenges of embedding safety into cutting-edge AI development.
2. The Talent War in AI: Mission-Driven Retention
The podcast delves into the competitive landscape for AI talent, highlighted by reports of Meta offering $100 million packages to researchers. Ben confirms these figures, stating, “I’m pretty sure it’s real,” and ties them to the immense value of AI expertise. He explains, “If we get a 1 to 10 or 5% efficiency bonus on our inference stack, that is worth an incredible amount of money.”
Yet, Anthropic has been less impacted by this talent war.
Ben attributes this to their mission-driven culture: “People here are so mission-oriented... My best case scenario at Meta is that we make money. And my best case scenario at Anthropic is we affect the future of humanity.”
This section examines the broader implications of the talent war, including how it reflects the industry’s $300 billion annual spending and the growing demand for skilled professionals. It also highlights Anthropic’s unique ability to retain talent by aligning employees with a purpose beyond financial gain, offering a contrast to profit-focused competitors.
3. The Exponential Pace of AI: Beyond the Plateau Narrative
Ben challenges the notion that AI progress is plateauing, arguing that it is accelerating. He points to the increasing frequency of model releases: “It used to be like once a year. And now... we’re seeing releases every month or three months.”
This acceleration stems from advancements in scaling laws and post-training techniques like reinforcement learning.
He explains, “If you look at the scaling laws, they’re continuing to hold true,” noting that the bottleneck is not intelligence saturation but the need for better benchmarks.
He also addresses why people misjudge this progress, saying, “People are really bad at modeling exponential progress... It looks flat and almost zero at the beginning, and then suddenly you hit the knee of the curve.”
This section unpacks the technical drivers of AI advancement, the role of scaling laws, and the perceptual gap that leads to underestimating the field’s trajectory.
4. Defining Transformative AI: The Economic Turing Test
Ben introduces the “economic Turing test” as a practical measure of transformative AI, avoiding the loaded term “AGI.” He describes it as, “If you contract an agent for a month or three months on a particular job, if you decide to hire that agent and it turns out to be a machine rather than a person, then it’s passed the economic Turing test for that role.”
He predicts that when AI passes this test for 50% of money-weighted jobs, it will signal a societal shift, potentially by 2027-2028, aligning with the AI 2027 report’s forecast of superintelligence.
This section elaborates on the test’s significance, Ben’s rationale for the timeline, based on trends in model training and data center scale, and the economic implications of reaching this threshold, such as a potential GDP growth rate exceeding 10% annually.
5. AI and the Future of Work: Transitioning to Abundance
The conversation explores AI’s impact on employment, with Ben referencing Dario Amodei’s prediction of 20% unemployment.
He offers a balanced view: “In customer service... 82% customer service resolution rates automatically without a human involved,” and in software engineering, “we write 10x more code or 20x more code” with AI tools like Claude Code. Yet, he acknowledges displacement risks, especially for lower-skill jobs, and envisions a post-singularity world where “capitalism will look [nothing] like it looks today.”
Ben stresses the need for society to manage this transition, saying, “It’s just something we as a society need to get ahead of and work on.”
This section discusses job augmentation versus elimination, the economic shifts AI may bring, and strategies for navigating the change.
6. AI Safety: Balancing Risks and Optimism
Safety is a cornerstone of Ben’s work, driven by his early exposure to Superintelligence by Nick Bostrom.
He estimates existential risk (X-risk) from AI at “somewhere between 0 and 10%,” but emphasizes its importance: “Even if there’s only a 1% chance that the next time you got in an airplane, you would die, you probably think twice.”
Anthropic’s approach includes constitutional AI, training models with principles like the UN Declaration of Human Rights, and transparency, such as publishing examples of AI misbehavior.
Ben notes, “Once we get to superintelligence, it will be too late to align the models,” underscoring the urgency of current efforts.
This section details Anthropic’s safety practices, the risks of advanced AI, and the role of policy in ensuring responsible development.
7. Claude’s Personality: Safety as a Feature
Anthropic’s focus on safety shaped Claude’s beloved personality. Ben explains, “One of the things that people really loved about [Opus 3] was the character and the personality. And that was directly a result of our alignment research.” By embedding values like helpfulness and honesty via constitutional AI, Claude handles refusals gracefully, enhancing both safety and user trust. He says, “It’s about the AI understanding what people want and not what they say.”
This section explores how safety efforts created a distinctive AI persona, the technical underpinnings of constitutional AI, and its impact on user experience, distinguishing Anthropic’s models from competitors.
8. Thriving in an AI Future: Curiosity, Creativity, and Kindness
Ben offers advice for an AI-driven future, focusing on skills he teaches his daughters: curiosity, creativity, and kindness. He says, “I just want her to be happy and thoughtful and curious,” prioritizing these over traditional academics in a Montessori setting. He also urges embracing AI tools ambitiously: “People who use Claude code very effectively... are asking for the ambitious change.”
This section provides practical guidance for listeners, emphasizing adaptability and human qualities that complement AI, alongside Ben’s personal reflections as a parent and technologist.
Conclusion
Ben concludes with a call to embrace the “wild times” ahead: “This is as normal as it’s going to be. It’s going to be much weirder very soon.”
He remains optimistic about AI’s potential, provided safety is prioritized. The lightning round features book recommendations like Replacing Guilt and The Alignment Problem, reinforcing his focus on ethics and strategy.
Crepi il lupo! 🐺