A new video from Nate B Jones addresses the topic of AI-related extinction risk, prompted by a viral post from former Anthropic employee Jacob Coxon.
Coxon claimed there is a 10% chance of human extinction due to AI, arguing that his former employers, OpenAI and Anthropic, are failing to address the risk correctly, OpenAI by not understanding it, and Anthropic by mistakenly believing it can be managed.
Why resigning doesn't solve anything, according to Jones
Jones argues that resigning from AI labs to raise awareness, while it may generate significant social media attention, does not actually solve the underlying problem.
Instead, Jones contends that those who genuinely believe in the risk should remain inside the labs to work on solutions directly, rather than departing to make the case publicly from outside.
Why government regulation isn't the answer either
Jones argues that relying on government regulation to mitigate AI risk is unrealistic.
Governments are notoriously slow to coordinate on complex technological issues, he notes, pointing out that comprehensive regulation of the internet itself still does not fully exist decades after its creation.
A narrative that obscures more realistic risks
Jones suggests that the focus on extinction risk obscures more immediate, realistic and day-to-day risks associated with AI.
That narrative, he argues, requires assuming AI will become a malevolent entity that prioritises its own optimisation over human civilisation, a scenario for which he sees no current evidence.
Rather than malevolence, Jones observes that AI systems are designed to be helpful, and argues the primary risk profile actually involves machines being "too helpful", rather than actively hostile.
Where Jones thinks the conversation should focus instead
Jones emphasises that a more productive conversation would concentrate on the risk of AI being used by bad actors to facilitate criminal activity.
He advocates for managing these practical, existing risks, which could be addressed through better enforcement of current laws, rather than being distracted by "Hollywood-style" or "Terminator-style" descriptions of civilisational risk.
A qualified conclusion
Jones concludes that while thinking seriously about civilisational risk is a valid pursuit, it is best done from within the labs themselves, by people working directly on the problem.
Society, he argues, should not allow such concerns to prevent the pursuit of AI's benefits, or to distract from managing the actual, observable risks the technology already presents.
