AI Whistleblowers, Safety, Ethics & the Race to Superintelligence

The AI Whistleblower Timeline

Whether you’re in technology or not, we all see that it is becoming increasingly important for us to adopt AI in our daily lives to stay relevant. And there are also debates about the ethical and moral implications of what AI tools we’re building and using.

Some of the most important voices in the AI debate today are actually coming from people who are helping build it.

OpenAI’s internal safety crisis became increasingly public in 2024, when researchers including Daniel Kokotajlo and William Saunders raised concerns about secrecy, whistleblower protections, and the race toward AGI.

Jan Leike, an AI researcher, has publicly explained his departure from OpenAI, saying that safety culture and processes had taken a backseat to “shiny products.”

Suchir Balaji left OpenAI after concluding that the company’s use of copyrighted material to train AI raised serious legal and ethical questions — and that generative AI could ultimately damage the very internet ecosystem it was trained on.

From Balaji questioning whether generative AI can legally consume the internet and potentially replace the ecosystem it depends upon, to Mrinank Sharma leaving Anthropic and warning that “the world is in peril,” a growing number of researchers and insiders have raised deeply uncomfortable questions about where AI is headed.

Then came Jacob Coxon, who resigned from Anthropic in September 2026 with an extraordinary warning that leading AI companies are racing toward self-improving superintelligence without having solved the fundamental problem of keeping it safe.

A Myriad of Problems

In the middle of such whistleblower news, job losses and layoffs as a result of AI automation may ultimately seem to be among the smallest of the problems we have.

The more profound question is what increasingly capable reinforcement-learning systems may eventually be able to learn, plan, execute, and pursue with limited human intervention. And will humans be able to stop anything that the machines have planned to destroy?

As we build increasingly powerful systems are we able to keep up with our ability to understand, control, regulate, and govern them?

We are already dealing with copyright battles, corporate pressure, safety concerns, questions about training data, and enormous commercial incentives surrounding AGI.

Yes, we already are seeing issues ranging from copyright, corporate pressure, safety culture and the enormous commercial incentives surrounding AGI that has led to a race game. But, when warnings are coming from people building this technology, we must pay ever more serious attention.

Because there is something uniquely unsettling about a whistleblower in artificial intelligence.

Thoughts from AI Experts and Whistleblowers

Here are some of the statements from people who have worked closest to these systems:

Daniel Kokotajlo

“I’ve gradually lost trust in OpenAI leadership and their ability to responsibly handle AGI.”

 

“Preexisting whistleblower protections don’t apply here because this industry is not really regulated…”

 

“OpenAI is really excited about building A.G.I., and they are recklessly racing to be the first there.”

William Saunders

“I really didn’t want to end up working on the Titanic of AI. And so that’s why I resigned.”

 

“There should be systems of oversight and accountability in place that genuinely hold the organization to that ideal.”

Mrinank Sharma

“The world is in peril. And not just from AI, or bioweapons, but from a whole series of interconnected crises unfolding in this very moment.”

 

“We appear to be approaching a threshold where our wisdom must grow in equal measure to our capacity to affect the world, lest we face the consequences.”

 

“I’ve repeatedly seen how hard it is to truly let our values govern our actions.”

Suchir Balaji

“If you believe what I believe, you have to just leave the company.”

 

“This is not a sustainable model for the internet ecosystem as a whole.” – on generative AI and copyrighted data

Jan Leike

“Over the past years, safety culture and processes have taken a backseat to shiny products.”

 

“Building smarter-than-human machines is an inherently dangerous endeavor.”

 

“We are long overdue in getting incredibly serious about the implications of AGI.”

 

“I have been disagreeing with OpenAI leadership about the company’s core priorities for quite some time, until we finally reached a breaking point.”

 

“These problems are quite hard to get right, and I am concerned we aren’t on a trajectory to get there.”

 

“We urgently need to figure out how to steer and control AI systems much smarter than us.”

Jacob Coxon

“Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.”

 

“The people building AI earnestly believe that it could kill us all by the end of the decade.”

Bonus Quote
“Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it’s >10% within the next decade.”
~ Evan Hubinger, Anthropic Alignment Science Lead

Important Questions To Ask

We as a community must thank these researchers and whistleblowers for asking questions and encouraging us to think critically of how we’re using AI. Whatever one ultimately believes about the probability of catastrophic AI risk, these people have forced an important conversation into the open.

There are two urgent questions.

What are companies willing to say publicly about what’s happening internally. What kind of internal debates, safety evaluations, disagreements, failures, or the decisions are being made behind closed doors?

And are our laws and governance policies moving fast enough to catch up with the changing happening with the technology. Technology is changing every day, but if regulations take months if not years to get executed, that’s not a sustainable and equitable AI ecosystem we’re creating.

And what terrifies me the most is, what will happen when we build something more powerful than our ability to govern it?

And how far is that day away?

– 0 –