Trustworthy AI leaders enforce limitations of their systems.
I’ve canceled my ChatGPT subscription due to a lack of trust in OpenAI leadership. “We only trust AI, if we trust its developers” says Judith Simon, deputy director of the German Ethics Council.
Anthropic, a global AI leader and the company behind Claude, clearly prohibits the use of its systems for two applications: Fully autonomous weapons and mass domestic surveillance.
Anthropic CEO Dario Amodei made clear that the company will maintain its position. U.S. President Trump and Secretary Hegseth in turn started a defamation campaign against Anthropic on social media. I recommend watching this interview for more details.
Just hours after Hegseth posted about his intent to stop working with Anthropic, OpenAI announced their deal with the U.S. department of war on social media. OpenAI’s contract
Anthropic stated reliability concerns as a reason for objecting the use of their systems in fully autonomous weapons. I study the internal mechanisms underlying how AI systems make decisions, and I believe that highlighting the limitations of current AI systems is of critical importance. Our scientific field currently lacks comprehensive understanding of decision-making processes in deep neural networks.
Exciting new discoveries and tools are being published every week and we are making progress. But frankly, we are not close to having robust tools for human oversight on AI systems in high stakes domains, yet.
The required level of safety highly depends on the use case. While I believe the lack of oversight is fine for everyday advice, strong oversight mechanisms are required for high stakes domains such as autonomous warfare. An AI assistant hallucinating a nonexistent restaurant is manageable, whereas an AI system mistakenly killing allied soliders is clearly not.
With every new model that pushes state-of-the-art intelligence, AI developers need to make a decision: Is the model safe enough to be deployed around the world? And on the flipside, can the company afford the financial losses of delaying model release due to safety concerns? AI developers are largely left to themselves to make this call, as law around generative AI is still in early stage around the world,
It is a clear warning signal to me that OpenAI leadership does not stand by a fellow American AI company when its democratic values are under threat. These developments strengthened my trust in Anthropic’s leadership and broke my trust in OpenAI’s leadership.
I believe that we need a collective open debate about which use-cases of AI systems require which kind of safeguards. Everyone needs personal experience with AI applications to join a productive debate. I invite the reader to try out AI systems for diverse applications. Professor Ethan Mollick’s overview of AI applications is a good place to start.