Microsoft CEO Satya Nadella says powerful AI needs an off switch that humans control.

In a long post on X on Saturday, Nadella called for advanced AI systems to be built with containment, independent controls and an "emergency brake," CNBC reported. The brake would let authorized people pause or shut down a model in the middle of a task.

"We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient," Nadella wrote.

He said it is time "to step back and assess the trust architecture" of AI, TechCrunch reported. "We can't treat Super Intelligence as a set of nested black boxes and simply accept or reject its recommendations, answers, and actions," he wrote.

Nadella said one way to build safer systems is to treat AI models "like insider risks." That is how companies treat employees who could misuse their access. The Verge summed up his view this way: assume all AI models could be "compromised."

He called for separating the AI model from the software that directs its work, and for keeping safety controls outside the model. He also wants systems to leave "tamper-proof human readable evidence" of what they do.

Many of Nadella's ideas match what others in the industry have proposed, The Verge said. Microsoft is a major investor in OpenAI.

What we know: Nadella wants independent controls and an emergency brake on advanced AI.
What we don't: whether Microsoft will commit to specific limits on its own models.