Microsoft CEO Satya Nadella has called for an ’emergency brake’ for advanced AI systems, saying humans must always be able to pause or shut down an AI model while it is performing a task. He warned that companies must keep AI actions under control, especially when these systems have access to sensitive data and important systems.
In a statement shared on X, Nadella said companies need to rethink how they manage AI systems as these models become more capable of carrying out tasks independently.
He stressed that businesses cannot leave the responsibility for AI actions entirely to model developers. “We simply can’t outsource responsibility for what intelligence does on our behalf,” Nadella said.
He also warned that today’s advanced AI systems are difficult to understand compared to traditional software.
While developers could earlier trace a software problem to a specific part of the code, understanding why an AI model produces a particular response or takes a certain action is much harder.
Nadella said companies are giving AI systems access to sensitive information and allowing them to perform important tasks. This makes it necessary to introduce stronger controls to prevent mistakes, security breaches and unauthorised actions.
Why Satya Nadella Wants An Emergency Brake For AI
As per Microsoft CEO, companies should treat advanced AI models as potential insider risks. This does not mean that these models are necessarily malicious. Instead, they could make mistakes or be compromised when given access to important systems.
He said organisations must ensure that AI models operate within clearly defined limits, with humans retaining control over their actions.
Nadella called for an engineering-based approach that combines human oversight, external safeguards and regular testing. He also stressed that companies should be able to pause or shut down an AI model while it is performing a task.
“An authorised person should always be able to pause or shut down a model mid-task,” he said.
Nadella Wants Companies To Keep AI Actions Under Check
The Microsoft CEO also outlined several principles for making advanced AI systems safer. These include ensuring that every important AI action leaves a record that humans can review, testing systems for possible failures and keeping security controls independent of the AI model itself.
He said companies should not rely on a single AI model to perform a task and then verify its own work. Independent checks are needed to identify errors and prevent AI systems from operating beyond their intended limits.
Nadella also called for timely disclosure when AI systems fail or are compromised. “The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least,” He noted.
