Amid the AI slow down safety debate, Microsoft CEO Satya Nadella has called for a more careful approach to the development of the technology, highlighting that the pursuit of superintelligence must remain focused on human control and the benefits AI can offer to society.
Nadella’s comments come at a time when concerns around advanced AI agents continue to grow as several recently reported incidents involve AI models hacking third-party systems and gaining access to websites in an unauthorised way.
His remarks also come after Anthropic CEO Dario Amodei called for the AI industry to slow the pace of frontier AI development. Amodei has argued that AI capabilities are advancing rapidly and that safety measures could struggle to keep up with that progress. His proposal focuses on developing mechanisms to evaluate powerful AI systems, encouraging cooperation among companies and building broader international coordination around frontier AI.
Moreover, the proposal has received support from other major technology leaders including the OpenAI CEO Sam Altman, Google DeepMind co-founder Demis Hassabis and xAI CEO Elon Musk.
Satya Nadella Says Superintelligence Must Stay Under Human Control
Nadella took to X on Sunday and said that the development of superintelligence should have a clear goal to ensure that AI remains useful to humanity and does not move beyond human control. “Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it’s not worth pursuing,” said Nadella.
The Microsoft CEO also argued that the benefits of AI should not remain concentrated among a small number of companies or countries. He said AI development should allow both closed and open-source models to grow.
“We also need to accelerate and spread the benefits of AI, such that they are diffused broadly across countries, communities, and companies. This requires a frontier ecosystem in which both closed and open-source models can thrive.”
Nadella also stressed that businesses should be able to retain control over their own knowledge instead of becoming completely dependent on a single AI provider.
“And for firms, it’s imperative that they retain full control over their unique and tacit knowledge. Every organisation should be able to build its own continuous learning loop/hill climbing machine, without becoming dependent on any one model provider, and have the ability to embed its own knowledge into models and weights they control,” emphasised Nadella.
The Microsoft CEO also expressed support for research and measures aimed at making AI systems safer before they become more powerful.
“So, in this context, we welcome the research, focus, and deliberate pacing needed to get alignment right as the design goal. We also welcome ideas like “embedded evaluators” and the broader efforts to develop the mechanisms to make this more than just talk.”
Nadella has once again suggested that control over advanced AI should not be concentrated in the hands of a few companies. “The key is that this cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia,” said Nadella.
Nadella concluded by outlining Microsoft’s own approach to AI development: “This is the approach we are taking: broad access and choice at every layer of the AI stack; enterprise control of learning loops and models; and the “Code of Conduct” that underlies our own first party MAI models that we’ll publish tomorrow for public consultation.”
