Share This Article
Artificial intelligence is moving on from the traditional chatbot model. Newer artificial intelligence systems can plan tasks, use software tools, interact with applications, write and execute code, and perform multi-step workflows with limited human intervention, rather than just responding to prompts.
This shift to so-called AI agents is creating new opportunities for businesses, but it is also posing difficult questions about safety, oversight, and how much autonomy artificial intelligence systems should be granted.
In this context, Microsoft is focusing more on AI governance and human control. On September 13, Microsoft Chairman and CEO Satya Nadella said that the company would publish a Code of Conduct for its first-party MAI models for public feedback.
His comments come as part of a wider industry debate about whether the speed of progress in developing increasingly powerful artificial intelligence systems should be slowed.
Table of Contents
How AI Agents Are Changing the Safety Conversation
Normal generative AI applications typically wait for a user to provide a prompt and then provide an answer. The difference with AI agents is that they are designed to pursue a goal over multiple steps.
An agent can choose which tools to use, gather data, interact with an application, perform an action, assess the outcome, and continue progressing toward the goal. In some implementations, multiple agents can also work together on different parts of a larger task.
That makes agents potentially much more useful than traditional chatbots. A company may use an agent to automate software development tasks, analyze information, coordinate workflows, or perform repetitive operational work.
But greater independence also creates more opportunities for something to go wrong.
A chatbot providing an incorrect answer is one type of problem. An agent that has access to company files, APIs, databases, or business applications can potentially turn an incorrect decision into a real-world action.
Microsoft has already acknowledged this difference in its current AI Services Code of Conduct. Its rules for autonomous artificial intelligence systems require customers building systems that can independently make decisions and execute actions to maintain appropriate human controls, monitor decisions and actions, detect anomalies, intervene when necessary, and provide transparency into the system’s capabilities and limitations.
Microsoft’s Latest Foray Into AI Model Governance
Nadella’s most recent comments add another dimension to that conversation.
According to reports on his September 13 announcement, Microsoft is planning to publish a Code of Conduct for its first-party MAI models and open it for public consultation. The move comes as leading AI companies are engaged in a debate over the pace at which frontier AI capabilities should evolve and what guardrails should come with them.
The timing is notable.
Anthropic CEO Dario Amodei recently urged the AI industry to “pace the frontier,” slowing the rate at which frontier-model capabilities are developed and increasing independent safety evaluation. Other technology leaders, including Microsoft, subsequently backed the proposal.
This does not mean that Microsoft is backing away from AI development. Instead, the emerging debate is increasingly about how companies can continue to build powerful models while putting meaningful safeguards around them.
Human Control Is Emerging as a Key Principle
Who is responsible when an AI system takes action? This is one of the biggest questions concerning autonomous AI.
One possible answer is Microsoft’s existing guidance. Human supervision should remain an option, especially when an autonomous system is making sensitive or irreversible decisions that could cause harm. Organizations also need mechanisms to identify failures and intervene when necessary.
As companies give artificial intelligence systems access to more tools, this is becoming increasingly important.
A document-search system attached to only one agent is limited in its capacity to cause damage. An agent linked to email, financial systems, production infrastructure, customer databases, or software deployment tools has a much wider range of possible activities.
Therefore, the security implications do not depend only on how intelligent an AI model is, but also on what the model is allowed to access and what actions it is allowed to perform.
Microsoft researchers have also highlighted risks associated with tool-enabled AI agents, including problems arising from excessive privileges, a mismatch between an agent’s capabilities and its intended purpose, and unintended access to authority in execution environments.
AI Agents Bring New Security Risks Too
Security researchers are increasingly turning their attention to risks such as prompt injection, overly broad permissions, and unsafe tool use.
Earlier, Microsoft published research showing how prompt-injection attacks could exploit a flaw in an AI-agent framework to achieve remote code execution. The research highlights a major difference between typical AI-generated text and an AI system that can interact with software and take action.
The broader AI security landscape is changing, too, with threat actors experimenting with more advanced models. Anthropic recently described examples of threat actors attempting to use Claude for malicious purposes, illustrating how AI capabilities are being integrated into increasingly sophisticated cyber operations.
Such developments do not imply that AI agents are inherently unsafe. Instead, they show that security controls need to evolve alongside their capabilities.
The Cost of More Autonomy in AI
But it is not just a safety issue.
AI agents can also require considerably more computing power than a simple question-and-answer exchange. An agent may take many intermediate steps, make multiple calls to models, use external tools, or run for an extended period.
WIRED recently reported that the move from simple chatbot queries toward agentic systems is contributing to increased demand for data-center capacity and energy. The resources required can vary significantly depending on the complexity and duration of an agent’s task.
For businesses, this creates another important consideration: an autonomous workflow may reduce human labor requirements while simultaneously increasing model usage and infrastructure costs.
What It Means for Businesses
The increasing focus on AI governance is not a reason for companies to shy away from AI agents. Rather, organizations will need to approach autonomous artificial intelligence systems differently from conventional productivity software.
Businesses deploying agents need to understand exactly what each system can access, what actions it can take, and where human approval is required.
Organizations should provide agents with access only to the resources they actually require. Additional controls should be put in place for sensitive operations, while monitoring systems should be used to detect unusual behavior.
Microsoft’s own responsible AI guidance for building agents highlights reliability and safety, privacy and security, transparency, and accountability. It also suggests incorporating decisions around data access, model selection, and human approval into the architecture itself, rather than treating responsible AI as a final review stage.
A Broader Change in the AI Industry
Microsoft’s latest move is part of a much bigger shift in the way the technology industry talks about AI.
Much of the debate a few years ago focused on whether generative AI could produce convincing text, images, or code. Today, the conversation is increasingly about what happens when these systems can act and not just generate.
This is a key difference.
A user can correct an AI model that produces an incorrect paragraph. However, an incorrect decision made by an autonomous system and executed across multiple interconnected systems can create a much larger problem.
As AI agents become more capable, governance will therefore need to address not only what models can generate, but also what they can access, what they can decide, and what they are allowed to do.
Microsoft’s move to put a new MAI Code of Conduct out for public consultation indicates where that discussion is heading. The industry is no longer asking only how to build more capable AI. Increasingly, it is asking how to build AI that can operate with greater autonomy without losing meaningful human control.
That may become one of the most important technology decisions of the next few years for companies adopting agentic AI.
