Do not pass go: AI agents might be sending you directly to jail

Identity used to be a strictly human affair. Now, in our digital days, that’s no longer the case. Today, companies find themselves managing various forms of identities spanning humans, IoT devices with minds of their own, service accounts, and, perhaps most significantly, a rapidly growing number of AI agents.

New forms of identity management present unique challenges. Authentication and authorization are only the starting point; the deeper challenge lies in responsibility and accountability. When AI agents act, it is still humans and organizations that will be held to account.

AI agents have reached a new level of maturity

AI agents now handle everything from chatty customer service to heavy-duty data crunching. Some rely on retrieval-augmented generation (RAG), an AI framework that plugs large language models (LLMs) into live knowledge bases, ensuring up-to-date information and visibility into how their answers are made.

Unlike human users or traditional service accounts, AI identities don’t log in and wait for instructions; they interact, adapt, and make decisions on their own. That autonomy makes new guardrails essential. Yet most companies are scrambling to protect their data and ensure employees don’t—intentionally or not—hand it over to the very AI systems they use.

Current guardrails are the weak spot of today's GenAI systems

It took less than 24 hours after the release of ChatGPT GPT-5 to bypass these guardrails and apply different attack patterns that were a mix of cyber and psychological methods—like an LLM-optimized version of an echo chamber.

But how does that affect you? Well, the agent has access to your systems, and there are zero-click exploits out there to inject malicious prompts via connectors to Google Drive and other repositories. No matter how strong the guardrails, clever prompts can still trick an agent into believing they don’t apply in a given situation. AI agents have already dropped live databases, and coding assistants have been tricked into injecting code capable of wiping out entire systems. It would be just as easy for cybercriminals to instruct them to publish false financials, transfer cryptocurrency, or disclose company secrets.

Here’s the scary thought: How would you be able to prove it wasn’t you, or could you be held legally responsible for the actions an agent has taken on your behalf?

This is opening an enormous attack surface that we have never experienced before. We need guardrails that apply the Zero Trust paradigm to these emerging agents so we do not lose control completely.

Responsibility and accountability remain critical in the rise of AI

This is especially true from a compliance and ethical standpoint. If an AI agent makes erroneous decisions or unauthorized actions, who is responsible? Organizations must establish transparent accountability structures, clearly delineating that while agents automate tasks, ultimate accountability resides with human supervisors. We will all have to learn a great deal before we can fully leverage the potential of GenAI—and learning, as always, comes through making mistakes.

Consider a financial services firm deploying an AI assistant to handle customer queries. An attacker embeds a malicious prompt inside a seemingly harmless PDF uploaded to a shared drive. The AI, following the hidden instructions, extracts and transfers confidential account data to an external system. The customer service team never intended such a breach, but without clear accountability structures and preventative controls, regulators could still hold the firm responsible.

How IAM is becoming the essential barrier in the AI landscape

The rise of AI agents necessitates smarter approaches to authentication and authorization. Rather than relying on static credentials, organizations should verify agents based on the patterns and context of their activity.

Following the principle of least privilege, agents must be given only the permissions they need—no more, no less. Properly scoped access and rights ensure that agents have enough authority to complete their tasks while remaining constrained within well-defined boundaries, reducing the potential for misuse or unintended actions.

Identity and Access Management (IAM) systems play a critical role here. By clearly defining supervisory roles, delegating rights, and managing the entire lifecycle of AI agents, organizations can establish the safeguards needed to keep these systems in check.

If you ask me, this is now mission-critical. Your next breach may not come from an external hacker at all—it could come from your own AI agent. The real question is no longer if AI agents will be exploited, but whether your organization will be ready when it happens.