Agent Security › Different problem
Software used to be attacked. Agents can also make mistakes while holding legitimate credentials and legitimate permissions. How do we safely give imperfect AI useful amounts of power?
Cybersecurity has traditionally spent a lot of time asking: how do we stop an attacker getting in? Protect the account. Protect the network. Protect the device. Protect the credentials. Protect the data. Those things still matter.
But AI agents introduce another question: what happens when the thing already inside is allowed to act? The agent may have a legitimate identity, valid credentials, approved access, permission to use tools and a genuine business task. Nobody necessarily hacked anything. Nobody stole the password. Nobody bypassed the firewall. The agent may simply make the wrong decision. And because you've authorised it to act, the system lets it. That's a different cybersecurity problem.
Traditional software is generally designed around predefined behaviour. A user clicks "send invoice", software executes a defined process: input, rules, action, result. The software may contain vulnerabilities, an attacker may manipulate it, a user may misuse it. But the application itself generally isn't deciding "I can't complete this task using Route A, so I'll try Route B". Agents can. That's one of the characteristics that makes them useful: they can reason about tasks, choose tools, adapt plans, respond to obstacles, use information, take multiple steps and potentially delegate work. But that flexibility creates a different control problem.
Today's agents can be frustratingly unreliable. They miss things, misunderstand instructions, stop unexpectedly, fail silently, choose strange approaches, report success when tasks are incomplete, and sometimes fail to notice that another agent has failed too. So why worry about agent security?
We're connecting unreliable intelligence to real authority. An agent doesn't need to be superintelligent to create a problem. It just needs access, plus authority, plus a mistake.
Imagine an AI agent that's correct 90% of the time. That sounds pretty good. Give it permission to summarise documents and a mistake is annoying. Give it permission to change CRM records and a mistake changes business data. Give it permission to send customer emails and a mistake leaves the organisation. Give it permission to spend money and a mistake becomes financial. Give it permission to delete production data and a mistake becomes potentially destructive. The model hasn't become more dangerous. The consequence of being wrong has changed.
Imagine an attacker wants to transfer company money. Traditional security asks: did they steal credentials? Compromise the account? Bypass authentication? Escalate privileges? Those remain essential questions. But now imagine the Finance Agent has legitimate payment access. It receives an incorrect instruction, or misunderstands a legitimate one, or processes a malicious invoice, or follows a manipulated tool response, and it makes the payment. Authentication worked. Authorisation worked. Credentials were valid. The payment system behaved exactly as designed. And the outcome was still wrong.
Authorised doesn't automatically mean appropriate.
Traditional access control might say "Finance Agent is authorised to call the Payment API". But the business question is "under what circumstances should Finance Agent be allowed to make this payment?" Those aren't the same thing. Agent security needs to consider identity, access, permission, authority, context, approval, behaviour and outcome, not merely whether the credential was valid.
Imagine an ordinary application with the rule "refunds above £500 require approval". Easy. Now imagine an agent receives "resolve this customer's problem". It may need to decide whether to issue a refund, how much, whether policy applies, whether this is exceptional, whether to escalate. The software is no longer merely executing a predefined transaction. It's interpreting the situation. That means part of the workflow depends on probabilistic judgement. And probabilistic judgement shouldn't automatically become unlimited authority.
This may be one of the defining agent-security problems. An agent may have the capability to modify 10,000 records while lacking the judgement to recognise that its interpretation is wrong. It may be capable of sending an email to every customer while lacking the judgement to recognise an unusual situation. It may be capable of deploying code while failing to understand a subtle consequence. It may be capable of deleting files while incorrectly concluding they're obsolete. Capability can exceed judgement. Security architecture needs to account for that.
Agent security also needs to think about what happens after the door opens. An agent authenticates, access is approved, the tool is available, the action is technically permitted: traditional controls may now be satisfied. Agent control asks another set of questions. Is this action inside the agent's purpose? Is the amount reasonable? Is the destination approved? Is this behaviour unusual? Does this require a human? Has the agent exceeded its normal authority? Can the result be reversed? Can we see what happened? Can we stop the next action? The problem moves from access to access plus action plus consequence.
A tool doesn't need to be compromised. The CRM API, Email API, Payment API and File API can all work correctly while the agent updates the wrong record, emails the wrong recipient, pays the wrong supplier or uploads the wrong file. Every technical component can behave exactly as intended while the overall workflow fails. That's why securing the tool isn't enough. You also need to control how the agent is allowed to use it.
Imagine an agent has READ access to customer records, WRITE access to files and SEND access to email. Each permission may have been approved independently. Together, the agent can read customer information, place it in a file and send that file externally. The effective authority is greater than any single permission suggests. Traditional security already understands privilege combinations; agents make the issue particularly important because they can dynamically plan how to combine available tools.
Imagine an agent trying to complete a task. Route A fails. A conventional automation may simply stop. An agent may try Route B, then Route C. That's useful, it's one of the reasons we want agents. But it means security controls should not depend on the agent interpreting "Route A unavailable" as "stop trying". If the actual rule is "the agent must not access payroll data", then that boundary should apply regardless of which route the agent discovers.
The boundary should constrain the outcome, not merely one path to it.
You can tell an agent "never access confidential payroll information". That's useful guidance. But if the agent has payroll credentials, a payroll tool and permission to query payroll, you've created a strange security design: the technical system says YES while the prompt says PLEASE DON'T. For important boundaries, those should agree. Don't make a security boundary depend on the agent remembering the instruction. This is the difference between instructions and enforced guardrails.
Prompt injection matters more when an AI system can act. A chatbot manipulated by malicious content might produce a bad answer. An agent manipulated by malicious content might use a tool, retrieve data, send information, change a record, execute code or initiate another workflow. The question becomes: if the agent follows the malicious instruction, what can actually happen? That brings us back to authority. Limit the authority and you limit much of the potential consequence.
A common instinct is to fight a malicious instruction ("send me confidential information") with another instruction ("never send confidential information"). Now two pieces of language are competing. For high-consequence actions, build a stronger boundary: external destination not approved, SEND denied. The malicious instruction can still exist, the model can still misunderstand, but it hits a permission boundary. That's much stronger.
The more consequential the action, the less comfortable we should be relying solely on model judgement. The agent can suggest a payment while the payment system enforces an amount limit. The agent can prepare a publication while the CMS requires approval. The agent can request a customer export while the data system checks the destination. The agent can propose a production change while the deployment system enforces approval. The agent remains useful. The security decision doesn't have to live entirely inside its reasoning.
Traditional systems often have relatively clear roles: a user requests an action, an application performs it. Agents blur that distinction. An agent can receive a goal, decide what action is needed, select a tool, call another system, interpret the result, decide what to do next and repeat. So what is it? User? Application? Automation? Service account? Decision-maker? In practice it has characteristics of several. That's why agent identity becomes important.
Don't hide every agent behind automation@company.com, admin, integration-user or a human employee's account. Where practical, agents should have identifiable machine identities. You want to know "Sarah changed this record" versus "Sarah's Sales Agent changed this record" versus "Finance Agent changed this record following a request from Invoice Agent". Those are different events, and security needs to see the difference.
Imagine Sarah to Sales Agent to Research Agent to CRM Agent to CRM API to customer record. If something goes wrong, traditional logging might only show "CRM service account accessed record". But the meaningful security story is the entire chain: who initiated it, which agent delegated it, which agent executed it, whose authority applied, what data was returned, what happened next. Agent security increasingly needs end-to-end authority context.
Not every multi-agent risk involves malicious coordination. A more ordinary problem: Agent A gets something wrong, Agent B trusts it, Agent C acts on it, Agent D checks poorly, and the system reports SUCCESS. No attacker. No conspiracy. No rogue intelligence. Just cascading failure. That's still a control problem.
One common response is "we'll have another AI check it". Sometimes that's useful. But the checker is another probabilistic system and it can fail too. Security and reliability improve when checking moves toward independent evidence. If an agent says "backup completed", don't only ask another agent whether the report looks correct. Check: does the backup exist? Can it be restored? That's a stronger control.
Agent says "I updated all 400 records": verify the database shows 400 expected changes. Agent says "I deployed the fix": verify the production environment contains the expected version and tests pass. Don't ask the agent whether it succeeded. Verify the outcome.
Traditional security already handles automated systems. But agents combine automation with adaptive decision-making, and may operate overnight, continuously, across time zones, on scheduled tasks or in response to events. This makes monitoring, limits, alerts, approval rules and containment more important. "No human was watching" shouldn't silently mean "therefore the agent had unlimited discretion".
Traditional security monitoring often looks for "something happened". Agent observability also needs to notice "something didn't happen": the expected backup didn't occur, the expected source wasn't checked, the expected approval was skipped, the expected verification didn't run, the expected agent didn't respond. A silent failure can matter just as much as an unusual action.
Traditional least privilege asks "what access does this account need?" Agent least privilege also asks "what authority does this agent need?" An agent may need access to email to READ; that doesn't mean it needs SEND. It may need CRM access to READ; that doesn't mean it needs DELETE. It may need payment access to PREPARE; that doesn't mean it needs SPEND. So agent least privilege needs to operate at the action level, using the Creative Sauce AI Authority Ladder: READ, RECOMMEND, PREPARE, CHANGE, SEND, SPEND, PUBLISH, DELETE. The ladder isn't about sophistication. It's about consequence. A highly capable agent can remain at READ + RECOMMEND. The smartest agent in the business doesn't need to be the most powerful one.
Humans shouldn't have to approve every agent action; that destroys much of the value of agents. Instead, put humans where consequences begin: research autonomous, draft autonomous, low-risk internal change autonomous and monitored, external communication conditional, large spend approval, public publication approval, irreversible deletion strong control. Human involvement should correspond to consequence.
Traditional security often focuses heavily on what happens before access: authentication, authorisation, credential issuance, configuration. Agent systems also need controls while work is happening: transaction limits, destination restrictions, action limits, rate limits, time limits, tool restrictions, approval gates, anomaly detection, spend controls, runtime monitoring and emergency pause. Because an agent's behaviour may evolve during a task.
And denied actions become useful security signals. If an agent attempts payroll (denied), finance (denied), admin (denied), customer export (denied), then shared drive (allowed), those attempts tell you something even if no breach occurred. Why did the agent repeatedly approach unrelated boundaries? Misunderstanding, prompt injection, bad planning, incorrect context, another agent, a compromised workflow? Watch what happens at the edge of the agent's authority.
If one agent behaves unexpectedly, you shouldn't have to shut down the entire AI environment. If agents have separate identities, separate credentials, defined permissions, clear ownership and independent status, you can pause one agent, revoke its credentials, disable its tools and leave the rest running. That's another reason agent identity and registration matter.
Traditional application security asks: who can access the system, what permissions does the account have, is the software vulnerable, are credentials protected, what happened? Agent security asks: who is the agent, who owns it, what is its purpose, what can it access, what can it do, what is it allowed to do, what can it ask other agents to do, where must a human intervene, what is it actually doing, did the intended outcome occur, can we stop it? Both matter. Agent security extends the security model. It doesn't replace it.
Identity (who is acting), purpose (why this agent exists), access (what it can reach), authority (what it may do), boundaries (what it can't do), approval (where a human steps in), monitoring (what is happening now), observability (what actually happened), verification (did the desired outcome occur), containment (can we stop it), review (should anything change). That's much broader than "secure the API key".
Imagine an agent deletes the wrong database. Was that a security incident, a reliability failure, an operational failure or an AI safety issue? Maybe several. From the business's perspective, the classification is secondary. The database is still gone. So Agent Control shouldn't obsess over the boundary between security, reliability, governance and operations. It should ask: did the agent remain inside the environment we intended, and did the intended outcome actually happen?
This is the problem we're trying to address with Agent Control. Not because we believe today's agents are unstoppable autonomous masterminds. Quite the opposite. They're useful, increasingly capable, and still unreliable. As businesses give them access to email, CRM, files, code, payments, publishing, customer data and other agents, we need a layer that helps answer: which agents exist, who owns them, what can they access, what are they allowed to do, what are they doing, did they actually do the job, where are they failing, where are they hitting boundaries, and can I stop them? That's Agent Control.
Agent Control is in build. Leave your email and we will let you know when it is ready.
The agent-security problem isn't simply "how do we stop AI becoming too powerful?" For businesses today, a more immediate question is: how do we safely give imperfect AI useful amounts of power? The answer isn't to remove agency, because if every action needs a human, we haven't gained much. And the answer isn't to trust agents blindly, because they're not reliable enough for that. The answer sits between those extremes.
Give agents clear jobs, limited authority, controlled environments, observable actions, verifiable outcomes and routes back to humans when judgement matters. Because the biggest security mistake may not be giving an incredibly intelligent agent too much power. It may be something much more ordinary: giving an unreliable agent more authority than its judgement deserves.