The Short Answer
AI agent security is decided by infrastructure, not by features, because an agent is not a static endpoint — it is a reasoning loop with production access that reads data, calls APIs, and takes actions autonomously. A compromised agent does not merely leak information; it acts on the compromised instruction. You cannot fully secure a reasoning loop that runs on someone else's servers.
The controls that matter — where inference executes, who can read the prompts and weights, whether every tool call is logged in a system you hold, whether you can enforce your own access policy — are all properties of the deployment, not items on a vendor's feature list.
That is why the strongest agent security posture is structural: self-hosted execution inside your perimeter, model-agnostic architecture so no single provider's breach is automatically yours, and complete audit trails you own.
Why is AI agent security an infrastructure problem rather than a feature?
Because the agent's attack surface is its reasoning, and reasoning happens wherever the inference runs. Traditional application security assumes a fixed set of endpoints with predictable inputs and outputs.
An agent breaks that assumption: it decides which tools to call, in what order, on data it retrieves at runtime.
The consequence is that a prompt-injection payload buried in a support ticket or a PDF is not merely a content problem — it is potentially an instruction to a system with credentials. Uber's security lead put it plainly this week: "AI agents are everywhere at Uber. The thing that keeps me up at night is how we are going to secure them." Uber published a comprehensive agent security framework alongside that remark — not because the problem is solved, but because it is outgrowing the available solutions.
The tooling is catching up. Google made Agent and Model Evaluations generally available in its Gemini Enterprise Agent Platform in the same week, giving teams a way to measure, test, and monitor agents in development and production. That is real progress at the observability layer.
It does not answer the layer beneath it. Evaluation tells you how an agent behaves; it does not tell you who else can see the data it reasoned over, or whose incident response covers the machine it ran on.
What does the typical enterprise agent security failure look like?
It follows a three-phase arc that repeats across industries, and the damage is done in the gap between phase one and phase three.
Phase one: build and ship. Teams deploy agents to prove return on investment. The question is capability — can this handle support tickets, triage IT requests, draft contracts, summarize case notes? Speed wins. Security is a checkbox on the launch review.
Phase two: discover the attack surface. Every deployed agent is a new entry point with credentials attached. Unlike a form or an API route, it makes decisions.
Teams find that the same agent that summarizes a document will happily follow an instruction embedded in that document unless something stops it.
Phase three: retrofit. Rate limiting, output filtering, permission scoping, audit logging — all necessary, all significantly harder to add after the agent is embedded in a workflow that a business unit now depends on.
Retrofitting permission scoping in particular tends to break the very integrations that justified the project.
The organizations avoiding this arc are not the ones with better filters. They are the ones that made the deployment decision — where the agent runs and who holds the code — before phase one.
What can't you secure when the agent runs on a vendor's infrastructure?
More than most security reviews acknowledge, because the questions that matter are answered by the deployment model rather than by the vendor's certifications. A SOC 2 report describes the vendor's controls; it does not give you the controls.
| Security question | Managed SaaS agent | Self-hosted agent |
|---|---|---|
| Where does the reasoning execute? | Vendor infrastructure | Your servers, your network |
| Who can read prompts and retrieved context? | Vendor staff, per vendor policy | Only your privileged users |
| Who holds the audit log? | Vendor, exported on their schema | You, in your SIEM |
| Blast radius of a vendor breach | Your data, disclosed on their timeline | Contained to your perimeter |
| Model change | Vendor's roadmap decision | Your configuration change |
| Policy enforcement | Limited to exposed settings | Anything the source code allows |
The right-hand column is not a longer feature list. It is the same list of questions answered by ownership instead of by trust.
The distinction becomes concrete during an incident: forensic reconstruction of an agent's tool calls requires logs at a granularity most managed platforms never expose, and a subpoena or breach notification runs on the log holder's timeline, not yours.
The same dependency logic applies further down the stack, as we covered in supply-chain attacks and why infrastructure ownership contains them.
What does infrastructure-first agent security actually require?
Five controls, each of which is a deployment property before it is a product capability.
Self-hosted execution. The agent reasons inside your security perimeter — your servers, your network segments, your encryption keys. Sensitive data never crosses a boundary you do not administer, and the attack surface reduces to infrastructure you already monitor.
Model agnosticism as a security control. Binding an agent framework to one provider's models means inheriting that provider's security posture wholesale; a vulnerability or outage there is yours by construction.
A model-agnostic platform lets you swap models without touching your controls, and run open-weight models locally when sensitivity demands it.
Complete audit trails. Every tool call, data access, and decision logged at forensic granularity in a store you own. Compliance is the easy case; the hard case is incident response, where you need to reconstruct what the agent did and why, months later.
Role-based agent permissions. Agents inherit the permissions of the user or service they act for — an HR agent cannot reach financial systems, a support agent cannot read personnel records.
Obvious in principle, and treated as an afterthought by most agent frameworks, which default to a single service credential with broad scope.
Input and output filtering with programmable rails. Screen inputs before they reach the model and filter outputs before they reach users or tools.
This matters most for agents with system access, where an injected instruction can trigger a real-world action. ibl.ai implements this layer with NVIDIA NeMo Guardrails — programmable rails, jailbreak and injection defense, PII redaction, and network isolation — described in detail on our NemoClaw service page.
Does self-hosting frontier-class models still cost more than an API?
Not the way it did a year ago, and that is what changes the security calculus.
This week DeepSeek released V4 Flash 0731 as open weights under an MIT license, scoring 50 on the Artificial Analysis Intelligence Index — frontier-class territory — at roughly 28x cheaper output tokens than comparable closed models.
The practical implication for security teams is direct: the standard objection to self-hosting was that keeping data inside the perimeter meant accepting materially weaker reasoning. That trade is largely gone.
Organizations can now run models of comparable capability on their own hardware, behind their own firewalls, under their own policies.
That does not make every workload a self-hosting candidate. It does mean the decision reverts to where it belongs — a data-sensitivity and compliance judgment per workload — instead of being forced by a capability gap.
The open-weight releases arriving on a monthly cadence keep moving the line in the same direction.
What should you ask an AI agent vendor before signing?
Run the perimeter test. Five questions, and any "no" tells you the security model is trust rather than control.
- Can this run entirely on our infrastructure — our cloud account, our VPC, on-premise, or air-gapped?
- Do we receive the full source code, under a license that lets our engineers read and modify it?
- Can we swap the underlying model — commercial or open-weight — without changing our security controls?
- Does every agent interaction generate an auditable log we hold, at tool-call granularity?
- Can we enforce our own access policies, or are we limited to the settings the vendor exposes?
A vendor can answer yes to all five only if the architecture was built for it. Retrofitting deployment ownership onto a managed product is not a roadmap item; it is a different product.
For organizations that need every answer to be yes, ibl.ai is an Agentic AI Operating System deployed on your own infrastructure — managed cloud, VPC, on-premise, or fully air-gapped — with full source code and data ownership and any model you choose.
It is family-owned and operated from New York, NY, which matters to government, defense, and regulated buyers evaluating who ultimately controls the vendor as well as the software.
In a world where agents hold production access to your most sensitive systems, the difference between securing them and trusting someone else to secure them is the whole of your risk position.
ibl.ai is an Agentic AI Operating System that organizations deploy on their own infrastructure with full source code and data ownership — model-agnostic, usage-based, and deployable anywhere from managed cloud to fully air-gapped. Family-owned and operated from New York, NY. Learn more about enterprise deployment.