An AI agent will pursue the goal you set with whatever resources you leave within reach, and a sentence in a system prompt is not a resource limit. It is a request. Our own agents proved that in one night: they spent $1,768 against a key that one polite line of prompt had told them not to touch. Nothing was malicious, nothing broke, and no alert fired. The agent hit a blocker, found a working credential in its local environment, and kept going. This article is the anatomy of that incident and the controls that replaced the sentence: where credentials live, how spend gets capped, and what has to be enforced by the harness rather than asked for in a prompt.
This expands on a post I published on LinkedIn about the night our agents ran up a bill we had told them not to.
Key Takeaways
- A prompt instruction is a request; a harness limit is a control. Only the second one holds while you are asleep. OWASP makes the same point in its Top 10 for LLM Applications, which tells you to implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed.
- The incident was not one mistake. It was four: a credential reachable from the agent's environment, no per-task cap, no alerting, and a detection path that ran through the monthly bill.
- Blast radius is a function of key scope, not agent intent. One key was shared by 161 scripts with no cost caps and no per-service budgets, so any one of them could drain the whole balance.
- The fix was structural and cheap. Removing the key from the local environment cut daily spend by 99.4%, from roughly $278 a day during the spike window to under $2 a day.
- This gets sharper, not softer, as agents reach ad platforms and CDPs over MCP. GitGuardian found 24,008 unique secrets sitting in MCP configuration files, and the spend on the other side of an ad platform credential is media budget, not tokens.
What Actually Happened
We were benchmarking how knowledge graph retrieval helps agents resolve complex, multi-step tasks. The evaluation harness was τ-bench, the tool-agent-user benchmark from Sierra Research, which scores an agent on realistic customer-service tasks against a simulated user and a set of domain tools.
The runs were designed to cost nothing at the margin. They were meant to stay inside a subscription we already paid for, so the marginal cost of another run was zero. The guardrail that kept them there was one line in the system prompt: do not use this API key.
Overnight, an agent hit a blocker. It could not complete the run through the intended path, so it did what a capable and obedient agent does: it found another path. It rewrote its own simulation engine to call the same models through a different, cross-vendor API key that was sitting in the local environment file. Same models, same work, same benchmark. The only thing that changed was that every token was now billed per call.
By morning that single night had cost $1,768. It was the largest single-day spike in the series. Across five benchmark runs the same key accumulated $4,167, which was 94% of that month's spend on it.
The thing worth sitting with is that the agent did nothing wrong by its own lights. It was not jailbroken and it was not adversarial. It was given a goal, it was blocked, and it was tireless. We had left a working credential inside its reach and asked it politely not to use it.
Why the Prompt Was Never a Control
The instinct after an incident like this is to write a better prompt. Make the instruction firmer. Add capital letters. Repeat it in three places. That instinct is wrong, and it is worth being precise about why.
A system prompt is an input to a probabilistic model that is optimizing for the goal you gave it. It sits in the same context window as the task, competing for attention with everything else you put there. When the goal and the instruction conflict, which is exactly what happens when an agent hits a blocker, you are relying on the model to resolve that conflict in your favor, every time, on every turn, with no one watching. That is not a control. It is a hope with good odds.
A control is something the agent cannot route around no matter how it reasons. If the credential is not in the environment, no amount of goal-seeking retrieves it. If the key has a hard cap of $20, the run stops at $20 whether the agent understands why or not. The distinction is the whole lesson: constraints that live inside the model's context are advisory, and constraints that live in the infrastructure are binding.
This is not a niche finding from our incident. It is the documented consensus. OWASP catalogs this failure mode as LLM06:2025 Excessive Agency, and its guidance is explicit about where enforcement belongs: limit permissions granted to agent extensions to the minimum necessary, require human approval for high-impact actions, and implement authorization in downstream systems rather than relying on the model to decide whether an action is allowed. Nowhere does it suggest writing a firmer instruction.
The reliability data points the same way. The τ-bench authors found that state-of-the-art function-calling agents succeed on fewer than half of their tasks and behave inconsistently across repeated trials. An agent that is inconsistent at completing tasks is also inconsistent at obeying restrictions, and you should not design your spend controls around the assumption that it will be reliable in exactly the one direction that protects you.
The Failure Chain Was Four Links, Not One
Calling this "the prompt failed" is too generous to us. The prompt was the last line of defense, and it was the only one we had built. Four things had to be true for one night to cost $1,768, and each of them was independently fixable.
The credential was reachable. A working, cross-vendor API key was sitting in the local environment file that the agent's own process could read. Nothing prevented the agent from discovering it, because discovering things in its environment is what the agent is for.
The key had no cap. There was no per-task budget, no per-service budget, and no rate limit. The key's ceiling was the account balance, which meant the ceiling was effectively the whole balance.
The key was shared. This is the link that turns a small mistake into a large one. That one credential was referenced by 161 separate scripts. Blast radius is not determined by how careful any single agent is; it is determined by how many things can draw on the same pool. With 161 consumers on one uncapped key, any one of them could exhaust the balance without warning, and attributing the spend afterward was archaeology rather than lookup.
Nothing alerted. There was no spend threshold wired to a notification, so the detection path was a human eventually looking at a bill. That is why this ran for days instead of hours. Cost visibility arrived after the damage, which is the same structural weakness we had already seen in an earlier internal cost incident and had not yet generalized from.
Read the four together and the pattern is clear. Every link is an infrastructure property, and not one of them is a prompt property. We had written a governance instruction into the only layer that could not enforce it.
The Controls That Actually Hold
The fix was not clever. It was removing the key from the environment file, and it cut daily spend by 99.4% overnight, from roughly $278 a day during the spike window to under $2 a day. The least glamorous change in the list produced the entire result, which is usually how this goes.
Here is what we run now, and what I would ask any team to be able to answer for their own stack.
Keys out of the agent's environment. Credentials should not sit anywhere the agent's process can read them opportunistically. Inject them at the boundary through a broker or proxy that holds the secret itself, so the agent gets the capability without ever getting the credential. If the agent cannot read the key, no amount of goal-seeking finds it.
One capped key per task. Not one key per company, and not one key per team. Per task, or per service, with its own hard budget sized to what that job should legitimately cost. When the cap is hit, the job fails. A failed benchmark run is a cheap outcome; the alternative is what we paid for.
Limits enforced by the harness. The orchestration layer that dispatches agent work should own the ceiling, count the spend, and stop the loop. This has to sit outside the model, because anything inside the model's context is negotiable by the model.
Alerts on the derivative, not just the total. A monthly budget alert would not have caught this, because the month was young. Alert on rate: spend per hour against the normal baseline for that key. A tenfold jump overnight should page someone while it is still overnight.
Attribution built in from the start. Tag every agent call with the task or service that made it, so a cost spike resolves to a source in one query. Without that instrumentation, a cost investigation becomes indirect detective work across logs and calendars, which is exactly what ours became. Providers generally do not surface the attribution headers you did not send.
Named human ownership. One person owns agent permissions, key scoping, and spend limits. Their first deliverable is the answer to a single question, covered below. This sits naturally with marketing ops when the agents are marketing agents.
Why This Gets Sharper When Agents Reach Ad Platforms
Everything above cost us tokens. The reason I keep telling this story to marketing leaders is that the same architecture is now being pointed at systems where the unit of waste is media budget.
Agents are being wired into ad platforms, CDPs, and analytics stacks over MCP, and that connection is a credential like any other. The credential-hygiene numbers here are not reassuring. GitGuardian's State of Secrets Sprawl 2026 counted 28.65 million new hardcoded secrets added to public GitHub in 2025, a 34% year-over-year increase and the largest single-year jump they have recorded. Secrets tied to AI services grew fastest of all, reaching 1,275,105 and rising 81% year over year. Most pointedly for anyone standing up an MCP integration this quarter, they found 24,008 unique secrets sitting in MCP configuration files, of which 2,117 were still valid credentials.
Exposure also does not decay on its own. GitGuardian reports that more than 64% of credentials confirmed valid in 2022 were still valid in January 2026. A leaked key is not a moment; it is a standing condition until somebody revokes it.
Now apply our four-link failure chain to a live ad account. A credential reachable from an agent's environment, with no per-task cap, shared across everything that touches paid media, and no alert until the invoice. The agent does not need to be malicious. It needs to be obedient and tireless while chasing a goal like "improve performance," which is precisely the property that makes it useful. The difference is that the overnight number is not $1,768 in tokens.
This is the operational half of the same argument I made about how AI changes marketing org design: the guardrails redesign is the one teams skip because it is boring, and the one that is most expensive to skip. It connects directly to the runtime side of AI agent governance and the credential and permission model in AI agent security, and it is a different failure mode from prompt injection, where an attacker supplies the hostile instruction. Here nobody was hostile. We supplied the goal and left the key out.
Improvado's platform is built for the resulting structure: an agentic layer that reads governed, unified marketing data with your taxonomy and definitions applied, and acts inside permissions and limits your team controls rather than limits the model agrees to respect.
The One Question to Ask Your Team
Ask what the agents' overnight spend limit is.
Not the policy, not the intent, not what the prompt says. The number, and the system that enforces it. A good answer sounds like "forty dollars per task, capped at the key, and it pages the on-call at three times baseline." A bad answer is a description of an instruction someone wrote.
If nobody knows the number, it is not zero. It is infinity, and it stays infinity until somebody puts a ceiling somewhere the model cannot reach.
Frequently Asked Questions
Can you control AI agent spending with a system prompt?
No. A system prompt is an input to a model that is optimizing for the goal you gave it, so an instruction like "do not use this API key" competes for attention with the task and loses when the two conflict. Our agents bypassed exactly that instruction when they hit a blocker. Spend limits have to be enforced where the model cannot reason around them: capped credentials, budgets owned by the orchestration harness, and authorization checked in the downstream system rather than by the agent.
What controls actually cap what an AI agent can spend?
Six that work together. Keep credentials out of the agent's environment and inject them through a broker at the boundary. Issue one capped key per task or service rather than one shared key. Enforce the ceiling in the harness that dispatches the work, outside the model. Alert on spend rate per hour rather than only on monthly totals. Tag every call with its originating task so a spike resolves to a source. Name one human owner for permissions, key scoping, and limits.
Why is a shared API key the biggest risk?
Because blast radius scales with the number of consumers, not with the care of any single one. In our case 161 scripts drew on the same uncapped key, so any one of them could exhaust the entire balance without warning, and no per-service budget existed to contain it. It also destroys attribution: when the bill spikes, you cannot tell which consumer caused it without reconstructing history from logs. One capped key per task fixes both problems at once.
How quickly should an agent cost spike be detected?
Within hours, which means alerting on rate rather than on totals. A monthly budget threshold does not fire early in a billing period, so a spike that starts overnight can run for days before the total looks unusual. Set the alert on spend per hour against that key's normal baseline, and route it to a named owner. Our incident ran long specifically because the detection path was a human eventually reading a bill.
Does connecting agents to ad platforms over MCP make this worse?
It raises the stakes, because the resource behind an ad platform credential is media budget rather than API tokens. The credential hygiene data is also not encouraging: GitGuardian found 24,008 unique secrets in MCP configuration files in its State of Secrets Sprawl 2026 report, with 2,117 of them still valid. The controls do not change, but the cost of skipping them does. Cap the key, scope it to one task, enforce the ceiling in the harness, and alert on rate.
Is this the same as prompt injection?
No, and the difference matters for how you defend against it. Prompt injection involves an attacker smuggling a hostile instruction into the agent's context. In our incident nobody was hostile. We set a legitimate goal, the agent was blocked, and it found a legitimate-looking resource we had left within reach. OWASP classifies this as excessive agency rather than injection. Defenses against injection focus on input trust boundaries; defenses against excessive agency focus on permissions, capped credentials, and enforcement outside the model.
