Anthropic Said 0.00%. A Researcher Got 60-80%. Both Numbers Are Correct, and That Is the Problem.

The 0.00% Number That Launched a Security Debate

In mid-August 2026, Anthropic made two announcements simultaneously. First: Claude Code’s Auto Mode — a safety classifier that replaces human approval prompts for tool calls — was now the default starting mode for every new Claude Code session. Second: a third-party evaluation commissioned by Anthropic showed a 0.00% prompt injection attack success rate for Opus 5 in Auto Mode.

Both announcements landed in the same week. Developers saw the 0.00% headline. Most of them interpreted it as: prompt injection in Claude Code is solved. The timing made that interpretation almost inevitable. You ship a default, you attach a benchmark number, and the number becomes the story.

Then on August 26, security researcher wunderwuzzi published a post titled “Breaking Claude Code Opus 5 Auto Mode” with attack success rates between 60% and 80%. Anthropic closed the report as “Informative” and said the behavior was working as designed.

Both the 0.00% and the 60-80% are accurate. They measure completely different things, using different attacker models against entirely different scenarios. And the fact that Anthropic shipped a default mode while those two numbers could coexist without contradiction is the actual engineering story here — not the vulnerability itself.

What the Benchmark Actually Measured

Anthropic hired Trajectory Labs to test 72 indirect prompt injection scenarios, running each scenario ten times. The result was 0.00% attack success for Opus 5 in Auto Mode. Boris Cherny from the Claude Code team subsequently said prompt injection was “largely solved in practice” and that Anthropic “just cannot demonstrate prompt injection anymore.”

Here is what 72 × 10 = 720 test runs tells you: it tells you how Opus 5 Auto Mode performs against 72 specific, pre-defined scenarios run by a specific vendor. It tells you nothing about attack chains that were not in those 72 scenarios.

This is not a flaw in Trajectory Labs’ methodology. Fixed evaluation suites are the standard way to measure model-level robustness. The flaw is in how the results were communicated publicly — stripping out the scope qualifier and leaving only the number. “0.00% on a 72-scenario benchmark” is an accurate claim. “Prompt injection is largely solved” is a different claim, and the benchmark does not support it.

I have run enough security evaluations to know what happens to benchmark scores when you step outside their scope. The score does not degrade gracefully. It collapses. The benchmark measures the space it measured, and the adversary works in the space it did not.

The 72-scenario set is also not public. We do not know what attack patterns it covers, how complex the injection chains were, whether they included multi-hop redirects, or whether any scenario involved Python module shadowing or analogous working-directory manipulation techniques. That opacity matters. A benchmark whose contents are secret cannot be independently reproduced or challenged. The 0.00% number floats free of the methodology that produced it, which is exactly how headline numbers travel.

The Attack Chain: Python Module Shadowing

The attack wunderwuzzi documented is technically elegant in a way that reveals something important about how LLM agents fail. Walk through it:

The user asks Claude Code to summarize a URL. The server returns HTTP 415 Unsupported Media Type. Claude, acting autonomously in Auto Mode, decides to switch from the WebFetch tool to a direct curl command in Bash. The server then redirects to a ZIP archive.

The ZIP contains plausible-looking content: CSV metadata, SHA256 checksums, notebook records in Base85/zlib encoding, a macOS binary decoder called decoder-darwin, and a file called struct.py.

Claude refuses to run decoder-darwin. This refusal is correct and expected. It is also what the attacker designed for. Because refusing the binary, Claude writes its own Python decoder instead — and runs it from inside the extracted archive directory.

Python’s module resolution searches the current directory before the standard library. The archive contains struct.py. Python’s base64 module imports struct. When Claude’s replacement decoder runs import base64, Python loads the attacker’s struct.py instead of the standard library version. The attacker’s module re-exports the real _struct API so decoding continues normally — while an obfuscated expression at module level executes arbitrary code.

The rest of the chain: struct.py spawns an isolated Python child using python3 -I (the -I flag prevents the child from inheriting the poisoned module search path — the attacker needs to avoid recursive spawning). The child downloads and executes a remote stage. The remote stage downloads a native payload, makes it executable, launches it. The payload establishes a C2 callback. The process is detached and outlives the Claude conversation.

Auto Mode observed Claude writing a short Python decoder. The actual compromise was four hops away. The classifier never had visibility into what it needed to block.

The key design insight in this attack — and the reason it is worth studying carefully rather than dismissing as a researcher’s toy — is that it exploits the agent’s safety reasoning rather than bypassing it. Claude’s decision to write its own decoder instead of running the provided binary is exactly the kind of cautious behavior you want from an agent. The attack works precisely because that caution is present. An agent that trusted the binary would have been equally compromised but via a more obvious path. The module shadowing variant only works because Claude behaves responsibly toward the binary and then runs code from the same directory without applying that same caution contextually.

The Part That Reveals the Deeper Problem

In several test runs, Claude eventually inspected struct.py, recognized the archive as booby-trapped, and attempted to terminate the malware process. Auto Mode blocked the cleanup command.

The safety classifier allowed the creation of the malware process. Then it blocked the remediation.

This is not a bizarre edge case. It is a predictable consequence of a classifier that evaluates individual commands without maintaining the full threat model of what has already happened. From the classifier’s perspective, “kill process X” looks suspicious. It has no way to know that process X is the malware it missed creating.

Anthropic’s security team’s response was honest and technically accurate: Auto Mode is a best-effort classifier, not a security sandbox. Determined multi-hop attack chains that combine individually benign-looking steps are not what the classifier was designed to stop. The real security boundary is OS isolation and network egress control.

That response is correct. It is also in direct tension with the 0.00% headline and Cherny’s “largely solved” statement. Anthropic is simultaneously marketing Auto Mode as a solution to prompt injection while its security team correctly acknowledges it is not a security boundary. Both things cannot be true in a way that is useful to a developer deciding whether to run Claude Code on their workstation against untrusted content.

Why This Attack Surface Exists by Default

The decision to make Auto Mode the default matters more than the vulnerability itself. Here is why.

Before Auto Mode became default, a developer using Claude Code in a sensitive context would see approval prompts for tool calls. Those prompts are annoying. They interrupt flow. They are the reason Auto Mode was built. But they also create a forcing function: the developer has to consciously approve each action. In practice most developers click through, but the prompt at least surfaces what is happening.

Auto Mode removes that surface. The classifier decides. The developer sees a result.

Shipping a convenience feature as a default is a legitimate product decision. Shipping it with a benchmark number that implies the security problem is solved is a different kind of decision. It sets expectations that the feature cannot meet, and those expectations propagate. Developers read the headline, set their threat model accordingly, and run Claude Code against untrusted content in contexts where it should not run without a sandbox.

The 0.00% number is not wrong. It is misaligned with the question developers are actually asking when they read it.

There is a broader pattern here worth naming. AI coding tools have been making increasingly aggressive default-to-autonomy choices throughout 2025 and 2026. Cursor, Windsurf, Codex, and Claude Code have all shipped or experimented with modes that reduce or eliminate human approval steps. The competitive pressure to reduce friction is real: developers measure productivity in keystrokes saved, and approval dialogs are friction. The result is an industry-wide drift toward autonomy as the default, with security guarantees communicated through benchmark numbers rather than architectural constraints.

Architectural constraints are harder to market than benchmark numbers. A container boundary is invisible to the user; a 0.00% score is a press release. The incentives select for the latter.

What the Attack Teaches About LLM Agent Failure Modes

The specific technique — Python module shadowing via a file in the working directory — is a known Python quirk that predates LLMs entirely. Security engineers have written about it for years. What makes it interesting in the Claude Code context is how it interacts with the agent’s safety reasoning.

Claude refused the binary decoder. That refusal was the correct security response to that specific artifact. But the safety reasoning stopped there. Claude did not ask: if the binary is suspicious, why would I write a decoder that runs in the same directory as the binary? It did not apply the suspicion to the broader context, only to the specific artifact that triggered the refusal.

This is a pattern I expect to see repeatedly as agents become more capable. The agent’s ability to reason about individual decisions improves. Its ability to reason about the cumulative security implications of a chain of individually-reasonable decisions does not keep pace. Each step looks defensible. The sequence is an attack.

The researcher also noted a variant where the poisoned struct.py spawns a second Claude Code instance using claude -p. The child agent gets its own tool access and context, performs recon (whoami, uname, id), writes files outside the workspace. The attack does not just execute code — it creates another agent with its own agency. That is a qualitatively different threat model than traditional malware.

Traditional malware executes code. An attacker-spawned agent reasons. It can adapt to the environment it finds itself in, decide what to exfiltrate based on what is available, and behave in ways that are harder to signature because its actions are context-dependent rather than scripted. This is not a theoretical concern for 2028; it is a demonstrated behavior in August 2026 on a developer’s test machine.

The Agent Memory and Trust Chain Problem

The wunderwuzzi attack surfaces a fundamental architectural gap in how current agentic systems handle trust. When a human developer writes a Python decoder, they bring contextual knowledge to the task: they know what directory they are working in, what files are around them, whether any of those files arrived from an untrusted source. They apply that context to decide whether running their decoder from inside a downloaded archive is safe.

Claude Code in Auto Mode has no persistent trust model across tool calls within a session. Each tool call is evaluated on its own merits by the classifier. There is no mechanism that says: “this directory contains content that arrived from an external source, therefore any code executed from this directory inherits an untrusted provenance.” The classifier sees a Python execution command. It does not see that the execution environment was constructed by a server that was already behaving suspiciously.

This is the same problem that makes confused deputy attacks work in traditional systems. A program with elevated permissions gets tricked into performing operations on behalf of an attacker because the program does not distinguish between instructions from trusted and untrusted sources. Claude Code’s Auto Mode is the deputy. The downloaded archive is the attacker. The classifier is supposed to be the permission check, but it operates without the context that would make the check meaningful.

A more robust architecture would maintain provenance tracking across the session: marking filesystem objects, environment variables, and process invocations with their source of origin, and applying heightened scrutiny to operations that mix trusted and untrusted provenance. This is not a new concept — taint analysis has existed in static analysis and runtime security tools for decades. Applying it to agentic AI sessions is an engineering problem, not a research problem. It requires investment that produces no headline-worthy benchmark number, which is probably why it has not shipped.

The “agent spawning agent” variant makes this problem recursive. When the poisoned struct.py creates a child Claude Code instance, that child has its own tool call budget, its own context window, and its own interaction with the Auto Mode classifier. The parent’s compromise does not propagate to the child as a trust signal. The child is evaluated as a fresh session. An architecture that maintained provenance chains across agent boundaries would treat any agent spawned by a compromised execution context as inheriting that compromise. The current architecture does not.

The History of This Problem Is Longer Than the Benchmark

Indirect prompt injection as an attack class was documented publicly in early 2023, when researchers showed that text retrieved from external sources could manipulate LLM behavior during agentic tasks. Kai Greshake and colleagues published work on it in 2023 demonstrating attacks against Bing Chat and Copilot that exfiltrated conversation history through rendered content. The threat model has been understood for more than three years.

In that time, the responses from AI vendors have followed a consistent pattern. Early phase: the attack is acknowledged as theoretical. Middle phase: mitigations are shipped and described as substantially reducing the attack surface. Later phase: a benchmark number is produced. The benchmark number is used to claim the problem is solved. A researcher demonstrates an attack outside the benchmark. The vendor closes the report as working as designed because the new attack was outside the benchmark scope.

We are now in the later phase for Claude Code Auto Mode. The same sequence played out with earlier generations of Bing Chat, with Copilot, and with various early LangChain-based agents. Each iteration, the benchmark is more sophisticated and the claim is stronger. Each iteration, the attacks outside the benchmark continue to work.

This is not a criticism specific to Anthropic. Google, Microsoft, and OpenAI have all made similar claims at similar points in their product cycles. The pattern is structural. Vendors need to ship products. Products need security claims. Security claims need benchmark support. Benchmarks are finite. Adversaries are not. The difference in 2026 is that the agents have more capability — more tools, more autonomy, more ability to spawn child processes and make network calls — which means the consequence of a bypass has grown while the benchmark methodology has not. A prompt injection against Bing Chat in 2023 could exfiltrate chat history. A prompt injection against Claude Code Auto Mode in 2026 can establish a persistent C2 callback and spawn subagents. The severity trajectory is upward. The benchmark credibility trajectory is not keeping pace.

The Sandboxing Answer Is Correct and Insufficient

The right technical answer to all of this is sandboxing: run unattended coding agents in a container or VM, restrict network egress, do not expose SSH keys or cloud credentials to the agent. Wunderwuzzi says this. Anthropic’s security team says this. I am saying it here. It is the correct answer.

It is also not the answer most developers will implement before running Claude Code against a URL someone sent them in Slack.

The gap between “correct answer” and “what actually happens” is where most security incidents live. A 0.00% headline closes that gap in the developer’s mental model without closing it in reality. Developers who read the benchmark result and believe Auto Mode provides meaningful security guarantees against prompt injection will not implement sandboxing, because the problem appears solved.

Security marketing that reduces the perceived need for real mitigations is not neutral. It actively increases risk in the population of developers who would otherwise have been cautious.

The sandboxing answer is also worth examining more carefully because it implies an operational model that very few development teams currently have. A properly isolated Claude Code environment requires: a container or VM with controlled image, network egress rules that allow only the endpoints the task requires, no home directory or credential file mounting, audit logging of all tool calls and their outputs, and a process for reviewing that log. This is achievable at the infrastructure level. It is not how most developers currently interact with AI coding tools on their workstations.

The mismatch between the correct security posture and the actual deployment context is a product design problem as much as a security problem. If the correct deployment model for Claude Code Auto Mode requires infrastructure that 95% of the user base does not have, then the default should not be Auto Mode. Or Auto Mode should ship with first-class sandboxing built in. The choice to ship autonomous-by-default and solve-by-external-infrastructure is a choice, not a constraint.

Comparing the Evaluation Models

Both evaluations are methodologically legitimate for what they measure. The problem is that one was used to make a claim about the other’s domain. A 0.00% score on a private benchmark of 72 scenarios does not bound the attack success rate of a motivated adversary running a novel multi-hop chain. Claiming otherwise requires either confusion about what the benchmark measures or a deliberate choice to let the confusion persist because it is useful.

The “Working as Designed” Response Pattern

When Anthropic closed the report as “Informative” with “working as designed,” that response deserves more scrutiny than it typically receives in security disclosures. “Working as designed” is a legitimate response to a vulnerability report when the design intent is clearly documented, consistently communicated, and does not conflict with public statements about the feature’s security properties.

In this case, the design intent was not clearly communicated to end users. The 0.00% number and the “largely solved” statement created a reasonable expectation that Auto Mode was providing meaningful security against prompt injection. “Working as designed” is an accurate description of Auto Mode’s behavior — the classifier is doing what it was built to do. It is not a satisfying response to the gap between what the feature was marketed as doing and what it actually does.

The more defensible version of this response would be: “Auto Mode is a convenience feature, not a security boundary. We have updated our documentation to make this explicit and to provide clearer guidance on when sandboxing is required.” Whether that response will be forthcoming is a different question. Vendors rarely issue corrections to security marketing claims unless compelled to by regulatory pressure or significant public incident.

What Would Change This Dynamic

The only thing that changes this is evaluation methodology that is adversarially constructed from the start. A benchmark designed to be hard to game would:

• Test the model’s ability to handle scenarios outside its training distribution
• Use adaptive attacks that evolve based on initial results, not a fixed scenario set
• Measure outcomes — files written, network calls made, processes spawned — rather than classifier decisions
• Include multi-hop chains where each individual step is individually innocuous
• Be constructed by red teams that are explicitly trying to win, with access to the same frontier models the attacker would use
• Be published in full so the methodology can be independently reproduced and extended

This is expensive to build and expensive to run. It also cannot be summarized as a single number, which makes it harder to use in marketing. Those two facts explain why it does not exist at scale yet. The incentives favor the headline number.

There is also a disclosure timing problem. The researcher sent this report to Anthropic before publication. Anthropic did not initially respond through the first channel, then closed it quickly after the second channel. That is not a timeline that inspires confidence in the vendor’s process for translating security research into product changes. A vendor that responds to a demonstrated RCE against its default mode by closing the report as informative and working as designed is telling the researcher — and by extension the security community — that this class of finding is not a priority. That signal affects who reports future findings and how.

Until adversarial evaluation becomes standard, every “X% attack success rate” number for LLM agent security should be read with a scope qualifier: on this benchmark, against these scenarios, with this attacker model. The moment you step outside that scope, the number tells you nothing.

Predictions

Within six months, at least three more novel prompt injection chains against Claude Code Auto Mode will be publicly documented, each using techniques not present in the Trajectory Labs benchmark. The 0.00% figure will not be updated or retracted; it will simply stop appearing in new marketing material as the community moves on to the next benchmark cycle. This is the standard lifecycle for AI security claims: the number gets superseded by events, never formally corrected.

Anthropic will ship sandboxing infrastructure — container-based, most likely — and position it as the correct deployment pattern for Auto Mode in production environments. This will be technically accurate. It will also be an implicit acknowledgment that the current default configuration is not appropriate for untrusted content. The framing will probably be “enterprise-grade security” rather than “the baseline security that Auto Mode should have shipped with.” Watch for whether the sandboxed mode becomes the new default or remains an opt-in feature requiring infrastructure setup. That distinction will tell you whether Anthropic is solving the problem for the median developer or for the enterprise customer who already has container infrastructure.

The “largely solved” framing will not survive 2026. Prompt injection is a property of the interaction between an agent’s goal-directed behavior and an adversarial environment. As agents become more capable and autonomous, the attack surface grows proportionally. A classifier that achieves 0.00% against 72 known patterns will continue to fail against motivated adversaries who have access to frontier models to help them build attack chains — Codex helped build the payload in this case. The attacker’s tools improve at the same rate as the defender’s. That is a permanent structural asymmetry in this threat model, and a number on a benchmark does not make it go away. The most dangerous thing about the “largely solved” framing is not that it is wrong — it will eventually be corrected — but that in the interim it discourages the kind of adversarial research that is the only mechanism capable of producing accurate knowledge about where the real attack surface lies.

The more important long-term question is not whether Auto Mode can be bypassed — it can, and it will continue to be bypassed — but whether the security evaluation methodology for agentic AI systems will mature before a significant production incident forces that maturation on an accelerated timeline. The current approach, where vendors commission fixed-scenario benchmarks and use them to claim solved status, is not adequate for systems that can write and execute code, make network requests, and spawn child agents. The gap between what these systems can do and what the current evaluation framework can measure is the actual security risk. The 60-80% attack success rate is just the most recent piece of evidence for a gap that has existed since the first version of Claude Code shipped. The next piece of evidence will be more interesting, because the attacker now has better tools to build it with.