↓ Skip to main content
  1. Signal Loop/

the check that gave it away: an ai agent outs its own operator

·740 words·4 mins·
Author
Prakash Ayyanagoudar
Table of Contents

An attacker’s AI agent gave away its own operator this week. Not through a bug. It followed an instruction, correctly.

That is the story worth your time. Everything else in my feed was a launch.

the tell
#

A SANS Internet Storm Center diary by Renato Marinho, published 11 September, describes an offensive coding agent that mistook a honeypot for a real, free LLM backend.

The request carried far more than an exploit payload. The honeypot received about 43 KB: a large AGENTS.md, an offensive playbook, infrastructure notes, reconnaissance scripts, collected API keys, previous targets, and parts of the agent’s working history.

One instruction told the agent to verify its proxy was active before attacking. That check exposed the operator’s direct, unproxied egress IP. The agent leaked it by doing exactly what it was told.

My read: nobody built this to fail. Every instruction was reasonable in isolation. What was missing was judgment, the ability to ask whether a check safe to run at home is still safe to run in front of a stranger’s server. SOC teams have the same gap with a well-behaved copilot that closes a ticket correctly without knowing the incident three rows up is still open.

what i’m ignoring
#

  • Any AI product pitched this week as a “decision engine”. Positioning until someone outside the vendor tests it.
  • Pledge-style headlines about labs slowing down. A pledge with no enforcement and no date is a press release, not a control.
  • Lab gossip. Not a SOC problem.

the real one
#

Pillar Security reported an active MCP supply-chain campaign they call Deadbugz. On 10 August, in a 74-minute window, one GitHub account opened 23 pull requests across unrelated AI, MCP and developer-tool repositories. Each added a server called productivity-suite.

The server offers two harmless tools, format_text and summarize, and behaves normally. Once it reaches three calls, its tool listings change and steer the connected agent toward SSH keys, AWS credentials, shell history and Kubernetes config.

Auto drivers have a line for this: meter kharab hai. The meter is “broken”, so you cannot check the fare. A tool description you cannot watch change is the same meter. Two clean calls prove nothing, because the server decides when to show you what it really does.

What I’d watch:

  • any MCP server pull request from an account with no history in your org’s repos
  • a tool’s description text changing between calls, not just between releases
  • an agent reaching for credentials a formatting tool has no business touching

Caveat: when Pillar published on 12 August, 19 of the reviewed pull requests were closed and four were still open. Do not assume the closures cover you. Check your own configs. If your team reviews MCP servers by reading the README once, this is the argument for reading the code.

the part that stays
#

Security angle: both stories say the same thing from opposite sides of the wire. An agent that verifies things literally, without asking whether the verification itself is safe, will hand someone a win eventually, whichever side it’s running for.

Human angle: neither agent here made a decision. One leaked its own operator by checking a proxy correctly. The other waited three calls before turning honest into hostile. Both just ran instructions, all the way through.

Honestly? I don’t have a tidy conclusion for what that means once a SOC hands one of these more access next quarter. I don’t think anyone fully does yet.

do this week
#

  1. List every MCP server currently connected in your environment and who approved it. An unowned list is the finding.
  2. Pick one reference value, like the proxy check in Marinho’s writeup, and check whether anything in your environment would leak it the same way if it were watched instead of trusted.

Which one worries you more: the agent that gives up its own operator, or the one that waits three calls before it shows you what it actually does?

a check that is safe at home is a leak in front of a stranger.

sources
#

Stay patched. Stay paranoid. Maximum effort.

i take help from ai to optimise what i have learned and share it with you. the reading, the lens and the calls are mine. weekly, so you stay updated without the scroll fatigue.

Related