Skip to main content

Blog / AI & security

My agent hardened the container "perfectly." The API keys walked out anyway.

Abhimanyu Selvan7 min read

I asked a coding agent to containerize an internal tool. It's an eval harness that runs a pack of tests against a model and grades the answers. I wanted it to run in a container so a pack couldn't touch the rest of my machine.

The agent did a clean job. Read-only root filesystem, every Linux capability dropped, no new privileges, tool commands switched off. If you review the pull request on a busy afternoon, you see a locked box and you approve it. I would have.

It also lets any pack I run read my API keys and POST them to a server in another country, straight out of that locked box. Both of those are true at the same time. That gap, between a container that looks hardened and one that's hardened against the thing that matters, is the whole post.

The short version:

  • What the tool does. A pack can ship a check written in Python, and the tool runs it. That's a real, wanted feature, not a bug.
  • What the agent built. A container with a read-only filesystem, all capabilities dropped, and no new privileges. Genuinely good hardening.
  • Why it doesn't help. The attack reads a file and opens a socket. Neither needs a capability or a writable filesystem. The defenses guard doors the attack never uses.
  • The contradiction. The compose file's own comment says "give the pack as little as possible." Fifteen lines later it mounts the entire repository, keys included, into the container.
  • The result. A pack I was sent read my .env and shipped it to a box I control in another data center. The eval run finished 100% green.
  • The fix. Not more container flags. You can't flag your way out of running someone's code. You move the code somewhere with nothing to steal.

The feature that's completely fine

Start with something harmless, because the attack wears its shape.

One of the checks the harness offers is called script. You point it at a Python file in your pack, name a function, and the tool imports that file and runs it against the model's answer. It's for things you want measured exactly, in code. The kind of rule a language-model judge would only estimate.

Here's a real one. It counts the characters in a reply and passes if it's under a limit:

def max_chars(output, ctx):    limit = ctx["args"]["limit"]    n = len(output.strip())    return {"passed": n <= limit, "detail": f"{n} characters", "value": n}

A test wires it up in one line of YAML:

checks:  - script: { file: ../../checks/text.py, function: max_chars, args: { limit: 300 } }

This is ordinary and useful. The replies are text messages, text messages have a length limit, and a counter in Python is free, exact, and never changes its mind. Packs ship Python checks, the tool runs them, everyone wants this. Hold onto that shape.

The container that looks safe

Now the container the agent built. Here is the hardening, straight from the compose file:

read_only: truecap_drop: [ALL]security_opt:  - no-new-privileges:truecommand:  - serve  - --no-tool-commands

Read-only root filesystem. Every Linux capability dropped. No process can gain new privileges. Tool commands off. Four real defenses. On a quick review this reads as a locked box.

And the agent knew what it was guarding against. Four lines above the hardening, it left itself a comment:

# A pack can run code (script checks): give it as little as possible to work with.

"Give it as little as possible." Then, fifteen lines later:

volumes:  - .:/work

It mounts the entire repository into the container. Your whole project. Your .env files. Your keys. Read-write, because a bind mount is read-write unless you say otherwise, and read_only: true only covers the container's own filesystem, not what you mount into it.

"As little as possible" became "everything," and nothing flagged it. The two lines contradict each other inside the same file, and the agent wrote both. The docs even say the quiet part out loud, serve only packs you trust, a sentence everyone nods at and nobody acts on.

Why the defenses are theatre

Here is where the pack's code actually runs, in the tool's script runner:

spec.loader.exec_module(module)   # import the pack's Python...fn(request["output"], request["ctx"])   # call it

From the moment exec_module runs, the pack's code is executing in the tool's own process. Python has no sandbox inside a process. That code can do anything the tool can: open any file it can reach, open a network socket. And here is the quiet mismatch that defeats all four flags. The two things the attack does, a file read and a socket, need nothing the container took away.

Reading a file needs no capability. Opening a socket needs no capability. Neither writes to the root filesystem, so read-only doesn't apply. Neither escalates privilege, so no-new-privileges doesn't apply. The tool-commands switch governs a different feature entirely. Four real defenses, and every one of them guards a door the attack walks straight past.

The lock was fitted to the wrong door. A strong lock on the wrong door is still an unlocked house.

The real boundary

The container drew a wall around itself. But the boundary that mattered was never the container's edge. It runs inside the container, between me, who owns the keys, and the pack, which a stranger wrote. That line is the one with no wall on it.

The container is hardened on every side. The boundary that matters runs down the middle of it, and there's no wall there.

The agent hardened the outer box because that's the boundary a container has. It never modeled the boundary a container doesn't have: me versus the author of the next pack I run.

The attack, live

Here's the whole thing on video. The written walk-through is below it.

So here's a pack someone sends me. The malicious check is the same shape as max_chars: same script wiring, same (output, ctx) signature. The eval tool can't tell them apart; both just say "run the Python this pack shipped." The only difference is what's inside the file.

It doesn't even know where my files are, so first it looks around. It reads its own path, climbs the folders, and looks for .env files along the way:

here = __file__                       # the tool handed it an absolute pathd = os.path.dirname(here)for _ in range(5):                    # climb the ancestors, no hardcoded path    found += glob.glob(os.path.join(d, "*", ".env"))    d = os.path.dirname(d)

It found my real keys at /work/example-pack/.env. It discovered the mount by standing on it. Nothing about my setup had to be known in advance. From there, a dozen more lines open a socket and POST those bytes to a server I control in another data center. I won't paste that part. The shape is the point, and you've already seen it.

I ran it against a local model, with a canary .env swapped in so nothing real moved. On one screen, the eval ran. On the other, my server abroad. I started the run, and the canary keys arrived on the remote box, across the internet, out of a container with a read-only filesystem and no privileges.

The path out: the pack's check is run, reads the mounted .env, and opens a socket to a server outside the wall. Every step needs nothing the container removed.

And the tool? Green. Every check passed.

4 hardening flags in the file
0 that stop this attack
1 mount the whole repo, read-write
100% the run still passed

If that had been a real key, it would be gone, and I'd have had no idea. The run looked exactly like a healthy one.

The fix is not another flag

You cannot flag your way out of running someone else's code. Every container control I could add still leaves the pack's Python executing next to my keys, and reading a file is not a thing a sandbox flag forbids.

So you stop trying to make the room safe to run code in, and you make the room empty. The pack runs somewhere with nothing worth stealing: a separate container, no network, only the one pack mounted read-only, and no secrets anywhere it can reach. The keys live somewhere the runner has no path to.

Today the pack and the keys share a room. The fix gives the pack its own empty room, no network and no secrets, and keeps the keys out of reach.

Then the same attack runs, finds its own path, globs for .env, opens its socket, and lands in an empty room. Nothing to read, nowhere to send it. The exploit still "works." It just comes back with nothing.

I've since built this, and written up how it's wired, the second container, the copied pack, and the one case it still doesn't cover, in the follow-up post.

Secure against whom

Here's what I keep coming back to. The agent didn't write a bug. Every line it wrote is correct. read_only: true does what it says. cap_drop: [ALL] does what it says. It optimized for exactly what I asked: make it work, make it look hardened. It even documented the threat in its own comment.

What it didn't do is threat-model the one boundary that mattered, me versus whoever wrote the pack, and then check its own work against that boundary. It hardened the container because a container is the thing in front of it to harden. "Looks secure" and "secure against whom" are different questions, and an agent handed the first one will answer it well enough to sail through review.

That second question is still yours. If you run untrusted packs, plugins, or any agent-generated code next to your secrets, the container flags on the box tell you almost nothing. Go look at what you mounted into it.

Takeaways

  • A hardened container is hardened against the host. It does nothing about code you deliberately run inside it reading what you mounted in.
  • read_only: true doesn't cover bind mounts. A mounted directory is read-write unless you say :ro, and the host's files are reachable either way.
  • Reading a file and opening a socket need no capability and no write access, so capability-dropping and read-only roots don't touch an exfiltration attack.
  • When the real boundary runs inside your sandbox, you versus the code you're running, isolate the code with nothing to steal. Don't try to list every bad thing it might do.
  • Agents optimize for "looks secure." Ask them, and yourself, "secure against whom," and review the answer against that.

Keep reading

All posts