Blog / AI & security
How I actually sandboxed untrusted code
My eval tool runs test packs against a model and grades the answers, and a pack can ship its own check written in Python. The tool runs that Python. That is the feature, and it is also the problem: a pack is someone else's code, and I was running it right next to my API keys.
A few days ago I watched exactly that go wrong. A coding agent had containerized the tool with real hardening, a read-only filesystem, every Linux capability dropped, no new privileges, and a test pack still read my keys straight out of that container and POSTed them to a server in another country. The flags were all real. The attack wanted none of them. It read a file and opened a socket, and neither needs a capability or a writable disk. (The full story is here, but you don't need it to follow this.)
The takeaway was a claim with no code behind it yet: you can't flag your way out of running someone else's code, so you move the code somewhere with nothing worth stealing. This post is that somewhere, built and shipped.
The short version:
- The move. Pack code (a
scriptcheck) no longer runs in the tool's process. It runs in a second container with no network, no environment, and no repository. - What that container sees. Only a copy of the one pack being checked, mounted read-only. The copy is made without the pack's
.envfiles, its symlinks, and its.git. - How the check gets there. The tool sends each check to that container over a Unix socket and waits for the result.
- The safety property. If the second container can't be reached, the check errors. It never falls back to running next to the keys.
- What it still doesn't cover. Running the tool directly on your laptop, outside the container, still executes pack code on your machine. The isolation is the container.
Where the code was running
The dangerous line was one call. A script check is a Python file the pack ships, and the tool ran it with spec.loader.exec_module, in the tool's own process, next to everything the tool could reach. The compose file mounted the whole repository, .env included, into that process's container. So "the pack's code" and "my keys" were the same room, and no container flag separates two things in the same room.
The fix is not to make that room safer. It is to run the pack's code somewhere else, somewhere that holds nothing it would want.
A second container with nothing in it
There are now two containers. The first serves the dashboard and talks to the models. It still has the repository (read-only) and the API keys, because it is the trusted part. It never runs pack code anymore.
The second exists only to run script checks. Here is its entire slice of the compose file:
checks: <<: *hardened network_mode: none volumes: - sandbox-files:/sandbox/files:ro - sandbox-socket:/sandbox/socket
network_mode: none is the line that matters. There is no network in that container at all, so the exfiltration step from that attack, open a socket and POST, has nothing to open. It also inherits no environment, so there are no API keys, and the repository is not mounted, so there is no .env on disk to find. On top of that it keeps the same hardening as the first container: read-only root, all capabilities dropped, no new privileges, memory and process limits.
Copy the pack, don't mount it
My first instinct was to mount the one pack into that container, read-only. That is not enough. A pack can carry its own .env, and a mount would hand it straight over. A symlink inside the pack could point back out at the repository. So the tool doesn't mount the pack. It copies it, and strips what the runner has no business seeing:
.envand.env.*files, the keys.- symlinks, which could point at anything.
.git.
The copy lands in a small in-memory folder shared with the checks container, placed at the pack's own path so relative imports still work. Only changed files are copied, and files the pack no longer has are removed, so it stays cheap on repeat runs. The test that guards this writes a canary .env, a .env.local, a .git, and a symlink pointing at the .env, runs the copy, and asserts the result contains exactly the two real files and nothing else.
Fail closed
The property I cared about most is what happens when something goes wrong. If the checks container is slow, that's fine, the check waits. But if it isn't there at all, the tool must not quietly run the pack's code in its own process instead. That is the exact failure that turns a locked door back into an open one.
So it doesn't. When the socket can't be reached, the check comes back as an error that says the checks container isn't running, and the pack's code does not execute. A test pins this down: it points the tool at a sandbox with nothing listening, runs a check whose only job is to create a file, and asserts both that the check errored and that the file was never created. The code did not run. Fail closed, on purpose.
The same attack, now
I ran that same pack again, unchanged. It climbs the folders looking for .env files, the same reconnaissance as before. This time it is standing in the copy, and the copy has no .env, and there is no repository above it. It finds nothing. Then it tries to open its socket to the server abroad, and there is no network, so the connection never forms. The exploit runs to the end and comes back with nothing. An empty room does to it exactly what a stronger lock never could.
What this doesn't fix
The honest footnote. If you run the tool directly on your machine, not in the container, there is no second container to send checks to. In that case the pack's code still runs on your laptop. The tool does strip the API keys out of the subprocess environment and points its home directory at a throwaway folder, so a careless read of ~/.config or an environment variable comes back empty. But it is not a sandbox. That process can still read files on disk and use the network. A full sandbox without a container means OS-specific machinery (seccomp, namespaces, and the rest), and I would rather be honest than pretend a half one is whole.
There is a smaller gap inside the isolation too. Every check shares the one checks container, so a script can leave a process running there that changes a later check's result, until the next down clears it. The room has nothing worth stealing, so this is a scoring problem, not a key-theft one, but it is real: a pack could lean on its own grades. It is on the list.
So the guarantee has a shape: the container path isolates untrusted packs. The local path reduces the blast radius and asks you to serve only packs you trust. That is the same advice the docs always gave, except now the shipped, containerized path actually enforces it.
Takeaways
- You can't sandbox code with the flags on the box it runs in. Move the code to a box that holds nothing it wants.
- "No network" is the cheapest, bluntest control you have against exfiltration, and most code that should be sandboxed doesn't need a network at all.
- Copy untrusted input, don't mount it, if the source directory can carry secrets or symlinks next to the thing you actually need.
- Decide what happens when the sandbox is missing before you ship. If the answer is "run it anyway," you don't have a sandbox.
- Write the limitation down. The attack you don't cover is the one your reader needs to know about most.