piiguardA PII leak guard for sandboxed AI agents that hardens itself.
Sandboxes like NVIDIA OpenShell decide where an agent may send data, but not what is inside it. piiguard is the content layer that closes that gap, and it gets better on its own.
Picked from 109 projects at the Recursive Agents Hackathon (Cerebral Valley, Crusoe, NVIDIA, You.com), San Francisco, October 2026. 65 commits, all written during the event.
The problem
An agent that is allowed to call a search API can paste a customer's SSN into the query, and the sandbox lets it through because the host is on the allow-list. Nobody broke a rule. The data still left.
Network policy answers where. Something has to answer what, without breaking the agent's task.
How it works
- Guard proxy. Sits between the agent and allowed egress. It swaps PII for reversible tokens before anything leaves the sandbox, then restores the real values in the response so the agent still finishes its job. Provider keys stay in the proxy.
- Red team. Nemotron 3 Super on Crusoe, plus deterministic encoders, generates attacks that try to sneak PII past the guard: base64, hex, reversed strings, spelled-out digits, unicode tricks, and values split across lines.
- Coach. Nemotron 3 Super reads what leaked and rewrites the guard's rules and the prompt for the Nemotron 3.5 Lightning span classifier.
- Gate. A change is kept only if the validation leak rate drops and false positives don't rise. Accepted changes are committed and tagged
gen-N; rejected ones are reverted and logged. A security reviewer reads a regex diff, not a black box. - Score. Leak rate is measured on a held-out attack set the coach never sees, comparing generation 0 with generation N.
Results
Held-out attacks (n = 30) that the coach never saw:
| System | Leak rate | False positives |
|---|---|---|
| NemoClaw network policy only | 100.0% (30/30) | n/a |
| Presidio, default | 53.3% (16/30) | 64.0% |
| Presidio, scoped | 70.0% (21/30) | 4.0% |
| piiguard gen-0 rules | 76.7% (23/30) | 0.0% |
| piiguard gen-14 rules | 16.7% (5/30) | 0.0% |
| piiguard gen-14 rules + classifier | 0.0% (0/30) | not reported |
On NVIDIA's public Nemotron-PII dataset (290 records, 2,630 spans), the routed setup with an industry pack and classifier reached 92.8% recall on structured identifiers, against 68.5% for default Presidio.
Design decisions
- Reversible tokens, not deletion. Redaction that breaks the agent's task gets turned off. Tokens keep the request useful and the values out of the third party.
- Improvement only behind a gate. The coach can propose anything; only changes that lower leaks without raising false positives survive.
- Auditable generations. Every accepted generation is a git tag, so rules can be reviewed, diffed, and rolled back.
- Pruning has to be free. Removing 10 apparently dead rules raised held-out leaks from 16.7% to 26.7%, so that change was not committed.
- Shape memory. A ledger of caught identifier shapes acts as a regression gate, so later generations can't quietly forget what earlier ones learned.
Where it falls short
- Presidio still leads on semantic PII like names and places, and on overall recall. Names are caught only when the classifier is on.
- The loop was tested against a synthetic red team, not production traffic.
- Single host with token auth; a team deployment needs a real identity layer. Streaming replies are converted rather than passed through.
- The OpenShell supervisor middleware path is a research preview, verified against the published proto rather than a running gateway.
Stack
- Nemotron 3 Super
- Nemotron 3.5 Lightning
- Crusoe Managed Inference
- NVIDIA NemoClaw
- OpenShell
- You.com search
- Microsoft Presidio (baseline)
- FastAPI
- gRPC
- Python 3.12
- uv