ConduitOne gateway for every model call, routed by complexity and guarded both ways.
When every team calls models directly, nobody controls cost, PII, or vendor lock-in. Conduit puts every call through one OpenAI-compatible process with an order the code enforces.
An enforced pipeline
- Auth. API key to team, quota, and policy.
- Ingress guard. PII redaction and prompt-injection detection before any model sees the request.
- Router. Complexity to tier to a model chain, with fallback down the chain instead of failing.
- Provider adapters. Vendor SDKs live only here, and a CI test enforces it, so swapping providers is a config change.
- Egress guard. Re-scans the response and restores tokens.
- Telemetry. Tokens, cost, latency, trace, and verdict for every call.
The stages are an enum, and running one out of order raises PipelineOrderError, so the router can't jump ahead of the ingress guard.
Routing that pays for itself
A two-stage router sends most traffic to cheap models. A zero-cost heuristic resolves 80%+ of requests, and a cached LLM tie-break handles the ones with no lexical evidence either way. One fix separated the instruction from pasted material: a quote pasted under a short question used to push a trivial request up to the complex tier.
Evals as a merge gate
- Golden datasets, rubrics, and an LLM-as-judge suite; a quality drop fails the pull request.
- CI runs lint, format,
mypy --strict, the vendor-SDK guard, unit and integration stages, a coverage gate, and a zero-key Docker smoke test. - Evals run offline at $0. Deploy is manual only, never on merge.
Reading the numbers honestly
The 88.6% routing figure and the 100% injection detection rate are upper bounds: both were reached by fixing rules after reading the failing cases. A held-out probe caught one over-broad and one too-narrow injection rule before shipping. The unfitted check is the held-out property test.
Stack
- Python
- FastAPI
- LangGraph
- Anthropic
- Ollama
- sqlite-vec
- Docker Compose
- pytest
- mypy
- ruff
- GitHub Actions