The short answer
Auditing an LLM agent means being able to replay what it did, re-score it with judges you choose, and confirm that every published number regenerates from raw files. Gubernaut publishes its own record on those terms: transcripts and four-family judge panels with SHA-256 provenance, the extraction scripts, and a replay cockpit that steps through the sealed runs turn by turn.
An agent is auditable when three things are recoverable after the fact: what it received, what it decided, and under what conditions it answered. Sampled text alone gives only the last of those, and second-hand. A deterministic control layer adds a decision trace that replays identically, so an incident review can diff behaviour instead of arguing about it.
An audit should ask for the raw transcripts of both arms verbatim, the judge panels with provenance hashes, the scripts that regenerate every published number, a pre-registration dated before data collection, and the nulls stated beside the headline. Gubernaut publishes all five under CC-BY-4.0.
Without the ungoverned arm there is no comparison, only an assertion.
SHA-256 per file, so a later edit changes the hash and becomes visible.
Extraction and statistics code that turns raw panels into every published figure.
Design, scoring protocol and analysis fixed in writing before any data existed.
A record with no null and no disclosed limit has usually been filtered rather than measured.
The replay cockpit on the research page steps through the sealed runs turn by turn, both arms side by side, with the controller telemetry as recorded at generation time. The cockpit is a recorded run replay with no live API, so what it shows is the record rather than a fresh generation.
Open the cockpitRe-scoring runs against the published transcripts. Every reply ships verbatim with SHA-256 provenance, and the extraction scripts that turn raw panels into published numbers are in the public repository. A different panel or a different rubric can be applied to the same replies and compared against the sealed matrices.
Data and scriptsInstall the proxy, point your client at it, and run your own adversarial sequence through both arms. A supervised audit session is also available on request: your probes, both arms, the controller telemetry on screen, and the same judging protocol the sealed record uses.
Request an audit sessionThe release gate covers concurrency, fail-closed behaviour and cross-implementation parity before a version ships. Version 1.0.1 recorded no posture cross-contamination under 240 concurrent mixed requests, no fail-open leaks, and bit-exact agreement between the Rust core and the Python reference.
| Gate | What it asserts | Result at 1.0.0 |
|---|---|---|
| Concurrency | 240 concurrent mixed hostile and benign requests | No cross-contamination, no dropped requests, upstream hit 120/120 |
| Fail-closed | Neither SDK reaches the upstream when the proxy is down | Canary upstream recorded zero hits in every dead-proxy case |
| Parity | Rust core against the Python reference | Bit-exact, checked against golden traces in the repository |
| Edge soak | 10,000 ticks in workerd, a Node worker and Node main | Bit-exact, flat memory, no latency degradation |
Source: HARDTEST v1.0 release gates, 2026-07-24.
Across 4 frontier model families the regulated arm was calmer in 15 of 16 generator×judge cells by sign (11/12 off-diagonal), 13 of 16 at p<.05, with homeostatic recovery replicating in 4/4 families. One cell, GPT generating and Gemini judging, returned a flat null at −0.04.
Source: Pre-registered cross-family evaluation. Paper: arXiv 2607.24339, DOI 10.5281/zenodo.21303518.
Any control claim that reports a headline and nothing else. The Gubernaut record carries a flat null at −0.04, three cells short of p<.05, five documented failure modes, and a sensor-recall limit on calmly worded hostility, all published in the same release as the headline and reproducible from the same files.
How to reproduce itLast reviewed 2026-08-07 · paper arXiv 2607.24339, DOI 10.5281/zenodo.21303518
Keep going
What deterministic control means for a language model, why the control layer belongs outside the model, how determinism is verified, and what to install today.
Read the guideWhy LLM agents loop, what a runaway loop costs across seven measured configurations, how to detect one, and where a spend cap stops being enough.
Read the guideWhy prompt injection works, what a token-free control boundary is, exactly what it does and does not protect, and how to test the boundary yourself.
Read the guide