Green Tests, Inert in Production

The browser automation arc did not fail because nobody cared about safety. It failed in the more dangerous way: the safety story looked green while parts of the production path were inert.

The shipped shape is deliberately narrow. Browsing is default-off. Browser tools are enabled only when an agent asks for read-only browsing, the tenant allowlist is non-empty, and a live enforcing egress proxy exists for that agent runtime. That three-way gate keeps the feature absent rather than degraded when containment is missing.

Containment is a system, not a toggle

The per-tenant proxy is the policy brain. It evaluates tenant allowlists, blocks private and loopback destinations, resolves hosts before connecting, and connects to the validated IP rather than resolving again later. The browser receives only the proxy endpoint — passed in through a single environment variable — so normal HTTP and HTTPS traffic must pass through tenant policy.

The proxy is not the only boundary. The launcher runs the browser under a host nftables/cgroup profile that permits outbound TCP only to the proxy endpoint, drops UDP, and rejects other TCP. Chromium flags such as --disable-quic and --force-webrtc-ip-handling-policy=disable_non_proxied_udp are defense in depth, not the containment boundary. If a protocol bypasses the HTTP proxy, the host profile is supposed to catch it.

That distinction matters because configuration can be present without being effective. The lesson from the arc was not just to render a safer config. It was to prove that the effect exists on a live host.

The live-host hardening wave

The first hard lesson came from a live-host hardening change. Running hosts could carry the config scripts without the host packages and primitives that made containment real. The same wave also fixed nftables-rule quoting and cgroup-v2 behavior so rules were not merely installed, but matched actual traffic. The live tests were strengthened to inspect non-zero enforcement counters instead of accepting rule presence as proof.

A readiness-probe fix came next, closing a fail-closed bug in the probe itself. The probe was invoked in a way that turned a shell conditional into a broken privileged command, so even a provisioned host could look unready and drop browsing. The fix made the probe execute under an explicit shell, with timeout and retry behavior that separated genuine absence from probe error.

Finally, a proxy-lifecycle fix covered a subtle bug: a routine client tunnel recreate could tear down the egress proxy the browser was actively using. The symptom was a browser fetch failing through a proxy that policy expected to exist. The fix separated client-tunnel shutdown from browser-proxy shutdown so transient client transport work did not kill the containment path.

Test the effect, not the config

The useful test was not "did we write firewall rules?" It was "does a confined browser child reach only the proxy, fail non-proxy TCP, fail UDP, and leave counters showing the rules actually matched?" That framing came straight out of the browser tool's live-host coverage: browser automation is safe only when the configured proxy, host profile, and runtime process behavior line up on the machine that runs the agent.

The same idea applies above the network layer. Approved actuation tools are configured as permission requests so every click or type can be audited. Observability writes are best-effort and off the decision path, so an audit outage cannot make a denied connection allowed. Those are effect-level contracts, not decorative settings.

The attribution episode

The pattern repeated with denial attribution. The system initially tried to learn browser intent from permission resolution, but read-only navigation tools are configured as allowed tools, so they auto-run without a permission request. In production, the signal the code listened for never arrived. An attribution fix moved capture to the agent runtime's session tool-event stream for the read-only navigation tool, scoped the intent index per agent runtime, and persisted an attribution method so a heuristic recent-active match does not look like fact.

That episode is the postmortem in miniature: passing synthetic events are not the same thing as observing the production effect. The safer pattern is to capture the signal at the point the live system actually emits it, record uncertainty plainly, and keep attribution off the containment decision path.

What this story is not claiming

This post is about the read-only, actuation, egress, and audit containment story. Credentialed browsing has a separate explicit-consent and host-bound storage-state model. The per-agent credentialed frontier belongs to a later in-flight line of work for this narrative scope, not to the original shipped containment proof described here.

The durable lesson is smaller and more reusable: never stop at green tests that inspect configuration. For an agent with a browser, the only result that counts is a live effect that stays contained when the browser, proxy, host network, and attribution path all run together.