The error is doing your thinking for you.

There's a quiet pattern most teams have slipped into without naming it. You ask an AI coding tool for a command. It gives you one. You run it. It throws an error. You paste the error back, it corrects itself, you run it again. Three or four rounds later it works, and you move on.

It feels productive. It isn't, quite — and the reason is the part nobody notices: the error message is doing your thinking for you.

The CLI is acting as a free verifier, but it only checks a narrow band — syntax, whether a resource exists, whether a permission is spelled correctly. It says nothing about whether the design is correct, whether it follows least privilege, whether it's the idiomatic pattern, or what it costs. So without realizing it, you've set your quality bar to does not throw — the lowest bar there is.

Here's the trap in one sentence: a command that succeeds feels like a command that's correct, and those are different claims. When something goes red, you scrutinize it. When it goes green, you don't. The dangerous output isn't the one that errors — that one you catch. It's the one that runs cleanly and is quietly wrong: the over-broad role, the public bucket, the unpinned image. That output passes every test and fails the audit.

In a regulated environment, that last line isn't a turn of phrase — it's a finding. Over-provisioned IAM that "works" is exactly the class of thing that survives every check in the build and then surfaces in the post-incident review, the access recertification, or the examiner's sample. The error caught the typo. Nothing caught the design.

And the IAM grant is the visible version, because it leaves a trace in a policy file. The same failure mode lives one layer up, where it's harder to see: a Claude-drafted discharge summary that reads perfectly and quietly contradicts the chart; a contract abstraction that's fluent and misses the carve-out; a model-risk memo that cites a control that isn't actually in place. Fluent and wrong clears every spell-check and fails the only review that matters. Whether the output is a shell command or a clinical paragraph, "it ran" was never the same claim as "it's right."

The problem, demonstrated

A real task: grant a developer group access to BigQuery and Google's hosted MCP tooling, role-based, least privilege. Watch the same request run two ways. (The commands are illustrative and lightly edited for length.)

Run A — the tool as most people use it today

$ gcloud iam roles create AgenticDev --project=$PID \
    --permissions="bigquery.jobs.create,run.developer,..."
ERROR: Permission run.developer is not valid.   # run.developer is a ROLE, not a permission

Paste the error back. It corrects. Run again.

ERROR: Permission mcp.tools.call is not supported in custom roles.

Paste the error back. It corrects — drop the permission, use the predefined role. Run again.

$ gcloud projects add-iam-policy-binding $PID \
    --member="group:agent-developers@yourdomain.com" --role="..."
ERROR: Group agent-developers@yourdomain.com does not exist.

The placeholder yourdomain.com was never substituted, and the group was never created. Three rounds in, you fix both, and it finally works.

But look at what also shipped in that session, with no error at all:

$ gcloud projects add-iam-policy-binding $PID \
    --member="group:..." --role="roles/iap.egressor"   # "for agent-to-agent comms"
# runs clean. green. done.

That binding executed perfectly. It's also conceptually wrong — IAP egressor governs reaching IAP-secured endpoints, not "agents talking to each other," which is an open HTTP protocol with no IAM surface of its own. It's an unjustified, over-broad grant. Nothing went red, so nobody questioned it. That's the failure that matters, and the error-driven loop is structurally blind to it.

Run B — the same request, grounded and gated

[GROUND]  reading current state before generating…
          account: dev-lead@acme.com   project: acme-agents-prod
          checking group exists… ✗ agent-developers@acme.com NOT FOUND
          → halting: will not emit a binding to a non-existent principal.
            Create the group first, or confirm the correct address.

No placeholder reaches your terminal, because the tool read the world instead of assuming it. Once the group exists:

[GENERATE]  proposed: custom role (BigQuery + Cloud Run perms)
            + predefined roles/mcp.toolUser  + roles/bigquery.dataViewer

[CRITIQUE]  architect review:
            ✓ two-gate MCP pattern is idiomatic
            ⚠ NO justification found for roles/iap.egressor — over-broad, removing
            ✓ custom role scoped to least privilege

[VERIFY]    deterministic gate: dry-run binding against IAM Policy Simulator… PASS
[COST]      IAM bindings incur no charge — N/A
[GATE]      2 bindings ready to apply. Approve? (y/n)

Same task. The first pass is correct, the silent over-broad grant is caught and removed before it's ever applied, and you're shown exactly what's about to change before you say yes.

What actually made the difference

Five things, and none of them is "a smarter model." They're an architecture around the model:

1. Ground before generate. Read real state — current account, project, whether the group exists — and inject it into context before proposing anything. This alone kills most of the iteration loop, because the tool can no longer emit a placeholder or a permission the caller can't grant.
2. A deterministic gate, not a polite instruction. This is the load-bearing distinction. You can tell a model "always validate before applying," and it will comply most of the time — but most of the time is not all of the time, and the gap is where the bad binding ships. Or you put the check in a pre-execution hook that runs every time and blocks on failure. For safety and policy, deterministic beats probabilistic, every time. The gate is what ends apply-then-see-error.
3. An architect-critic in its own context. A separate reviewer scores the proposal against least privilege and idiomatic pattern, and flags the unjustified grant the generator was happy to ship. It's advisory — critics can be wrong too — which is why it sits in front of the deterministic gate, not in place of it.
4. A cost view, automatically. Every proposal that touches billable resources gets a line-item breakdown before approval, so "best available option" arrives with a price tag attached instead of a surprise on the invoice.
5. Selective triggering. None of this fires for "what's the syntax for X." The heavy path engages only for design-class work — IAM, provisioning, migrations, network policy, architecture choices — and stays out of the way for everything else.

You don't need a platform team to build this

The whole thing runs on a local install — no IDE, no internal developer platform. The grounding step is a read-only shell call. The critic is a sub-agent with its own context and rubric. The gate is a pre-execution hook that exits non-zero to block. The router is a few lines of standing instruction plus a skill keyed on the task signature. The lightweight, instruction-only version captures much of the value in an afternoon; the deterministic gates are where you spend the next day or two, and they're the part worth spending it on.

How you'd know it's working

Resist the urge to measure only the average. The obvious win — fewer rounds to a working command — is real, but it's the small one. The win that matters is the variance collapsing: a pipeline whose worst case is bounded is worth more than one whose average is good and whose tail occasionally ships roles/owner to a developer group. So when you evaluate this, report the 95th-percentile outcome next to the mean, and measure the silent-defect rate — the fraction of outputs that run clean and are still wrong. That number is invisible to the tool itself, and bringing it down is the entire point.

The error catches the typo. Only review catches the design.

"It ran" was never the same thing as "it's right." The fix isn't a better model — it's giving the model a world to read before it speaks, and a gate it cannot talk its way past. That is the difference between a deployment that demos well and one your risk, security, and compliance teams will actually let run.

We run this pattern ourselves. The assistant on this page answers only from an approved knowledge base, behind a gate, and routes anything consequential to a person — the same grounded-and-gated shape, pointed at a different problem.

This is the control layer we build around Claude for regulated teams — grounded, gated, and measured, so the same discipline that protects a deployment also produces the evidence your auditors accept. If "it passes every test and fails the audit" sounds familiar, that's the gap we close.