Designing human-in-the-loop gates

Industrial quality control has gone through four stages, and at each one the inspector moved further from the bench and higher in abstraction.

In the craftsman era, the maker and the inspector were the same person. A blacksmith struck the iron, looked at the iron, and decided whether to strike again. The gate was inside the act of making, and "is this good enough" was answered by the person whose hands were already on the work.

Then the factory arrived, and the gate moved to the end of the line. Volume rose, the inspector job got pulled out as a separate role, and one person checked every unit that came off the conveyor. This worked until volume outran the inspector. Then statistical process control took over: sample instead of checking every unit, and move the human's attention from the output to the method. The next move is the one most people forget. Quality kept rising, and the human went up one more rung, from inspecting samples to inspecting the process itself, deciding which controls the process needed and where in the line they should sit.

The gate did not disappear at any stage; it changed altitude. The maker, the inspector at the end of the line, the sampling statistician, the process designer: each was a real human at a real gate, just placed further from the work and higher in abstraction.

Agent-assisted engineering is the next move in the same arc, but the factory-inspector posture has not gone away: one human reading every PR, at a volume the posture cannot survive. The output is predictable: gates exist on paper, and reviewers skim and approve.

A gate is real when the human at it has three things:

• enough time to decide
• enough context to know what they are deciding
• enough power to say no

Remove any one and the gate stops being a safeguard: it becomes decorative.

Give the gate enough time

Under volume, the reviewer runs out of time before anything else. An engineer who reviewed two PRs a week is now expected to review fifteen, several spanning systems they did not write, each touched by an agent whose output reads cleanly even when it is wrong. The reviewer's minute is finite, and you cannot solve a budget problem by asking the budget to stretch.

The first instinct is to ask people to be more careful, but it does not scale, and it pushes a structural problem onto the reviewer's character. The opposite instinct, to remove the human and let the agent gate its own work, is also wrong: today's models are not good enough, and the human is the last line of defense. The right move is to put an agent layer in front of the human, where the agent does the bandwidth work and the human stays as the final gatekeeper.

In practice that means two layers:

• An agent layer doing the first pass. On every action, it catches obvious issues, suggests fixes, surfaces uncertainty, and annotates the change so the human reviewer can decide quickly.
• A human review as the final gatekeeper. The human gates what ships, reading pre-screened and annotated material rather than raw agent output.

The diff the human reads should be small enough to actually read, and the count should be small enough that thirty minutes of attention is plausible. If your gate's volume requires the reviewer to spend less than a minute on each item, you do not have a gate; you have a counter.

Hand the reviewer named context

The reviewer who knows the system, the spec, and what the change is supposed to do can decide in five minutes. The reviewer staring at unfamiliar code with no anchoring story is doing pattern-matching against confident-sounding code, which is exactly the failure mode automation bias describes.

The clearest model here is the surgical checklist Atul Gawande wrote about, the WHO Safe Surgery Saves Lives protocol that cut surgical mortality by roughly a third with a ninety-second ritual before the cut. The mechanism was not added diligence, since surgeons were already diligent. It was named context at the gate: the team stated, out loud, the patient, the procedure, the allergies, the instrument count. The checklist forced engagement, and the engagement is what caught the misses.

The agent-era gate needs the same shape. Rather than a diff and a thumbs-up button, the reviewer should be presented with five things:

• what the change is supposed to do
• what the agent thought it was doing
• where the agent was uncertain
• what test coverage exists
• the single most likely place a translation gap occurred between the spec and the output

The gate's first job is no longer "is this code well-written"; it is "does this output match the intent that was asked of it."

Make blocking cheaper than approving

The third condition is power, and it is the one most likely to be present on paper and absent in practice. A reviewer who can technically block but who pays for the block in peer friction or a manager's raised eyebrow learns within three cycles that the gate's signal is approve.

The levers here are structural.

• Separation of duties. The person who wrote the change cannot be the one who deploys it, and the reviewer cannot be the author's direct report.
• Blameless escalation. A written record establishes that surviving the gate matters more than passing through it.
• Funded review time. The reviewer's time has to be in the team's plan rather than stolen from their evenings.
• A remembered precedent. At least one example, recent enough that everyone remembers it, of a senior person blocking something high-profile and being publicly defended for it.

A block should be a normal Tuesday; if it is a career event, your gate is decorative.

Place the gate where fixing is cheap

Gate content and gate placement are different design choices, and the second one is easy to get wrong without noticing; I learned this on our own agent code review.

The first place we put the agent reviewer was on PR open. The intent was honest: catch issues before a human's time was spent on them. What we missed was the timing: by the time the agent ran, the author was done. They had written the change, pushed the branch, opened the PR, and mentally moved on to the next thing. The agent's findings arrived as a tax on a finished artifact: the author had to context-switch back, rerun the IDE, fix, push again. Each cycle cost a small amount of motivation, and after a few weeks the team treated the agent reviewer as a step to clear.

We moved the gate one step left, into a pre-push git hook inside the IDE. Same agent, same diff; only the placement changed. Now the agent reviewed before the code ever left the local machine, and the author was still in IDE context when the feedback arrived. The cost of acting on a finding dropped close to zero. The cleaner PR for the downstream human reviewer was real, but the primary win was upstream: the author had not mentally moved on.

This is the craftsman move with modern tools. We started at the factory-inspector gate: PR open, every unit checked. Moving the gate left was a move back toward the maker inspecting their own work, with an agent helping watch what the maker can no longer catch alone. The lever is cost-of-fix for the producer, and everything downstream is a bonus.

Never stand a gate alone

A single gate is a single point of failure, and safety engineering has known this for fifty years. The Swiss cheese model makes the point graphically: each defensive layer has holes, and an accident happens only when the holes align across layers. No one barrier catches everything, and any design that pretends otherwise is brittle.

The four-eyes principle from banking and editorial publishing gives the structural rule: two pairs of eyes on consequential actions, and the second pair is not the originator. The action does not commit until both have engaged.

The residual risk is common-mode failure: when both layers share an assumption, they share a blind spot. If the upstream judge is the same family of model as the agent, and both miss the same kind of subtle wrong, the second layer offers less protection than the diagram suggests. Mix the layers when the action's blast radius warrants it: a different model, a different kind of check (a static analysis or a property test alongside the model judge).

Defend both paths to theatre

A gate becomes theatre by two routes. The first is the lazy path: the reviewer disengages over cycles, the work feels low-stakes, and approve becomes the default. The second is the over-reliance path: the reviewer's own confidence has eroded against a confident model, and they defer to the machine even when they would have gotten it right alone. Both routes produce the same observable behavior, and both are common after a few months of running gates against a confident model.

The design move against both is the same: make engagement a precondition of approval, so the reviewer cannot pass an item without naming something specific (what could break, and how they would know). A two-sentence justification beats a thumbs-up. If your gate's approval costs less than five seconds, the gate's signal is already approve regardless of what the reviewer thought.

What's next on the ladder

The work in agent-era engineering is not to add more inspectors; it is to design the gates well, and then to design the system that designs the gates. The next move up the abstraction ladder is process design:

• which actions deserve a gate at all
• where on the producer's flow each gate sits
• what the layer stack looks like
• what the audit looks like that tells you the gates are still earning their cost

Two hundred agent actions a day at three minutes of review each is ten hours of work, and per-action human approval cannot scale to that volume. The honest response is not to remove the human; it is to move them up a rung, from approving every action to watching the system and intervening on anomalies, with override preserved for the few things that matter.

A gate that looks like oversight but does not provide it costs more than the time of the person standing at it. It also displaces the question the team should be answering, which is whether this action deserved a gate at all and where it should have sat on the producer's flow. Decorative gates are not neutral: they cost reviewer hours and train the team to read approve as the default, and they remove the pressure to design the real one.