Make the Agents Argue: Adversarial Review for Code That Cannot Fail Quietly

How a solo developer ships an AI-built options-market data and research system: two AI architects working in isolation, priced adversarial findings, human gates, and the bugs the process actually caught.

Published Aug 28, 2026 updated

By Sayan Biswas

21 min read

I trade options on the NIFTY 50, and the trading account lives with one broker - call it BrokerX. The data pipeline behind that trading, though, runs on a second broker’s API - call it BrokerY. BrokerX’s API serves live option-chain data well, but its historical options data is thin; BrokerY offers both live market data and a much richer history, which is what research needs. The split bought a second, less obvious benefit: isolation. The BrokerY account holds effectively zero rupees - it exists purely to serve its API - so even a fully compromised credential cannot place a meaningful order on my behalf. That caution is not paranoia: the credential does not sit on my laptop. It sits on a cloud VM that refreshes it on a schedule, unattended.

Unattended renewal is where this story starts. BrokerY’s token-renewal endpoint is atomic: renewing mints a new access token and invalidates the old one in the same instant. Concretely, here is the failure that shapes everything:

At 08:45, half an hour before market open, the VM calls renew, holding token A.
BrokerY mints token B and kills A. The response dies on the way back - say, a gateway timeout between the broker and the VM.
Server truth: B is live, A is dead. Client truth: A is the only token that exists. The fresh token is stranded.
The client’s next renewal attempt, made with A, gets rejected. That rejection is poisonous, because a refused renewal of a token we believe to be live is the only theft signal the system ever gets - it must be treated as a possible credential compromise, which halts automated recovery. A retry after an unknown outcome therefore risks trading days of manual recovery with a false security alarm on top. The entire authentication design of the trading system I am building rests on one rule: probe, never retry.

One morning we added scheduler-level retry with exponential backoff for renewal failures. It was carefully designed and human-approved, with a status protocol that distinguished retryable from non-retryable outcomes. The same afternoon, a code reviewer found that a gateway 429 or 503 propagated through the renewal path as an uncaught exception, which the runtime converted into a 500 - and the new retry configuration would answer a 500 with a blind second renewal call. Two individually reasonable changes, each fine alone, composed into exactly the forbidden behavior. The deploy was scheduled for that evening. The review gated it, and the fix shipped together with the retry, safely.

The reviewer that caught this had no memory of writing any of the code involved, because it had none: it was a fresh instance of an AI agent, handed the requirements, the design, the diff, and the tests, and nothing else. The implementer that wrote the bug was also an AI agent. So was the architect that designed the retry protocol, and so was the second architect whose competing design it had been weighed against. I am one person; the “team” is a process.

This post describes that process: an adversarial, multi-agent review structure for building software where silent failure is unaffordable. Throughout, “I” is me, the human owner, and “we” is me plus the agents, which is the honest voicing for a system where the agents write almost all the code and I sit at the gates. The post covers why the structure exists, the rules that make it work, the bugs it actually caught (with numbers), and what it costs.

One Person, Real Money, Nobody Watching

The system is FnO-Trade, a data and research platform I am building for futures-and-options trading on the NIFTY 50 index (F&O is the Indian market’s shorthand for the derivatives segment; FnO-Trade is simply the project’s name). It captures live market data on a schedule, manages a broker credential, and will eventually recommend real trades. Three properties make it unlike most side projects:

Mistakes are unrecoverable. A missed market session cannot be re-captured at any price. A destroyed access token costs trading days.
Mistakes are expensive. Bad data silently poisons every model trained on it. A bad trade recommendation loses actual money.
Nobody is watching. The system runs unattended. The failure modes that matter are the quiet ones.

AI agents write nearly all of the code, and that creates a risk profile worth stating honestly:

An agent reviewing its own work is blind to its own assumptions, exactly like a human implementer, only faster and more confident.
Agents are agreeable by default. Ask “is this design good?” and the answer is “yes, with minor suggestions” far too often.
A green test suite is weak evidence. Tests written by the implementer exercise the implementer’s mental model; the bugs live outside that model.
Models trained on similar data share blind spots. Diversity of models is not a luxury; it is where less-correlated evidence comes from. Not fully independent - two large transformers still share training influences, the same requirements, and similar reasoning habits - but far less correlated than one model reviewing itself.

The answer we converged on: make disagreement structural. Do not ask agents to be critical. Build a process in which criticism is a role with a schema, a price, a cap, and a paper trail - and a human sits at every gate where meaning is decided.

The Process at a Glance

Before the rules, the shape. An architectural feature walks a fixed pipeline from idea to merge:

Classify the task. Only genuinely architectural work takes the full lane below; small changes travel light.
Freeze the requirements with me, as numbered acceptance criteria (states 1 and 2 in the diagram).
Design twice, independently. Two different models produce competing plans without seeing each other’s work (state 3).
I choose one plan as the base. That decision is mine alone, and nothing moves forward until I make it (state 4).
Adversarial review rounds attack the chosen design with structured, priced findings until deterministic gates pass (states 5 and 6).
Implement test-first, then hand the result to two independent, memory-less code reviews (states 7 and 8).
I merge, and the feature records what the process caught, missed, and should learn (state 9).

Every stage is enforced by tooling rather than by memory or goodwill, and each rule inside it earned its place through a specific failure. The next two sections walk through those rules and the machinery, and then we get to what all of this actually caught.

Criticism With a Price, a Cap, and a Paper Trail

Three rules carry most of the weight, and each came from a failure, not from theory.

Criticism must carry a price. When flagging a concern costs nothing, reviewers flag everything, and vague “this might be a problem” comments bury the real ones. So:

A BLOCKER or MAJOR finding requires a concrete failure scenario: specific inputs and state leading to a specific wrong outcome.
No scenario, and the gate tooling itself automatically downgrades the finding to a minor, non-blocking note. This single rule filtered nearly all the noise out of roughly eighty findings on the first big feature.
The downgrade lowers the finding’s evidentiary standing, not the stakes of the defect it might describe, and it is not a rejection: the finding stays on the record, and refiling it with a concrete scenario attached restores full severity. The bar is on evidence, never on the reviewer’s conviction.

Criticism must also be capped. Reviews that can run forever do run forever. So:

At most three open BLOCKERs per reviewer per round, and at most three rounds.
After that, the disagreement escalates to me as a “crux brief”: both positions in five lines each, the single crux, and what evidence would settle it.
The cap bounds what is open at once, not what may be reported: findings beyond it queue for the next round, and anything still unresolved when the round budget runs out rides the escalation path to me rather than disappearing.

The fresh-context reviewer must never see the implementation conversation. A fresh-context reviewer is a brand-new agent instance with an empty memory: it took no part in writing the code and has watched none of the discussion around it; in the pipeline, this is the reviewer at code-review (8). Its job is to judge, from the artifacts alone, whether the change actually does what the requirements demand, and to hunt for the concrete cases where it does not. That blankness is the asset, and it has to be protected:

A reviewer receives exactly four things: the requirements, the design, the diff (the code changes themselves, line by line, against the previous state), and the tests. Not the chat log, not the implementer’s reasoning.
The implementer’s narrative is a contamination vector: it explains away exactly the assumptions the review exists to check.

Around those rules sits a division of labor: humans own meaning, agents own mechanism.

Five gates are mine alone and cannot be automated: approving requirements (the gate into 2), picking the base architecture (at 4), settling escalated disagreements (at 5), approving contract amendments (the loop back into 1), and merging (the gate into 9).
Everything between the gates is agent work under deterministic checks.

And none of it lives in anyone’s memory - the process is enforced by machinery:

Every feature walks a state machine, and a CLI tool validates every transition.
Editor hooks physically refuse writes to source directories unless the feature is in a state that permits them.
The requirements file is immutable outside the requirements state (1).
A session with open blockers refuses to end.
When the process blocks us, that is the process working. The rule is written down: do not “fix” a hook block by bypassing it.

The Machine

Triage: not everything deserves the full lane

Every task is classified first. A spike answers a feasibility question cheaply and keeps no code. A bounded task is a small, well-scoped change: short design in chat, my nod, implementation, one fresh-context review of the diff, done. Architectural work - a new subsystem or a restructuring - gets the full lane described below. The rule for doubt: take the heavier path. Hidden complexity discovered mid-task upgrades the classification; nothing ever downgrades mid-task.

Requirements: the immutable spine

Requirements are drafted with me (state 1) and then frozen at approval (state 2). Every acceptance criterion gets an ID (AC-1, AC-2, …) in Given/When/Then form, and those IDs become the spine of everything downstream: designs must map every AC, tasks must cite their ACs, tests are named for the ACs they prove, and an unmapped AC is an automatic reject. A deterministic gate checks the mapping mechanically.

Changing requirements after approval means walking the state machine back to the requirements gate (state 1) and editing under an explicit “amended at a reopened gate” annotation, with fresh approval. That sounds bureaucratic until the day it matters: twice we discovered the deployed system had drifted from the written contract, and the reopened-gate mechanism is how the contract got repaired honestly instead of rotting.

Two architects who never meet

For architectural work, two design plans are produced independently during planning (state 3). Plan A comes from a Claude architect agent, from the requirements alone. Plan B comes from Codex, a separately trained model, invoked headlessly from the requirements alone. The independence is enforced mechanically: the second model’s prompt bundle must never contain the first model’s plan, and the process treats leaking it as a defect. Two architects who can see each other’s plans converge. Two, who cannot see them, produce genuinely different architectures, and the differences are information.

Synthesis (state 4) is then a choice, never a union. Merging two plans produces a camel. An agent drafts the synthesis brief in choice format: a proposed base, at most two grafts from the other plan (each individually justified), and an explicit rejected list with reasons. I make the call at a hard gate: pick the base and accept or refuse each graft. On our historical-backfill feature, the Claude plan had the better overall shape, but the final design kept two grafts from the Codex plan. Neither model produced the final design alone, and that has been the norm rather than the exception.

Review roles, not review moods

Once a design exists, review rounds (state 5) run with reviewers launched in parallel, each under a canonical role definition shared by both models. A role is an objective, not a temperament: each reviewer is told what to hunt for and what counts as a finding, so an instruction like “be critical” never has to carry the weight.

The adversary attacks the design against the requirements. Its job is to find concrete failure scenarios, not to give feedback. The role exists because a model asked for feedback defaults to polite suggestions; a model asked to break the design goes looking for the input that breaks it.
The minimalist finds everything that can be deleted while preserving all acceptance criteria. Complexity is treated as a defect with a burden of proof. This is the counterweight to the models’ natural drift: left alone, they add layers, options, and defensive code, so one reviewer’s entire job is to push the other way.
The security reviewer hunts for leaked credentials, unsafe handling of tokens and order requests, and trust-boundary violations. It joins only when the design touches secrets, credentials, order placement, or external APIs. On a feature with none of those, it has nothing real to find, and whatever it files anyway just wastes one of the few blockers a reviewer is allowed.

Every finding lands in a YAML file with a fixed schema. The pricing rule and the caps apply to these entries mechanically:

  
id:                # stable identifier, so later rounds can refer to it
role:              # which reviewer filed it: adversary, minimalist, or security
severity:          # blocker (stops the merge) or a minor, non-blocking note
claim:             # one-sentence statement of the defect
evidence:          # what the claim points at: the file, requirement, or behavior
failure_scenario:  # concrete inputs and state under which it goes wrong
status:            # open, settled, rejected, or escalated
round:             # the review round it was filed in

Three rules about these findings have special force:

Rejected is final. Rejection is my decision, and reviewers must not refile a rejected finding in later rounds. This stops the process from relitigating settled questions forever.
- The rule is deliberately blunt, and its failure mode is obvious: a wrong rejection becomes permanent unless something material changes. We accept that trade, because reviews that reopened settled questions every round were the disease this rule cured.
- A finding backed by genuinely new evidence or new code is a new finding, not a refile.
Escalated means the ball is with me. The design is still not ready, but the session may end, because the outstanding work belongs to a person.
An empty findings list is a legitimate pass. The report must not be padded to look thorough.

Three deterministic gates then read the same findings file the reviewers write. Each is a plain command-line program with an exit code:

Zero open blockers. A blocker with no failure scenario attached is first downgraded, loudly.
Full coverage. Every AC maps to a task and to a named test that actually exists.
Round budget. The round counter is within its cap.

Because the gates and the reviewers share one file, there is no separate bookkeeping to drift out of sync.

Design may defer a mechanism, never a meaning

The most expensive lesson in the whole system. A design does not have to decide how something will be built: which library, which retry strategy, which data structure. Those choices can safely wait for implementation. What it must never leave open is what a value means, or which component owns a piece of state. Does a “failed” status mean the request never left the machine, or that the broker rejected it? Is the server or the client the authority on which token is live?

Leave a question like that open and it does not stay open. The implementer answers it in passing, a test pins that accidental answer down, and then every reviewer who reads the code trips over the same ambiguity and files it again, round after round. The worst review churn this system ever produced traced back to exactly this - a design that carried its unanswered meaning-questions into implementation - and it is what motivated the hard cap on review rounds.

Fresh-context review, twice, in parallel

When implementation (state 7) is complete, two reviewers examine the result independently at code-review (state 8): a fresh-context Claude instance and Codex, each given the same four inputs (requirements, design, diff, tests), each running the suite itself, each verifying findings against the production code path rather than the test path. Each writes to its own findings file - we learned that one the hard way, after two concurrent reviewers clobbered a shared file twice. The orchestrating agent consolidates afterward, merging cross-model duplicates and keeping the higher severity.

Disagreement between the models is surfaced, never smoothed. The rule is written into the process: never edit the other model’s output to agree with a different view. Cross-model disagreement is signal, and it is mine to settle.

Review the fixes

At merge (the gate into state 9), the feature directory gets an outcome.md: which findings were real, which were noise, which would have been expensive to miss. It is a young feedback loop, but it has already forced us to write down an uncomfortable fact - the two worst bugs of the whole project were both found in fix diffs, code written to repair earlier findings. Fix diffs get full review. That is not ceremony; the next section is the proof.

What It Actually Caught

Beyond the retry bug that opened this post, the catches fell into five categories:

Test blind spots: code that behaved differently under test than in production, so a green suite proved the wrong thing.
State-machine traps: rare but legal event orderings that stranded the system in manual-recovery states; the worst hid inside an already-reviewed fix.
Misfiring alerts: monitors that could never fire, or would page constantly on a healthy day.
Contract drift: the running system no longer matching what the written requirements say it does.
A false premise I had approved: human gates are necessary, not sufficient.

The receipts live in the feature’s outcome.md, the merge-time record of which findings were real. An excerpt:

| Round                     | Filed | Real (code fix) | Real (amendment) | Rejected | Noise |
|---------------------------|-------|-----------------|------------------|----------|-------|
| Design r1-r3              |  51   |       47        |        -         |    4     |   0   |
| Code review r1            |  22   |       18        |        3         |    1     |   0   |
| Code review r2 (fix diff) |   8   |        7        |        0         |    0     |   1   |

Cross-model overlap ~40%; neither model's list subsumes the other's.
Lesson carried forward: fix diffs get fresh-context review - both rounds'
worst findings were in code written to fix earlier findings.

So: 81 findings filed, one of them noise (a back-compat concern that turned out to apply to an object that never existed - and even that check was cheap enough to be worth running). The rejected findings were my judgment calls, not reviewer errors. And the first unattended production week was fully clean.

What It Costs

The structure is not free, and pretending otherwise would undercut everything above.

Wall-clock latency. A full review round is one to two hours of agent time; the full lane for a feature is days, not hours.
Token cost. Two models, multiple roles, multiple rounds: the review spend for a large feature is comparable to the implementation spend. Measured over the two-week window covering this feature’s full lane, the Claude side alone generated about 17 million output tokens and close to five billion input tokens. Almost all of that input was cached reads: the same requirements, design, and findings files fed back in, round after round. On a flat subscription that is capacity, not a bill; at metered list prices the same traffic would run to a few thousand dollars.
Process friction. State machines and hooks refuse work at inconvenient moments. That is their job. It is still friction.
It is heavy for small changes. Triage exists precisely so the full lane is reserved for architectural work; bounded changes get a short design and one fresh-context diff review, nothing more.

For a system where a silent defect costs unrecoverable data or real money, the trade is obviously right. For a CRUD app, most of this would be overkill - but three pieces are cheap enough to steal individually: the pricing rule (no failure scenario, no blocker), the fresh-context reviewer (four inputs, never the chat log), and synthesis-as-choice (one base, at most two justified grafts).

The process also has known weak points:

Consolidation is manual. After the two code reviews, deciding that two findings describe the same defect seen from different angles is still done by hand. That worked at 22 findings; it will not scale to hundreds.
Reviewer accuracy is not measured over time. Each feature’s outcome.md records which findings were real, but nothing aggregates that into per-role, per-model precision - the numbers that should eventually decide how much weight a finding carries and whether a role is earning its cost.
The model diversity is only two. Claude and Codex disagree usefully, but both are large transformers with overlapping training data. A third, cheaper model doing narrow mechanical sweeps (dependency changes, permission diffs, literal values) would add coverage exactly where the big models are weakest: exhaustive, boring checks.

The Takeaway

The one-sentence version: treat criticism as infrastructure.

Give it roles, so it is nobody’s mood.
Give it a schema, so it is comparable.
Give it a price, so it stays scarce and serious.
Give it caps and escalation, so it terminates.
Give it independence, so two reviewers agreeing actually means something.
Keep humans at every gate where meaning is decided.
And review the fixes, because that is where the worst bugs were hiding.

None of the individual pieces is novel. Red teams, design reviews, and staged gates are old ideas. What changed with AI agents is the price: running two isolated architects and a bench of adversarial reviewers on every feature used to cost a team; now it costs an API bill. The level of review once reserved for safety-critical systems is available to a solo developer building a trading system at their kitchen table.

The process caught the bugs that everything else had already waved through - the tests, the reviews, and me - and it did so without burying me in false alarms.

That is the case for making the agents argue.

ML Platform