itstauf
The Compounding BrainChapter 7 of 12Reading time: 6 min

The critic

Illustration: The robot hands a sheet of paper across a desk to a second, taller and sharper-looking angular robot with narrow eyes who holds a lime red-pencil and studies the page critically. Behind them on a hat stand hangs a small hat.
Act II, scene 7. The second voice

The fear in the room

The hire's beliefs were now earned. But a belief only matters if the work follows it.

When I got ready to show this system to a room of technical and non-technical people, I knew which question I had to answer. Not "how does it remember?" The real one sits underneath every AI conversation I have with founders.

What stops it from confidently shipping rubbish?

It is a fair fear. I have it too. A language model writes a weak proposal in exactly the same calm, fluent voice as a strong one. It will invent a number with a straight face. It will produce a paragraph that sounds smart and could have come from any consultancy on earth. And the better the writing gets, the harder the rubbish is to spot.

The obvious answer is that I read everything myself. I run three ventures, largely solo. Reading everything myself means I become the bottleneck, and a tired reviewer is a worse one than I like to admit.

So I gave the brain a second voice. Its whole instruction fits on a sticky note:

"Your job is not to be nice. Find reasons this will fail."

Nothing the brain writes for the outside world leaves without passing through it. Not proposals, not posts, not decision memos. The critic does not help the draft. It attacks it.

The idea

A draft is not finished until something whose only job is to make it fail has tried and lost.

Asking the same model to write and then check its own work, politely, is a student marking their own exam. A critic works because it is told, in plain words, that its loyalty is to the reader, not to the writer.

And when it misses something, the fix starts with the critic, not with the draft.

How adversarial review works

Every outward-facing draft gets handed to the critic before I see it. The critic runs twelve checks. Each one is a plain question with a plain consequence. A few of them, in everyday words:

  • Is there a claim with no source? Fail. Numbers, quotes and results must point to where they came from.
  • Does this sound like any other consultancy? Fail. If you could swap my name for a competitor's and nothing breaks, it is generic.
  • Does it contradict something the brain already learned? Fail. If a rule climbed the ladder to the top, a draft does not get to quietly ignore it.
  • Does it recommend a tool before fixing the people problem? Fail.
  • Does it name a client who has not cleared it? Fail, every time, no exceptions.

The tool check is my favourite piece of the system. One of the rules the brain earned on its own, back in Chapter 5, is "buying a better tool never fixes a team that keeps failing the same way." The critic now enforces it against my own drafts. The brain learned something, and then it started holding me to it.

Each finding comes back as a line: the quoted text, what is wrong, and a suggested fix. Then the critic gives one of three verdicts.

A finding is built to route to a fix. Made-up example. One critic finding broken into its four parts, top to bottom. The quote, 'Clients see results in week one', points at the exact words in the draft. The check, unsourced claim, names the rule it broke. The fix, cite the source or cut the line, is something the draft can act on. The verdict, rewrite, sends the draft back rather than to me.
Every finding quotes the words, names the check, offers a fix and feeds a verdict, so it routes to a fix instead of a complaint. Made-up example.
Tap to enlarge
A finding is built to route to a fix. Made-up example. One critic finding broken into its four parts, top to bottom. The quote, 'Clients see results in week one', points at the exact words in the draft. The check, unsourced claim, names the rule it broke. The fix, cite the source or cut the line, is something the draft can act on. The verdict, rewrite, sends the draft back rather than to me.

Every finding quotes the words, names the check, offers a fix and feeds a verdict, so it routes to a fix instead of a complaint. Made-up example.

Ship. It found nothing that should stop the draft. Rewrite. It found problems the draft can fix on its own. Escalate to me. It found something that needs my judgement.

   DRAFT
     |
     v
 +------------------------------------+
 |  THE CRITIC                        |
 |  "Your job is not to be nice."     |
 |  12 checks: unsourced? generic?    |
 |  contradicts a rule? tool-first?   |
 |  uncleared client? ...             |
 +------------------------------------+
     |            |              |
     v            v              v
   SHIP        REWRITE     ESCALATE TO ME
                  |              |
                  +--> draft --->+
                                 |
               I overrule the critic?
                                 |
                                 v
                 FIX THE CRITIC FIRST, then the draft

The last box is the rule that makes it compound. The critic carries this line about itself: "If a founder rejects something the review approved, update this skill's prompt before anything else." When I overrule the critic, I fix the critic first. Every miss becomes a sharper check, so a miss I have caught once has a check waiting for it next time.

What I learned the hard way

A finding with no route to a fix is just noise. Elsewhere in my system, one report flagged the same missing tag on 9 consecutive runs. Nine times it was right, and nine times nothing changed, because nothing connected the report to a fix. A critic that only complains teaches you to stop reading it. Repeated findings now become a decision addressed to me.

Check the checker. A critic can be wrong with exactly the same confidence as a writer. When it flags something, I look at the check before I rewrite the draft.

Better is not the same as longer. I still have no clean way to tell whether an output got better, as opposed to longer. The critic catches what is wrong. It does not yet prove what improved.

Try it

the-criticAn adversarial reviewer that checks every draft before it leaves, and the rule that when it misses, you fix the critic first.Open the repo

The honest limit

In my own system the first version of the critic was the same model wearing a different hat, given a different prompt in the same session. It was much better than no critic, but it shared the writer's blind spots. The answer is a separate sub-agent that sees only the draft and the checks, never the reasoning that produced it, and that is the version the repo ships. Adversarial review only works if the reviewer can't see the excuses.

The reviewer never hears the excuses. Side by side. Version 1, the same model in a different hat: its context holds the draft, the checks, and its own reasoning for why it wrote the draft that way, so it can be talked round. Version 2, a separate sub-agent: its context holds only the draft and the checks; the writer's reasoning is absent, so there is nothing to talk it round with.
The separate critic only sees the draft and the checks, so there's nothing to talk it round with.
Tap to enlarge
The reviewer never hears the excuses. Side by side. Version 1, the same model in a different hat: its context holds the draft, the checks, and its own reasoning for why it wrote the draft that way, so it can be talked round. Version 2, a separate sub-agent: its context holds only the draft and the checks; the writer's reasoning is absent, so there is nothing to talk it round with.

The separate critic only sees the draft and the checks, so there's nothing to talk it round with.

Read this next

The critic checks the work. But something has to do the work, and one job I needed every morning had no routine at all.

Next... Skills that write skills, the routines that do the work the critic reviews. For why the critic trusts some rules and not others, see the ladder.

Previous chapterWhat's worth trackingNext chapterSkills that write skills
Brain Builders

Building one of these? You don't have to do it alone.

Join Brain Builders