The model that writes your content cannot grade it

What it cost me to learn that, measured across 233 items.

Four explanations my app almost taught to German learners:

Here is the part that should worry anyone shipping AI-generated content. In every one of those items, the answer was correct. The exercise worked. A learner would have chosen the right option, felt good about it, then read one sentence of confident, fluent, grammatically-worded nonsense and believed it.

No spot check catches that. You read the item, the answer is right, you move on.

None of these ever reached a learner. Catching them is what this article is about.

What I was building

I run KorrektWort, a German vocabulary app I build alone. I am also sitting a B1 German exam myself, which is how I ended up here: German exams have a gap-fill grammar section, one sentence, one blank, three options, exactly one correct. I wanted to generate those from the sentences already in my content cache, and I wanted each one to explain itself in a single line, because an exercise that only marks you wrong is a test, not a teacher.

The safety property is unforgiving. Exactly one option may be acceptable. If two work, a learner who knows German correctly gets told they are wrong, which is the worst thing a language app can do. And the explanation has to be true, because a false rule taught with confidence is worse than no explanation at all.

The first pipeline, and what it actually scored

Version one was the obvious thing: one call to a fast, cheap model (Claude Haiku), structured output, give me the gap, the answer, two distractors and the explanation.

It looked good. Reading through the output, I would have shipped it.

So instead I ran a blind batch: 106 items from sentences the generator had never seen, graded by independent examiner agents against a fixed six-grade rubric, with disagreements adjudicated by a stronger model. No re-grading of anything the prompt had been tuned against.

15.1% of items contained something false. My own read-through had caught almost none of it, because the answers were right.

The diagnosis

Two failures, and only one of them was about model size.

The explanation was a rationalisation. The same call invented the gap and then wrote the reason. By the time it explained anything, the decision was already made, so the explanation's job had quietly become justifying a choice rather than describing a rule. That is exactly the condition under which a language model produces fluent, false text.

The model was weakest at precisely the thing it was explaining. Grammar terminology is where a small model's confidence and its competence diverge most. It knows the words dative, genitive, transitive, and it will use them in a well-formed sentence that happens to be wrong.

So I split the work up.

The rebuilt pipeline

Every stage below is a separate call, with one rule behind the whole design: no stage may grade its own output.

1. Deterministic hygiene, in code. Sentences written with ae/oe/ue instead of umlauts, and other cheap-to-detect junk. No model needed.
2. Naturalness gate. Is this source sentence correct, standard German at all? Errors only, not taste.
3. Propose the gap, on a strong model now (Claude Opus). Blank a token that tests a grammar rule, with two distractors that are tempting but wrong.
4. Structural validator, in code. The gapped sentence plus the answer must reconstruct the original exactly; two distinct distractors; no content nouns blanked; no gap inside a word; no genitive article blanked in front of a noun that already carries its own genitive ending, because then the wrong options are impossible rather than tempting. Every rule I could express as code, I moved out of a model. These checks are free, instant, and never hallucinate.
5. Write the explanation separately, from a checklist, in its own call. It never sees the reasoning that produced the item, so it cannot rationalise it. The checklist forbids the exact failures above: name a governing word only if it truly governs, state a case or gender only when certain, never invent a word that is not in the sentence.
6. Validate the explanation, in code. Length, jargon, and a citation check: every German word quoted in the explanation must appear in the sentence or be a legitimate dictionary form of one. That single check catches springt quoted when the sentence says sprang, and uberweisen written without its umlaut. Its false-positive rate, measured over 206 previously generated explanations, is 1 in 206.
7. Adversarial ambiguity judge. A separate call whose entire job is to put each distractor into the gap and argue that it also works. Its incentive is the opposite of the generator's.
8. Two acceptance panels. Three lenses each (ambiguity, truth of the explanation, naturalness and distractor pull), unanimous PASS required from all three, then the same again on a stronger model.

Multi-agent is about independence, not parallelism

This is the part I would most want another engineer to take away, because I had it wrong at the start.

I originally reached for multiple agents because they could run at once. That is the least interesting reason. The reason that matters is that agents can be made structurally different from each other, and a judge that shares the generator's framing shares the generator's blind spots.

The evidence is uncomfortable and worth reporting. My first acceptance panel, three lenses on a strong model, unanimous agreement required, passed a set of items with a clean sweep. I then added a second panel on a stronger model, same rubric, and it rejected thirteen items the first panel had unanimously approved. Among them: two source sentences containing a spelling error nobody had noticed, and three items where a distractor genuinely produced correct German.

Unanimity within one framing is not independence. It is one opinion, repeated three times.

So when I built the grader that would measure the pipeline's real quality, I deliberately made it differ in three ways at once:

• Different prompt structure. It never sees the rubric the pipeline's own gates use.
• Opposite incentive. It is framed as the appeals board of an exam. Candidates who chose each wrong option have appealed, arguing their answer also works, and the board must justify rejecting an appeal. The pipeline's gate must justify accepting an item; the grader must justify refusing a student. Those pressures point in opposite directions.
• The verdict is computed, not chosen. The agent returns atomic judgments (is this option acceptable, is this claim true, is the sentence natural), and ordinary code turns those into a grade. A model that never names the grade cannot be lenient about the grade.

The number that decides whether you can ship

Partway through, the framing shifted, and this is the most transferable idea here.

I had been reporting yield: what fraction of generated candidates survive. It is a natural number to reach for and it is the wrong one to make decisions with. Yield is an economics metric. It tells you what content costs.

The metric that decides whether you can put content in front of a human is precision among accepted items: of everything your filter approved, how much is actually correct?

The two can move in opposite directions, and you want them to. A stricter filter lowers yield and raises precision. If you are optimising yield, you will loosen exactly the gate that was protecting your users.

Here is the funnel for one run, all numbers from the artifacts:

43.6% looks like a broken generator. It is not a number I care about. Across 233 accepted items from this pipeline, independently graded, the harmful false-accept rate is 2 items, 0.9%: one ambiguous, one that tested vocabulary rather than grammar. Zero contained a factually wrong explanation.

Cost: USD 0.168 per accepted item, paid once. The content is shared across every user of the app, so the marginal cost per learner trends to zero.

That is the trade. Burn 56% of what you generate, and ship content you can defend.

The experiment I expected to work, and did not

The remaining weakness was not truth, it was sharpness. Plenty of accepted items were correct and dull: the wrong options were so obviously impossible that no grammar knowledge was needed.

The hypothesis was that the generator lacked a model of what real exam items test and how their distractors are built. The expensive version of fixing that is to ground generation in real published exam material, which raises copyright and terms-of-use questions I did not want.

So I tested the cheap, zero-risk version first. I had a strong model read my own 281 independently graded items and abstract them into a taxonomy: grammar categories, learner confusion types, distractor strategies, what context a sentence must supply for exactly one answer to work. No third-party material, no copying, only my own data and general knowledge of the format. Twenty-four patterns, drafted for USD 0.63.

Then a controlled A/B, both arms through the identical, untouched trust pipeline. I pre-registered the primary metric before running it: the share of accepted items where an independent grader judged both wrong options genuinely tempting.

Pattern grounding did nothing at all for the thing I built it for. Standardised to the same mix of grammar categories, the two arms score 71.1% and 72.0%. Identical.

What it did instead was fix a problem I had not framed as the priority. Sentence-first generation could only test whatever grammar happened to appear in my cached sentences, and those sentences are beginner material, so it produced one single B1 item out of 75. For a feature aimed at B1, that is not a quality problem, it is an existence problem. Pattern-first produced 33.

Two things I would keep from this. Test the cheap, zero-risk version of an idea before the expensive one, because the cheap version may answer the question completely, as it did here: I no longer have any reason to touch copyrighted material, since the mechanism I wanted it for turned out not to matter. And pre-register what would count as success, because with a result this ambiguous I could have told myself a convincing story either way.

What I cannot fix by adding more agents

An article about validation that does not state its own limits is marketing. Three:

Every grader is a Claude model. They differ in prompt, incentive and decision procedure, which is enough to catch prompt-level blind spots and demonstrably did. It is not enough to catch a blind spot they all share. If every model in the family misjudges some construction, it passes every stage of my pipeline and I never find out. No number of panels fixes that, because the panels are drawn from the same distribution.

I found a fail-open bug in my own measurement harness. My grader combined two sub-verdicts into a grade in code, and when a sub-agent failed, the missing verdict fell through to PASS. It would have reported perfect precision on 36 items containing no data whatsoever. I caught it while tallying, only because the pass rate looked too clean. Grading code needs the same paranoia as production code, and it should fail closed. If it did that to me, it is doing it to somebody else right now.

One of my measurements was confounded and I nearly reported it as a clean comparison. My harshest grader was cut off partway and had only seen the easiest third of the items, which made its 57.9% pass rate not comparable to another grader's 89.3% on the full set. Different items, not different strictness.

The honest conclusion is that all of this validates the filter, not the models' knowledge of German. Which is why the last stage of the pipeline is not a model at all: every exercise in the app carries a "this looks wrong" button, and those reports go to a human. A learner who knows German correctly is the only instrument that reaches the errors my graders share.

What transfers

If you are building a pipeline that generates content for humans to learn from:

Separate the thing that decides from the thing that explains. An explanation written by the call that made the decision is a rationalisation, and rationalisations are where the falsehoods live.

Move every check you can into code. My cheapest, most reliable gates are regular expressions. They never hallucinate, they cost nothing, and they run on every item forever.

Make your judges structurally different, not just numerous. Different framing, opposite incentive, and let code compute the verdict from atomic judgments the model does not know it is aggregating.

Report precision among accepted items, not yield. They move in opposite directions, and only one of them is about your users.

Keep trust and polish as separate bars. I nearly built a gate that rejected correct, well-explained exercises for being slightly too easy, which would have starved the feature of content while protecting nobody. Trust is absolute: an item that might be wrong never ships. Polish is a preference: serve the sharper item when you have one, and a safe easy one otherwise.

Never repair a doubtful item. Drop it and generate another. Repairing means carrying a patched-over doubt into production forever. Sentences are cheap; trust is not.

The first version of this pipeline would have taught thousands of learners that des is a preposition. The measured version does not. That gap, between output that reads well and output you can defend with numbers, is most of the work.

I build KorrektWort alone, and KorrektCode is where I work on other people's systems. If you are shipping AI-generated content and cannot yet say what your accepted-item precision is, that is the number I would start with.