What May Enter the Corpus

Our reasoning engine learns from one thing: signed annotations written by credentialed forensic accountants on historical filings. We have written about why we refuse to let a model generate that reasoning for itself. This post is about the other half — the standing rules for what is allowed into the training set at all, and one rule about what may never be used as the answer key.

These are written down because a corpus does not degrade by decision. It degrades by a series of individually reasonable admissions.

The exclusion that costs us the most #

An enforcement database is a textbook, never an answer key.

There is a public record of accounting enforcement actions. It is the obvious supervision source, and building on it is the default in this field. We read it, we learn from the cases in it, and we do not use it to define what our system is aiming at.

The reason is a selection effect that cannot be corrected away. Enforcement records contain frauds that were caught. Getting caught correlates with being large enough to investigate, visible enough to attract attention, and long-lived enough for a case to conclude. Train a system to reproduce that set and it learns the shape of detected-and-prosecuted fraud, which is not the shape of fraud. It will be most confident about the companies someone else already found.

We learned this the expensive way rather than the principled way. A graph model we built on exactly that supervision failed on held-out data, and the follow-up tests showed the label set was leaking size and survival through the structure. The rule was already written; the experiment is why it is now enforced rather than believed.

So enforcement cases are study material — for calibrating what a real misstatement looks like, for the reading list, for training the humans. They are downstream reference. They are not the target.

The five standing exclusions #

These govern the annotation corpus itself, and none of them has an override:

1. Client-commissioned work never enters. If a subscriber asks us to look at something, that engagement and everything in it stays out of the training set permanently. The value of a commissioned investigation belongs to the party who commissioned it.

2. Unpublished cases never enter while unpublished. A case only becomes eligible once it has been published. The reason is not secrecy — it is that an unpublished case has not yet survived the process that makes it worth learning from. One amendment: a closed unpublished case may enter, de-identified, as a negative example. A case we investigated and dropped is genuinely informative about where the line sits, and it is one of the few sources of good negatives we have.

3. Founder overrides enter with the fingerprints removed. When a judgment call is made at the top, the decision can teach; the individual’s stylistic habits should not. What goes in is the reasoning, not the voice.

4. Retracted cases enter — labelled as errors. This is the one that surprises people. When we publish something and later determine it was wrong, that case does not get deleted from the corpus. It goes in with an explicit error label, so the engine learns the shape of our mistakes as well as our successes. A model trained only on the cases we got right has never seen what being wrong looks like from the inside.

5. Personal data stays in its jurisdiction. Where processing happens is constrained by where the data originated, and vendor and node selection follow that constraint rather than convenience.

Why these are structural, not policy #

A rule that lives in a document gets followed until the week it is inconvenient. So the ones that can be structural are.

The most important: the training gate rejects any sample whose author is not a credentialed forensic expert — enforced in the database, not in a style guide. There is no code path that writes an annotation with a non-expert author, because the constraint rejects the row.

The retraction rule works the same way. Its label is a fixed enumerated value, so a retracted case cannot be quietly re-filed as an ordinary one; it either carries the error label or it does not go in.

This matters more than the individual rules. Every one of these exclusions costs something real — usable examples, and in the enforcement-database case, the largest labelled set available in the field. The pressure to make an exception will always arrive at a moment when the exception looks harmless. Written rules bend under that pressure. A check constraint does not have a bad day.

What we are actually building #

The corpus is not a dataset we assembled. It is a record that accumulates: every published flag carries the independent, blind-written reasoning chains behind it, and every one of those chains is simultaneously curriculum. Every dispute between annotators is a lesson. Every adjudication is a test case.

That is why the disagreements go in rather than getting cleaned out. A corpus containing only the cases where three experts agreed would teach a model that forensic judgment is easy and consensus is normal. Neither is true, and a judge trained on sanitised agreement is a judge that has never seen a hard case.

All figures are system-level results current as of publication date; methodology parameters are intentionally omitted.