The Key Was Not the Verdict

For a stretch of time our public case pages told institutional readers that certain detection dimensions had fired on certain companies. On some of those companies, those dimensions had not fired. They had done the opposite — the evidence they carried argued against raising a flag, and we printed them as though they argued for it.

Nobody wrote a wrong threshold. There was no threshold at all. That is the whole defect, and it is worth walking through because the shape of it is general.

What the engine actually stores #

When our engine marks a company, it snapshots the evidence value contributed by each dimension. These values live on a scale where one is the zero point: an evidence value above one means that dimension supports raising a flag; below one means it argues against; exactly one means it declined to bet at all — the mathematical form of “no data, no opinion.”

So the set of dimensions that genuinely contributed is not the set of dimensions present in the record. It is the subset whose value exceeds one. The record contains every dimension that was evaluated, including all the ones that looked and found nothing.

What the projection layer did #

Two separate pieces of code — the public endpoint, and the annotation console — independently arrived at the same wrong answer. Both enumerated the keys of the stored evidence object and treated that key list as the list of triggered dimensions.

It is an easy mistake to make and almost impossible to see in review, because the code reads correctly. You are looking at a record of the engine’s evaluation; the keys are the dimensions; presenting the dimensions seems like presenting the finding. The step that is missing is invisible: nobody dropped a filter, because nobody had written one.

What it cost, measured after the fact:

• altman_z appeared as triggered on 28 of 28 companies where it was present — and on all 28, its value was below one. In every single case the dimension was arguing against the flag, and the card said it supported it.
• zmijewski_x: 30 of 30, the same shape.
• ohlson_o: present on 29 companies, genuinely above one on exactly one of them.

Three dimensions, nearly a hundred company-level statements, essentially all of them inverted. Not noise around an edge case — a systematic reversal in the same direction every time.

Why it is not a threshold bug #

The tempting summary is “we used the wrong cutoff.” That would be a smaller and more comfortable problem. The truth is that the cutoff did not exist, and the reason it did not exist is structural: the projection layer invented a verdict the engine had never produced.

The engine produced a set of measured values. Somewhere between the database and the page, that turned into a categorical claim — this dimension fired — which the engine had not made and which no code was responsible for computing. The display layer filled the gap with the most available proxy, presence, and presence is not evidence.

So the rule we wrote afterwards is not about thresholds:

A projection layer may not produce a judgment the upstream did not make. If the upstream did not make it, leave it blank and say so.

Blank is a legitimate output. A card that says “the engine evaluated these dimensions” is honest. A card that says “these dimensions fired” when nothing computed which ones did is not — and the second is more comfortable to build, because it never leaves an awkward empty space on the page.

The part that generalises #

Two things came out of this that we now apply well beyond that endpoint.

One definition point. Both offenders were correct-looking implementations of the same missing idea, written independently. That is the signature of a concept that has no home in the code. “Which dimensions contributed” now has exactly one definition, and the endpoint and the console both call it. Two implementations of the same rule will diverge; the only question is when.

Read the snapshot, never recompute. The evidence values are read from what was stored at the moment of the mark — recomputing them today is forbidden. It is tempting, because today’s numbers are right there and the stored ones are stale. But a card explaining a decision made months ago must show the evidence as it stood then; anything else is using today’s information to justify yesterday’s judgment. We have written separately about why point-in-time discipline reaches everywhere — this is the same rule arriving at the display layer, where it is easiest to forget it applies.

How it was found #

Not by a test. The comparison that exposed it — pulling the stored values for every dimension the cards claimed had fired — is one we could have run on day one and did not think to. What prompted it was someone reading a case page and finding a listed dimension implausible for that company.

We have a lot of automated checks. The one that caught this was a person with domain knowledge looking at output and saying that doesn’t sound right. We have not found a way to automate that, and we have stopped trying to.

All figures are system-level results current as of publication date; methodology parameters are intentionally omitted.