The Three Ways a Language Model Breaks Inside a Statement of Advice

Every licensee evaluating advice software asks the same question first: is the model any good? It is a fair question, and it comes with a number attached, which makes it a comfortable one to ask.

The more useful question is what the system does on the occasions when the model is not good. That answer is a design rather than a score, and a design is something you can read, argue with, and hold someone to.

Three Failures a Schema Check Cannot See

Structural validation on a Statement of Advice is the easy half. Every required section present, every placeholder filled, the right disclosures in the right order. A section can satisfy all of that and still be unusable, because the failure is in the prose rather than the structure.

Three classes of failure come from the model itself, and we catch them in a separate pass:

1. Refusal, or the persona breaking. The model writes "As an AI language model, I cannot" where the section should be. The same pass catches "I am a language model", "I do not have access to real-time data", and the various ways a model declines to assist with a request.
2. Leaked scaffolding. The model echoes its own instructions back into the document. "Write the X section." A word limit token. A line of formatting rules in capitals, sitting inside what is meant to be a recommendation to a retail client.
3. A bare JSON dump. Asked for prose, the model returns a structure.

None of these need a model call to detect. All three are found with patterns, in under a millisecond per section. That cost matters more than it sounds: at that price the check runs on every section of every document every time, with no sampling and no budget conversation about whether it is worth running today.

Why a Refusal Detector Ruins This Document Type

Here is where the obvious implementation falls over.

A Statement of Advice quotes the client verbatim. It has to. "I can't afford to lose capital before retirement" is a sentence that belongs in the document, in the client's own words, as part of the record of why the recommendation is what it is. First person speech is legitimate content here, and a naive refusal detector reads that line as the model declining to work.

So the check cannot be absolute. A refusal pattern only counts if it appears near the start of a response, or in a response short enough that the match could not plausibly be quoted material. The same pattern in the middle of a long section is left alone.

That is a positional rule, and it buys correctness on the common case by giving up coverage on an uncommon one. A section that produces four hundred words of usable prose and then breaks into a refusal will pass this check. We accepted that, on the reasoning that a refusal is something a model does instead of the work rather than after most of it. That is a bet about where refusals live, not a proof, and it is the part of this design most likely to need revisiting.

How a Pattern Earns the Right to Reject Work

Alongside the patterns that fail a section, there is a shadow set that does not: contracted refusals, references to policy, knowledge cutoff disclaimers, and short answers that end on a question mark. Those record a warning and let the section through.

The reason is that a check which rejects work is expensive when it is wrong. A false positive on a live document either discards good prose or sends an adviser back into a queue for no reason. A new pattern goes into the shadow set, fires against real documents for a while, and only becomes an enforcing rule once we can see what it actually catches. Observation first, enforcement second.

This is the least glamorous part of the design and the part that has saved the most rework.

Refusal Is a Feature in One Document and a Defect in Another

There is a tension worth naming, because it runs against everything else we publish about refusal.

In our hallucination benchmark from March 2026, half the test set was deliberately unanswerable: 120 questions, 60 with an answer in the corpus and 60 constructed to be domain plausible but impossible. Another manager's proxy voting policy. A Sharpe ratio for a quarter that appears in no document. An SEC rule. The point of that half was that in regulated work, the ability to refuse is a feature under test rather than an edge case.

On that half, BackPro refused 59 of 60, a 98.3 per cent refusal rate. Standard RAG refused 55.0 per cent and fabricated an answer on 31.7 per cent, which is more fabrication than the model with no retrieval at all managed on the same questions. Retrieval made it worse, because retrieval always returns something. Those are March 2026 figures on the architecture as it stood then, and the method is set out in the benchmark document.

Now put the same behaviour inside an advice document. A model that declines to draft the scope and objectives section has not exercised good judgement about missing evidence. It has produced a Statement of Advice with a hole in it where a client's circumstances should be.

Same behaviour, opposite sign, depending on which document the model is inside. Which is the real argument for making the check positional and document aware rather than a global rule about the model saying no. A single "did the model refuse" signal, applied across a product that answers due diligence questionnaires and drafts advice documents, would be wrong in one place whichever way you set it.

A Failed Check Should Come Back as an Instruction

The last piece is what happens after a pattern fires.

A failed check does not simply discard the section. It becomes a retry reason, and that reason is handed back to the model verbatim, in the same words the checker used. The second attempt is told what was wrong with the first rather than being asked again and hoped over.

Two things follow from that, one useful and one awkward. The useful one is that a failure leaves a named reason in the record, so a reviewer looking at a regenerated section can see what triggered the regeneration instead of finding a silent gap. The awkward one is that the checker's phrasing is now prompt text. A vaguely named check produces a vague instruction, and the quality of the retry depends on wording we originally wrote for a log line.

So the design, stated plainly: deterministic checks cheap enough to run constantly, matches that are contextual rather than absolute because the document quotes people, new rules that observe before they enforce, and failures returned to the model as instructions rather than swallowed.

These checks sit inside Statement of Advice drafting, and a practitioner could reasonably reject the positional rule. It leaves the long section that turns into a refusal partway through uncaught, and we do not currently have a pattern for that case.