Problem class: constrained generation. An AI whose correct output is sometimes a refusal, enforced by a control layer rather than by prompt compliance.
An estate-planning client completes intake. Somewhere in the free-text notes: a second marriage, and a child from the first who is mentioned once and never again.
The drafting assistant assembles a will. It produces a clean, complete, internally consistent document — because a complete document is what it was built to produce. The ambiguity about which children are provided for was not flagged. It was resolved. Quietly, by whichever reading made the template close.
The attorney reviews it. Nothing looks wrong, because nothing looks wrong — a confident document reads as a correct one. It is executed. The defect surfaces at probate, years later.
A language model optimised to produce output will produce output. Given an incomplete instruction it does not stall — it interpolates, fluently, and the fluency is indistinguishable from correctness at review time. The same mechanism produces the intake assistant that answers a question constituting legal advice, the review pass that resolves a contract ambiguity instead of surfacing it, and the citation to a case that does not exist. These are not four problems. They are one problem wearing four hats.
Instructing the model not to do this does not work. A refusal held only by a prompt degrades in two directions: under pressure it capitulates and answers anyway, or it over-triggers and refuses routine work until staff stop using it.
A K–12 learning platform engaged us with a product requirement that is, structurally, the one above. The AI's job was to advance a student's reasoning without resolving it for them. Give away the answer and the product has no reason to exist.
A single model with a well-written prompt could not hold the line. We tried. It capitulated under pressure and stonewalled on routine input. What worked was moving the decision out of the generating model entirely: five specialised agents orchestrated in LangGraph, release authority isolated in a supervisor that is not a generating model, retrieval grounded in a vector store, every step traced. No agent that generates text also holds the authority to release it. That separation is code, not instruction.
“Our biggest challenge was taking a complex ‘teachable agent’ concept where the AI helps students learn without just giving away the answers. They didn’t just build the software. They engineered a robust AI framework that solved the actual teaching challenges.”
FOUNDER & CEO · K–12 LEARNING PLATFORMStructurally identical, with the agents re-specified for legal drafting: retrieval over firm precedent and actual authority, a drafter limited to cited clauses, a conflict checker, a gap detector — under a completeness supervisor whose halt path is a first-class output. A halt is a successful output. The system is measured on what it refuses to produce, not only on what it produces.
The $2,000 deposit is credited toward the build and refundable for two weeks after the plan lands. The plan is yours either way.
Book a scoping call →