Simon Willison published a post on August 14 pointing to a technique from search engineer Doug Turnbull: Don’t classify, hallucinate. The premise is straightforward — rather than feeding an LLM a vocabulary of existing tags and asking it to classify, have the model generate novel tag descriptions freely, then use vector embeddings to find the closest real tags. Constrained classification underperforms; free generation plus post-hoc grounding outperforms.
Most enterprise teams are doing the constrained version. They have a taxonomy, they hand it to the model, and they wonder why classification quality is mediocre. The framing here suggests the problem is not the model — it is the architecture.
flowchart TD Doc[Document to classify] Doc --> C[Constrained path:<br/>give LLM vocabulary list] Doc --> F[Free-generation path:<br/>no vocabulary constraint] C --> C2[LLM selects<br/>closest label] C2 --> C3[Misclassification<br/>when vocab has gaps] F --> F2[LLM generates<br/>semantic description freely] F2 --> F3[Vector embedding<br/>finds closest real tag] F3 --> F4[Accurate classification<br/>even across vocabulary gaps] class C3 bad class F3 accent class F4 good classDef good fill:#163a26,stroke:#44cc77,color:#d7ffe6; classDef bad fill:#3a1620,stroke:#ff5555,color:#ffd9d9; classDef warn fill:#3a2e16,stroke:#ffaa33,color:#ffe9c7; classDef accent fill:#15233b,stroke:#4488ff,color:#dce9ff;
The rundown: what Willison is pointing to
Turnbull’s core argument, as Willison frames it: when you constrain an LLM to classify into a fixed vocabulary, you are asking it to do something that sounds precise but actually degrades output quality. The model has to fit what it would naturally say into whatever bucket is least wrong. If the vocabulary does not have the right bucket, you get the closest approximation — which may be misleading.
The alternative: let the model generate freely. It produces richer, more accurate semantic descriptions when unconstrained. You then run those descriptions through a vector embedding model and find the closest match in your actual taxonomy. The “hallucination” — the model generating something not in your vocabulary — becomes useful signal rather than error, because it accurately describes what the content is about.
This reframes “hallucination” as a feature in the right architectural context. The LLM is not confabulating. It is generating an accurate description of something your vocabulary does not have a label for yet.
For the working software engineer
If you are building or maintaining a classification pipeline, this is an architectural change, not a prompt tweak. The implementation: strip the vocabulary constraint from your classification prompt, add an embedding step, and route generated descriptions through a similarity search against your tag index.
The practical tradeoffs to evaluate:
Latency. You are adding an embedding step. In most enterprise pipelines where classification happens offline or in batch, this is a non-issue. For real-time classification at the user-facing layer, it adds a small amount of latency worth measuring.
Cost. An additional embedding call is cheap relative to the classification call. Net cost impact is usually negligible.
Vocabulary maintenance. The constrained approach requires curating the vocabulary you hand to the LLM. The free-generation approach shifts that curation upstream — the vocabulary lives in your embedding index rather than in your prompt. In practice this is easier to maintain, because you add tags to the index without touching prompt instructions or rerunning evaluation sets.
Quality at the edges. Where this technique materially outperforms constrained classification is at the margins — unusual content, emerging topics, documents that cut across multiple categories. If your content set is well-bounded and your taxonomy is stable, the quality difference will be small. If your content is heterogeneous or your taxonomy is evolving, the difference is significant.
For business owners and operators
If you are investing in enterprise knowledge management, content classification, or RAG systems, the architectural choice your team makes at the classification layer will determine how well those systems hold up as content grows and diversifies.
Constrained classification systems tend to work when the vocabulary was built. They degrade over time as content drifts from what was anticipated when the taxonomy was designed. Documents end up classified into the “other” bucket or mis-tagged into the nearest available category, and search and retrieval quality quietly erodes. The maintenance loop becomes: update the vocabulary, retune the prompt, retest. Repeat quarterly.
Free generation plus embedding grounding avoids some of this. The model does not become less accurate as vocabulary gaps grow, because it is not constrained by those gaps. Adding a new tag means adding it to the embedding index — no prompt changes, no retest cycle.
The business case is really about maintenance cost and downstream quality. Classification errors are invisible until they surface in search results, a misrouted workflow, or a compliance audit where a document was incorrectly labeled. The free-generation approach is architecturally more resilient to the content drift that happens in any real enterprise over time.
My take
I spent time at H&R Block re-architecting the SOA for TaxCut 2007 — their flagship consumer tax software. The existing system had a rules engine that classified tax scenarios into a predefined set of categories, each mapped to a form and a calculation path. The rules were extensive, carefully maintained, and expensive to update whenever the tax code added an edge case.
What Turnbull’s approach reminds me of is the pattern that makes constrained classification attractive in the first place: it feels safe. The vocabulary is controlled. The outputs are bounded. You know what the model can and cannot say. That feels like governance.
The problem is that safety through constraint is brittle. Tax law adds a new edge case. Content expands into a new domain. The model starts picking the “other” bucket at a higher rate, and the constrained vocabulary becomes a liability rather than a control.
The version of this that holds up over time: constrain the actions the system takes — what it routes, triggers, or writes — not the language it uses to describe what it is looking at. Let the model generate freely at the classification step, then ground the output. The controls live downstream, not in the vocabulary list.
Put constraints where mistakes are costly, not where mistakes are informative. That is a consistent principle across AI system design, and this technique makes it concrete.
