Schema-constrained LLMs: identical inputs, identical outputs

A model that gives a different answer on Tuesday than it did on Monday can’t sign a certified document. That was the whole problem on the asset-extraction pipeline: the extraction was good, but “good” wasn’t the bar. Repeatable was.

The fix has two halves. Constrain the model to a JSON schema so the shape of the output is never in question, then run a deterministic post-processing pass so the values aren’t either — normalise units, resolve references against the register, and drop anything the schema didn’t ask for.

response = model.generate(
    prompt,
    response_schema=AssetRecord,   # the shape is fixed
    temperature=0,
)
record = normalise(AssetRecord.model_validate_json(response.text))

Identical inputs, identical outputs, every run. That is what lets a language model touch documents somebody has to put their name to.

The temperature is zero, the schema is a Pydantic model, and the post-processor has golden-file tests. None of it is clever. All of it is what made the client comfortable letting the pipeline run unattended, which was the only thing that actually mattered.

The thing I’d tell anyone starting here: the model is the least interesting part of the system. Decide what “correct” means first, write the test that proves it, and only then choose where the language model goes.