One model reported its confidence honestly, while the other claimed 100% certainty even when it was wrong.
An evaluation tested five confidence signals for LLM document extraction on 200 receipts and invoices; model choice changed which signals worked best
Key Takeaways
- An evaluation of five error signals for LLM-based document extraction showed that the most useful confidence signal depends strongly on the model.
- Stated confidence was well-calibrated for Gemini 3.5 Flash-Lite, enabling 52.5% of documents to be auto-accepted at a 5% error threshold, but it was nearly useless for Gemini 3.1 Flash-Lite, which claimed 1.0 confidence on most fields including many wrong ones.
- Arithmetic checks performed below chance, cross-model agreement helped mainly when the checking model was stronger, and a combined regression was most effective for the less calibrated model.
- Field-level accuracy scores were not enough: document-level coverage under a fixed risk threshold revealed which signals could actually reduce human review.
- For practical document automation, confidence calibration should be measured on the target model and task rather than assumed.
Meta description: An evaluation tested five error signals for LLM document extraction on 200 receipts and invoices; combined and stated confidence worked differently by model.
Tags: LLM document extraction, model confidence, document AI, evaluation metrics, Gemini Flash-Lite
Featured image: A clean abstract illustration of invoices, receipts, and confidence gauges in soft blue and amber tones, with no real people, no logos, and no text.
An evaluation of LLM document extraction tested five ways to flag wrong fields on 200 receipts and invoices, using Gemini 3.1 Flash-Lite and Gemini 3.5 Flash-Lite to extract every field and score each output with stated confidence, arithmetic checks, repeated-sample agreement, cross-model agreement, and a combined regression. For teams that automate document review, the test offers a practical warning: the best signal depends on the model. Stated confidence was useful for Gemini 3.5 Flash-Lite but nearly useless for Gemini 3.1 Flash-Lite, cross-model agreement helped only when the checking model was stronger, and arithmetic checks scored below chance. The test uses the same harness and runs as the author's previous article on per-field regression tests; the code and every number are in extraction-evals.
The test setup and scoring metrics
The documents were 100 real receipts from CORD (Park et al., 2019) and 100 generated invoices with exact labels, taken from the test split. That produced about 2,000 scored fields per model. On these documents, Gemini 3.1 Flash-Lite got 4.0% of fields wrong and Gemini 3.5 Flash-Lite got 1.4% wrong. At the document level, where every field must be right, 32% of Gemini 3.1 Flash-Lite documents contained an error and 10.5% of Gemini 3.5 Flash-Lite documents did.
Five signals were computed for each field. Higher scores meant the field was more likely to be correct.
- stated: the model's own confidence, a number from 0 to 1 returned for every field.
- checks: domain checks on the output, such as line totals summing to the subtotal, or subtotal plus tax equalling the total. A field got 0 if a failed check touched it, 1 if only passing checks touched it, and 0.5 if no check covered it.
- consistency: Gemini 3.5 Flash-Lite was run five times per document. The score is the share of the other samples that give the same value (Wang et al., 2023; Manakul et al., 2023).
- cross_model: 1 if the other model gives the same normalized value, 0 if not.
- combined: a logistic regression over all the above signals. It was cross-fitted: documents were split into five folds, and each fold was scored by a model fitted on the other four, so no document was scored by a model that had seen it.
Token log-probabilities would have been a sixth signal, but the Gemini models used did not return them.
The signals were judged with three numbers. AUROC is the probability that a wrong field gets a lower score than a right field; 0.5 is chance. ECE, or expected calibration error (Guo et al., 2017), asks whether a probability signal is trustworthy: when the signal says 0.9, are 90% of those fields right? Coverage at a fixed risk (Geifman and El-Yaniv, 2017) is the number that matters most in practice. Documents were sorted from most to least confident, accepted from the top, and the process stopped before the accepted set contained more than 5% errors. A document's confidence was its least confident field, and a document counted as correct only if every field was right. This measures the share of documents that never need a person. The evaluation also plots document-level risk against coverage by accepting the most confident documents first; the y-axis shows the share of accepted documents that contain at least one wrong field.
Results at a glance
| Model | Signal | AUROC | ECE | Docs auto-accepted at 5% error |
|---|---|---|---|---|
| Gemini 3.1 Flash-Lite | stated | 0.67 | 0.037 | 0% |
| Gemini 3.1 Flash-Lite | checks | 0.41 | - | 0% |
| Gemini 3.1 Flash-Lite | cross_model | 0.88 | - | 0% |
| Gemini 3.1 Flash-Lite | combined | 0.93 | 0.007 | 58.5% |
| Gemini 3.5 Flash-Lite | stated | 0.79 | 0.004 | 52.5% |
| Gemini 3.5 Flash-Lite | checks | 0.41 | - | 0% |
| Gemini 3.5 Flash-Lite | consistency | 0.76 | 0.009 | 0% |
| Gemini 3.5 Flash-Lite | cross_model | 0.65 | - | 0% |
| Gemini 3.5 Flash-Lite | combined | 0.76 | 0.005 | 7% |
The table shows a split result. For Gemini 3.1 Flash-Lite, the combined signal was the only one that auto-accepted documents at the 5% error threshold, reaching 58.5%. For Gemini 3.5 Flash-Lite, stated confidence alone accepted 52.5% of documents, while the combined signal accepted only 7%. Several field-level AUROC scores looked good, but at the document level they accepted nothing.
Why the signals behaved differently
Stated confidence was the clearest model-dependent result. Research cited in the evaluation suggests that language models often say they are very sure whether they are right or wrong (Xiong et al., 2024), while verbal confidence can be better calibrated than token probabilities for models tuned with human feedback (Tian et al., 2023). Both patterns appeared, one per model.
Gemini 3.1 Flash-Lite said 1.0 on 96% of fields. About 50 of its 80 errors came with that 1.0 confidence. On the 21 fields where it said 0.9, it was right on 57%. Gemini 3.5 Flash-Lite used the confidence range, and the range meant something: at 0.9 it was right 90% of the time, at 0.95 it was right 96%, and at 0.99 it was right 99%. Its expected calibration error was 0.004. Ranking documents by its least confident field auto-accepted 52.5% of them at a 5% error rate, with nothing else involved.
The reason for the difference is not established. The models are from the same family and used the same prompt. One possible explanation is a mix of training and difficulty: the task is harder for the smaller model, and overconfidence is often worst on the hardest questions in human calibration studies. The practical point does not depend on the explanation. Whether stated confidence is usable is a property of the model, and it must be measured on your documents. A threshold tuned on one model does not transfer to the next one, even within a family.
Cross-model agreement was a strong filter for the weaker model. Gemini 3.1 Flash-Lite and Gemini 3.5 Flash-Lite gave the same value for a field on 96.2% of fields. When they agreed, that value was wrong about 1% of the time: 19 of 1,922 fields for Gemini 3.1 Flash-Lite. When they disagreed, on 76 fields, Gemini 3.1 Flash-Lite was the one wrong 61 times, Gemini 3.5 Flash-Lite was wrong 9 times, and both were wrong once.
That means disagreement with Gemini 3.5 Flash-Lite found 61 of Gemini 3.1 Flash-Lite's 80 errors, giving it an AUROC of 0.88 and making it the best single signal for the smaller model. For Gemini 3.5 Flash-Lite, the same disagreements flagged only 9 of its 27 errors, because most disagreements were Gemini 3.1 Flash-Lite mistakes. Its AUROC was 0.65. A second model is a good error detector exactly when it is at least as good as the first. Checking a strong model with a weaker one mostly finds the weaker model's errors.
The 19 errors where both models agreed were the interesting ones. They were shared mistakes. When audited against the images, they were mostly the same few things: zero-priced toppings listed as items, a total filled in on a receipt that does not print one, and a brand name taken for the legal name. Both models are from the same family, so two models from different vendors might share fewer of these mistakes, but the evaluation did not test that.
Cross-model agreement accepted 0% of documents because agreement is a yes-or-no signal. At the document level there were only two groups: documents where the models agreed on every field, 141 of 200, and the rest. Among the 141 agreed documents, 11 still had an error in Gemini 3.1 Flash-Lite's output, or 7.8%. With nothing to rank inside that group, the signal cannot get under a 5% error rate. It is a good filter and a bad ranker, and teams need something continuous next to it.
Self-consistency helped only at the sampling-noise level. Gemini 3.5 Flash-Lite was sampled five times per document, and the consistency score had an AUROC of 0.76, close to its stated confidence, but at five times the calls. The weak point was that on 1,970 of about 2,000 fields, all five samples agreed, and 13 of Gemini 3.5 Flash-Lite's 27 errors were among them. Half of the better model's mistakes were mistakes it made every time. Self-consistency catches errors caused by sampling noise, such as a digit misread on one pass and not another. It cannot catch systematic misreadings, such as reading the brand in the header as the company name, because the model makes that mistake every time. For extraction, where most inputs are clear and many errors are systematic, that limits the signal a lot.
Arithmetic checks scored below chance, but that behavior is explainable. Their AUROC was 0.41 for both models, below a coin flip. A failed check is a strong signal: for Gemini 3.1 Flash-Lite, a failed check touched 15 fields, 7 of them were wrong, an error rate of 47%. But it almost never fired. Only 15 fields out of 2,000 were touched by a failed check, partly because a failed check triggered a repair turn first. Fields covered only by passing checks numbered 1,203, with 64 wrong fields and a 5.3% error rate. Fields with no check covering them numbered 780, with 9 wrong fields and a 1.2% error rate.
The checks mostly covered amount fields, which are the fields that go wrong most. Fields with no check at all were nearly always right. As a result, "covered by a passing check" ended up meaning "this is a field that goes wrong," and the ranking came out inverted. The lesson is that a check tells you about the documents it fails on, and tells you almost nothing about the documents it passes. Most errors pass every check, because a misread subtotal and a misread total can still add up, and a field the checks do not touch cannot fail one. Checks are worth running for the repair turn and as a hard flag, but not as a confidence score.
Combining the signals gave a large gain for Gemini 3.1 Flash-Lite and a loss for Gemini 3.5 Flash-Lite. For Gemini 3.1 Flash-Lite, the combined score reached AUROC 0.93, calibration error 0.007, and auto-accepted 58.5% of documents at a 5% error rate, where no single signal accepted any. The evaluation suggests that most of the work is done by agreement with Gemini 3.5 Flash-Lite, with stated confidence and the checks breaking ties inside the agreeing group, though the weights were not inspected fold by fold.
For Gemini 3.5 Flash-Lite, the combination accepted only 7% of documents at 5% error, against 52.5% for stated confidence alone. The likely reason is simple: Gemini 3.5 Flash-Lite made 27 errors in about 2,000 fields, and a regression fitted on four-fifths of that data had about 20 errors to learn from. It learned weights for signals that were noisy for this model, such as cross_model and checks, and those noisy weights drowned out the signal that worked. Cross-fitting keeps the number honest, but it does not make 20 examples enough. On a model this accurate, the safer choice is to keep the single signal that is calibrated and stop there, until there is enough labeled data to fit something better.
What builders should do next
The evaluation points to a few practical rules for teams using LLM document extraction in invoice and receipt workflows.
First, measure stated confidence before using it, per model. It was the best signal for one model and nearly useless for the other, even with the same prompt. A threshold tuned on one model does not transfer to the next one, even within the same family.
Second, use a second model as a filter, not as a score, and choose a strong checker. Agreement removes most errors when the checking model is at least as good as the model being checked. Because agreement is a yes-or-no signal at the document level, pair it with something continuous to rank inside the agreeing group.
Third, do not expect resampling to find systematic errors. Half of the better model's errors appeared in all five samples. Self-consistency can catch unstable mistakes, but not mistakes the model makes every time.
Fourth, treat arithmetic checks as flags. A failed check means look at the document. A passed check means very little, because many errors pass every check, especially when a misread subtotal and a misread total still add up.
Fifth, report coverage at a fixed risk, at the document level. AUROC on fields looked good for several signals that, at a 5% document-error threshold, accepted nothing. The useful number is what share of documents can be accepted without a person.
Sixth, do not fit a combined model on a handful of errors. For Gemini 3.5 Flash-Lite, the combination did worse than one good signal. With 27 errors, there was not enough signal for the regression to learn cleanly.
All of these results come from two small models from one family on 200 documents, so the specific numbers should be treated as one data point, while the method is the part worth reusing. The evaluation says that different-vendor cross-checks and token log-probabilities are the two signals it would most like to add next.
The signals, the risk-coverage code, and every number are in github.com/abdulm-ai/extraction-evals. The command python -m xeval report rebuilds them from the committed predictions without an API key.
References
- Xiong, M. et al. (2024). Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs. ICLR 2024.
- Tian, K. et al. (2023). Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback. EMNLP 2023.
- Guo, C., Pleiss, G., Sun, Y. and Weinberger, K. Q. (2017). On Calibration of Modern Neural Networks. ICML 2017.
- Geifman, Y. and El-Yaniv, R. (2017). Selective Classification for Deep Neural Networks. NeurIPS 2017.
- Wang, X. et al. (2023). Self-Consistency Improves Chain of Thought Reasoning in Language Models. ICLR 2023.
- Manakul, P., Liusie, A. and Gales, M. J. F. (2023). SelfCheckGPT: Zero-Resource Black-Box Hallucination Det.
Comments (0)