Cases · Run on
Receipt extraction: 71.1% field accuracy against 51.6% for Sonnet
A fine-tuned gpt-oss 20B turned receipts into JSON more accurately than Claude Sonnet 5.5, GPT-6 Luna and Gemini 3.8 Flash, at $0.124 per 1,000 documents.
Setup
- Data: 800 receipts from the CORD v2 train split (CC BY 4.0). Input is the OCR words in reading order, output is the JSON with menu, sub_total and total.
- 649 training and 114 held-out test receipts (37 rows skipped). No synthetic data.
- Goal: “Lowest cost”. Candidates on 18 Oct 2026: gpt-oss 20B and Nemotron 3.5 Lightning 30B. Winner: gpt-oss 20B.
- Compared with Claude Haiku 5.5, GPT-6 Luna, Gemini 3.8 Flash, Claude Sonnet 5.5 and the untrained base model.
Result
| Model | Field accuracy | Median latency | Cost per 1,000 calls |
|---|---|---|---|
| Your model (gpt-oss 20B, fine-tuned) | 71.1% | 701 ms | $0.124 |
| gpt-oss 20B (untrained) | 14.3% | 1,374 ms | $0.204 |
| Claude Haiku 5.5 | 3.9% | 3,452 ms | $0.419 |
| GPT-6 Luna | 49.7% | 3,746 ms | $0.212 |
| Gemini 3.8 Flash | 37.0% | 4,678 ms | $2.559 |
| Claude Sonnet 5.5 | 51.6% | 1,474 ms | $3.220 |
Time and cost
- Time: 13 min 23 s from upload to result. Training was 9 min 15 s.
- Run cost: $1.38.
Notes
- A field is a top-level key (menu, sub_total, total) and must match in full. Almost all of the difference is in menu: the line items and their shape.
- On a one-item receipt the expected menu is an object. GPT-6 Luna, Sonnet and Gemini returned a list, so menu counted as wrong although the values were right.
- Claude Haiku 5.5 added every key from the description with null, so no object matched.
- In this run the personal-data masker replaced some prices with placeholders (210 of 800 rows). Every model got the same masked input, so the comparison is fair, but absolute scores are lower than they would be.
The full write-up: Receipt and invoice extraction: own model vs GPT, field by field.