Cases · Run on
Support routing: 100% on 294 held-out tickets
A fine-tuned GLM 5.3 Flash routed customer messages to the right intent on every held-out ticket, against 95.9% for Claude Sonnet 5.5 and 90.5% for Claude Haiku 5.5.
Setup
- Data: the public Bitext customer-support dataset, 2,000 rows. Input is the customer message, output is one of 27 intent labels.
- 1,671 real training examples and 294 held-out test examples, all from the upload. No synthetic data.
- Goal: “Most accurate” (the default). On 18 Oct 2026 it trained three candidates: Inkling Small, DeepSeek V4.1 Flash and GLM 5.3 Flash. Winner: GLM 5.3 Flash with LoRA (rank 32, 2 epochs). Inkling Small scored 99.7% (293 of 294). Since 20 Oct 2026 the goal trains Inkling Small alone.
- Compared with Claude Haiku 5.5 and Claude Sonnet 5.5 on the same test set.
Result
| Model | Accuracy | Median latency | Cost per 1,000 calls |
|---|---|---|---|
| Your model (GLM 5.3 Flash, fine-tuned) | 100.0% | 530 ms | $0.21 |
| Claude Sonnet 5.5 | 95.9% | 790 ms | $0.89 |
| Claude Haiku 5.5 | 90.5% | 580 ms | $0.042 |
| GLM 5.3 Flash (untrained) | 88.4% | 480 ms | $0.22 |
Time and cost
- Time: 1 h 49 min from upload to result. Training was 1 h 43 min 36 s, 95% of the time.
- Run cost: $13.77 (€12.23). Training was 95% of the cost.
Notes
- Macro F1: 100.0% for your model, 95.8% for Sonnet, 90.6% for Haiku, 88.1% for the untrained base.
- 100% is an upper bound. Bitext messages follow templates, so the task is easy. On real customer data accuracy will be lower.
- The fine-tuned model is 4.2× cheaper per call than Sonnet and 1.5× faster (median). Haiku is 5.1× cheaper per call than the fine-tuned model.
- The training cost is the app’s estimate (training tokens × price), not a provider invoice. One run on one dataset.
The full write-up: A fine-tuned small model vs Claude Haiku on support routing.