Blog ·

Content moderation with a small model you own

Moderation has three requirements that pull against prompted frontier models. It runs on every post, so the cost per call adds up. It needs consistent decisions, because users notice when the same comment is removed one day and allowed the next. And your policy is specific: what counts as a personal attack on a gaming forum is different from what counts on a parenting forum.

A fine-tuned model learns your policy from your past decisions. At temperature 0 it gives the same answer for the same input, and the weights stay the same until you retrain.

The setup

Data. Civil Comments (CC0): about 2 million public comments, each scored for toxicity and subtypes such as insult, threat and identity attack by human raters. We sampled 500 comments per class and turned the toxicity score into three labels: allow below 0.3, remove at 0.7 and above, and review in between.

Split. 1,275 comments for training, 225 held out.

Models. With the "lowest cost" goal, Honed trained gpt-oss 20B and Nemotron 3.5 Lightning 30B and kept gpt-oss 20B. It compared the winner with Claude Haiku 5.5, GPT-6 Luna, Gemini 3.8 Flash, Claude Sonnet 5.5 and the untrained base, each prompted with the three labels and a written policy.

Scoring. Accuracy and macro F1 on held-out comments. With three balanced classes, guessing scores 33%.

Results

Model Accuracy Macro F1 Median latency Cost per 1,000 comments
Fine-tuned gpt-oss 20B 74.7% 74.9% 372 ms $0.05
GPT-6 Luna 44.4% 38.7% 1,350 ms $0.05
gpt-oss 20B, untrained 43.1% 38.7% 472 ms $0.07
Claude Haiku 5.5 41.3% 33.1% 637 ms $0.06
Gemini 3.8 Flash 40.4% 30.4% 2,501 ms $0.74
Claude Sonnet 5.5 40.4% 31.1% 806 ms $1.08

The fine-tuned model was 30 points more accurate than the best prompted model. It was also the fastest and the cheapest per call. The run took 26 minutes from upload to result and cost us $1.52 in compute.

Where the models disagreed

The prompted models were lenient. Given the same written policy, they marked many comments allow that the Civil Comments raters had scored as toxic. The fine-tuned model learned where the raters drew the line.

  • "Only a complete moron would raise business taxes 20%…" Raters: remove. Fine-tuned: remove. Haiku, Gemini and Sonnet: allow. Luna: review.
  • A comment ending in an abbreviated insult. Raters: remove. Fine-tuned: remove. Haiku, Luna and Gemini: review. Sonnet: allow.
  • "…These scumbags are in both parties…" Raters: remove. Fine-tuned: review, a miss. Every other model: allow.

In the third example the fine-tuned model missed, but it was still closer to the raters than any other model.

The published baseline

LoRA Land included a toxicity task built on Jigsaw data. In that 2024 study the best fine-tuned 7B model scored 0.867 against GPT-4's 0.754 (arXiv 2405.00732).

The same decision tomorrow

A hosted model can change without your code changing. Gemini API aliases route to newer models (Gemini models), and Anthropic can retire a model 60 days after notice (Anthropic). For moderation, a model update can change your policy without anyone deciding to.

A fine-tuned model on an open base keeps its weights until you retrain it. When you change the policy, you change the training examples, retrain, and check the new version on the same held-out set before it goes live.

On your own queue

Export past moderation decisions: the post text and what your moderators did with it. Add the reason codes if you have them, since they make useful labels. Mask usernames before export. Then compare the fine-tune with the model or rules you use now.

Sources