Direct Preference Optimization
AssessTechniques
A training method that optimizes models from chosen-vs-rejected preference pairs.
Why it's here
Placed in Assess: 3 article(s) of evidence from 1 source(s), led by research-stage coverage, with 1 in the last 30 days. Confidence 41%.
Evidence (3)
- 6Hugging Face Blog·7/16/2026researchDharmaOCR beats newer OCR models on Brazilian Portuguese
Hugging Face Blog reports that DharmaOCR outperformed Mistral OCR4 and Unlimited-OCR on Brazilian Portuguese benchmarks despite newer architectures. The article attributes the advantage to domain-specific supervised fine-tuning followed by Direct Preference Optimization, which improved both extraction quality and output stability.
- 7Hugging Face Blog·6/3/2026researchDPO Cuts Text Degeneration in OCR Models Beyond Chatbots
Hugging Face Blog describes how Dharma-AI used Direct Preference Optimization (DPO) after supervised fine-tuning to reduce text degeneration in structured OCR models. Across tested model families, degeneration fell in every case, with an average reduction of 59.4% and a best-case drop of 87.6%. The article argues that DPO can use a model's own failure outputs as rejection pairs, extending preference optimization beyond chatbot alignment.
- 7Hugging Face Blog·3/31/2026framework_updateTRL v1.0 marks a stability shift for post-training tooling
Hugging Face released TRL v1.0, presenting it as a more stable library for post-training workflows that now powers production use. The update emphasizes adapting to a fast-changing field, with support for more than 75 post-training methods including PPO, DPO-style approaches, and RLVR methods such as GRPO.