BBQ
HoldTools
A benchmark for evaluating social bias and stereotype reliance in language models.
Why it's here
Placed in Hold: 1 article(s) of evidence from 1 source(s), led by research-stage coverage, with 0 in the last 30 days. Confidence 24%. Low accumulated evidence, so it defaults conservatively pending more signal.
Evidence (1)
- 7Hugging Face Blog·9/1/2026researchBenchMIRT audits what LLM benchmarks measure
Hugging Face highlights BenchMIRT, a new auditing method that uses multidimensional item response theory to analyze LLM benchmark prompts at the item level. Applied to results from 100 models across 16 benchmarks, it found two stable underlying dimensions—safety and general reasoning—and showed that some benchmarks, such as BBQ, may measure different capabilities than their stated purpose suggests.