Item Response Theory
AssessTechniques
A psychometrics framework for estimating ability from patterns of test responses.
Why it's here
Placed in Assess: 1 article(s) of evidence from 1 source(s), led by research-stage coverage, with 1 in the last 30 days. Confidence 24%. Low accumulated evidence, so it defaults conservatively pending more signal.
Evidence (1)
- 7Hugging Face Blog·9/1/2026researchBenchMIRT audits what LLM benchmarks measure
Hugging Face highlights BenchMIRT, a new auditing method that uses multidimensional item response theory to analyze LLM benchmark prompts at the item level. Applied to results from 100 models across 16 benchmarks, it found two stable underlying dimensions—safety and general reasoning—and showed that some benchmarks, such as BBQ, may measure different capabilities than their stated purpose suggests.