BenchMIRT: What Do LLM Benchmarks Really Measure?
BenchMIRT is a new auditing method for LLM benchmarks that applies multidimensional Item Response Theory (MIRT) at the individual prompt level to uncover which underlying capabilities—such as safety o…