HumDial-EIBench: A Human-Recorded Multi-Turn Emotional Intelligence Benchmark for Audio Language Models

HumDial-EIBench addresses a significant gap in the evaluation of Audio Language Models (ALMs) like GPT-4o or Gemini Live: the transition from synthetic, single-turn emotional detection to real-world, multi-turn human interaction. With 8 forks in just 24 hours, the project shows immediate engagement from the research community, likely tied to the ICASSP 2026 challenge. Its defensibility stems from 'data gravity'—the use of human-recorded dialogues is significantly more valuable than synthesized speech, which frontier labs often rely on for scale but which lacks nuance. While frontier labs are building the models this benchmark evaluates, they generally prefer third-party benchmarks for objective validation, lowering the risk of direct platform competition. However, its longevity depends on whether it can become the 'MMLU for Audio EI'; otherwise, it faces displacement in 1-2 years as larger, more diverse datasets are inevitably released by well-funded academic-industry partnerships. Compared to existing benchmarks like IEMOCAP or MELD, this project’s focus on multi-turn causal reasoning (why an emotion changed) provides a much-needed depth that standard classification tasks lack.