# Interspeech 2026: MSU-Bench
*A diagnostic benchmark for evaluating how well Large Audio-Language Models (LALMs) understand **who says what**, and **what happens between speakers**, in real multi-speaker conversations.*
[](https://aslp-lab.github.io/msu-bench.github.io/)
[](https://arxiv.org/abs/2606.22868)
[](https://huggingface.co/datasets/ASLP-lab/MSU-Benchmark)
[](LICENSE)
[](#license)
**Zhaokai Sun**\*, **Shuai Wang**\*, **Zhennan Lin**\*, Chengyou Wang, Dehui Gao, Yuang Cao, Chunjiang He, Pan Zhou, **Lei Xie**\*\*
Audio, Speech and Language Processing Group (**ASLP@NPU**), School of Software, **Northwestern Polytechnical University**, China
School of Intelligent Science and Technology, **Nanjing University**, China
**Shenzhen Loop Area Institute**, China
Base Model, **Li Auto**, China
\* Equal contribution ยท \*\* Corresponding author ยท ๐ฎ `zksun@mail.nwpu.edu.cn`
---
## ๐ข News
- **2026-07-07** ยท QA construction pipeline open-sourced (see [`bench_generation_pipeline/`](bench_generation_pipeline/)).
- **2026-07-04** ยท Dataset released on ๐ค [`ASLP-lab/MSU-Benchmark`](https://huggingface.co/datasets/ASLP-lab/MSU-Benchmark).
- **2026-06-04** ยท MSU-Bench is accepted at **Interspeech 2026**.
---
## Overview
Spoken Language Understanding is moving from task-specific pipelines toward **Large Audio-Language Models (LALMs)** that generate natural-language responses over speech. Yet existing speech benchmarks still focus on **single-speaker** audio or isolated subtasks (ASR, speaker verification, emotion recognition, โฆ), leaving **speaker-centric understanding in real multi-speaker conversations** insufficiently evaluated โ real conversations involve rapid turn switching, overlaps, interruptions, and speaker-dependent variations in style, emotion, and intent.
> **Who is speaking? ยท Whose viewpoint is this? ยท What is the relationship between speakers? ยท How do their emotions and stances evolve across turns?**
**MSU-Bench** is a diagnostic benchmark built around these questions โ a **5M design** (*multi-tier ยท multi-speaker ยท multilingual ยท multi-scenario ยท multi-task*) organized as a **two-tier** framework of **5 capabilities** and **16 tasks**, with **2,300 human-verified multiple-choice QA instances** sampled from **~731 hours** of Chinese/English telephone, meeting, podcast, and film/TV audio. Diagnostically-designed distractors let us pinpoint *why* a model fails: wrong-speaker attribution, hallucination, or incorrect "unknown" answers.
- ๐ **Demo:** https://aslp-lab.github.io/msu-bench.github.io/
- ๐ **Paper (arXiv):** https://arxiv.org/abs/2606.22868
- ๐ค **Dataset:** https://huggingface.co/datasets/ASLP-lab/MSU-Benchmark
- ๐ป **Code:** https://github.com/ASLP-lab/MSU-Bench