**Do Audio LLMs Listen or Read? Analyzing and Mitigating Paralinguistic Failures with VoxParadox**
Jiacheng Pang
\*, Ashutosh Chaubey
\*, Mohammad Soleymani \
University of Southern California \
* Equal contribution
[](https://icml.cc/Conferences/2026)
[](https://arxiv.org/abs/2605.27772)
[](https://voxparadox.github.io/)
[](https://huggingface.co/datasets/IHP-Lab/VoxParadox)
[](https://huggingface.co/IHP-Lab/AF3_PCLM_DPO)
[](https://huggingface.co/IHP-Lab/Qwen2-Audio_PCLM_DPO)
[](LICENSE)
[](#)
[](https://www.python.org)
---
Official **v1.0.0** code release for our **ICML 2026** paper. We introduce
**VoxParadox**, an adversarial benchmark that isolates the paralinguistic
understanding of Audio LLMs under controlled linguisticβacoustic contradiction,
plus two methods for closing the gap:
* **PCLM** (Prompt-Conditioned Layer Mixer) β exposes intermediate audio-encoder
layers to the LLM via a prompt-conditioned mixture.
* **DPO** (Direct Preference Optimization) β biases the LLM toward acoustically
grounded answers over transcript-implied alternatives.
## Layout
```
.
βββ af3/audio-flamingo/ # AF3 with PCLM β inference, eval, probing
βββ qwen2audio/ # Qwen2-Audio with PCLM β inference, eval, probing
βββ clap/ # CLAP encoder probing
βββ hubert/ # HuBERT encoder probing
βββ assets/ # logo, figures
βββ LICENSE # USC Research License
```
## Installation
Each model has its own environment β the AF3 and Qwen2-Audio toolchains pull
incompatible versions of `transformers` and `deepspeed`, so don't mix them.
> **We recommend starting with Qwen2-Audio.**
```bash
# Qwen2-Audio (recommended starting point)
conda create -n qwen2audio python=3.10 -y && conda activate qwen2audio
pip install torch torchaudio transformers accelerate librosa soundfile
# Audio Flamingo 3
cd af3/audio-flamingo
bash environment_setup.sh af3
```
Match the torch/CUDA build to your hardware. See each model's README for
version pins and probing-specific extras.
## Quick start
### 1. Get the dataset
```python
from datasets import load_dataset
ds = load_dataset("IHP-Lab/VoxParadox", split="test")
```
Or download the directory (`voxparadox.json` + `audio/`) directly from the
[dataset page](https://huggingface.co/datasets/IHP-Lab/VoxParadox).
### 2. Run inference
Both models ship an entry point that writes a single `predictions.jsonl`
(`{"id": ..., "response": ...}` per line). **Start with Qwen2-Audio** β it's
the fastest path to a working round-trip.
```bash
# Qwen2-Audio
python -m qwen2audio.eval.run_eval \
--model_path