# BeTraC — Rules & Approved Resources (v2 DRAFT) > 💡 **Missing a dataset or model?** Additional datasets or models may be proposed for inclusion by **May 4, 2026** — contact **[betrac@googlegroups.com](mailto:betrac@googlegroups.com)** (see §7). > ⚠️ **Unsure about anything?** If you are unsure whether a model, dataset, architecture, or technique is permitted, contact the organizers at **[betrac@googlegroups.com](mailto:betrac@googlegroups.com)** before the submission deadline. Rulings will be shared publicly on the challenge website. To encourage exploration of end-to-end optimizable architectures, no textual representation of the audio's spoken content — whether transcript, paraphrase, summary, or generated clinical note — may be passed between separate models or pipeline components during inference (multi-task, distillation etc. during training is allowed, with the chain-of-thought exception below). Only the final model in the pipeline may produce text derived from the audio. The final model's chain-of-thought is unconstrained in form, but the model must not have been trained to produce a transcript of the input audio as a CoT target or intermediate supervision signal (see §2). ## 1. Tracks | | Lightweight (≤ 6B) | Heavyweight (≤ 36B) | | -------------------- | -------------------- | ------------------- | | **Architecture** | Single end-to-end model | Tools / agents allowed | | **Open-weight only** | Yes | Yes | **Lightweight:** A single model (within the track parameter limit) receives audio and outputs the SOAP note. The model must be invoked **exactly once** per audio file. No external tools, no multi-model pipelines, and no separately executed preprocessing models. Learned components such as audio encoders, diarization modules, or VAD may be **integrated as submodules within the single model architecture** (their parameters count toward the track budget), but may not be run as separate inference steps whose outputs are fed into the main model. Standard signal processing such as resampling, codec decoding, and normalization is permitted. **Heavyweight:** Multi-model pipelines and tool use are allowed, but all learned components that participate in a forward pass — including tools — are summed toward the track parameter limit. Unused sub-modules may be stripped (see §8 for counting rules). Proprietary API-based models are not permitted in either track. ## 2. Allowed - **ASR-pretrained encoders** as feature extractors (Whisper encoder, HuBERT, WavLM, etc.). - **Chain-of-thought** inside the final model: unconstrained in form. Short verbatim quotes from the audio that arise from task-relevant reasoning are permitted, and emergent transcript-like content in CoT is not a violation. **However**, the model must not have been trained with a transcript of the input audio as a CoT target, prefix, or intermediate supervision signal. This includes rich or structured representations that encode the spoken word sequence (e.g., word-level timestamps with speaker, prosody, or affect tags used as a reasoning step), and distillation from a teacher whose CoT is transcript-aligned. The distinction is in how the system was trained, not in what the CoT tokens look like at inference. See §9 for verification. - **Data augmentation** of any kind, using allowed data only. Any model used to generate synthetic training data (text or audio) must meet the eligibility criteria in §8. - **Acoustic preprocessing tools (Heavyweight only as separate models):** diarization, VAD, speaker embeddings, segmentation. Anything outputting timestamps, speaker IDs, or labels (no linguistic content). When run as separate inference steps, these are heavyweight only and their parameters count toward the budget. In the lightweight track, such components are permitted only if integrated as submodules within the single model architecture (see §1). - **Knowledge retrieval (Heavyweight):** e.g. retrieval augmented generation (RAG) over medical ontologies, drug DBs, guidelines, PubMed etc. is allowed as it returns external knowledge. - **Deterministic post-processing:** Regex, string formatting, section header insertion, and other non-learned transformations applied to the model's output. These must be implemented in the extraction script (see §4). ## 3. Not Allowed - **Intermediate text derived from audio** passed between separate models/components (transcripts, paraphrases, SOAP drafts, keyword extraction). - **Post-processing LLMs** that rewrite a SOAP draft from the main model. - **Models that do not meet the eligibility criteria** (see §8). - **Synthetic training data generated by ineligible models:** Any model used to generate synthetic training data (text or audio) — including SOAP note generation, TTS audio synthesis, and distillation targets — must meet the eligibility criteria in §8. Using proprietary or closed-source systems (e.g., GPT-4o for SOAP notes, closed TTS APIs) is not permitted. - **Test label use** at any stage. - **Cross-file context** — each audio file processed independently. - **Tools returning linguistic content** from audio (transcripts, summaries, extracted entities). - **Output-level ensembling:** Generating multiple SOAP notes and selecting or merging them (majority voting, reranking, fusion) is not permitted. This includes self-consistency prompting (sampling multiple outputs and voting). - **Multiple model invocations per audio file** in the lightweight track. ## 4. Submissions Each submission must include **all three** of the following: 1. **Raw model output:** The complete, verbatim output for every test audio file — every token generated, including chain-of-thought, tags, delimiters, and any other content. For heavyweight systems with multiple model calls, the full trace of every call must be included, covering all tool inputs and tool outputs. 2. **Extraction script:** A Python 3.11 script that extracts the final SOAP note from the raw output. Dependencies must be managed via `uv`, with a `Makefile` to set up the environment and run the extraction on the raw submission. Teams may use a regular expression or more complex logic, but the script must be deterministic (no learned components). The organizers will validate submissions by running this script. 3. **Extracted SOAP notes:** One plain-text SOAP note per audio file (sections: S, O, A, P) in the scoring format (to be specified). **Heavyweight submissions** must additionally include a **model inventory table** listing every model used in the system, its parameter count, its role (e.g., "main audio-LLM," "diarization," "speaker embedding"), and a sum confirming compliance with the track budget. - System description paper due **July 8, 2026** (required for ranking). - **One submission per team per track.** Teams receive test audio after submitting a system description by the deadline. - Non-compliant systems may be submitted but will be excluded from rankings. ## 5. Tools (Heavyweight Track) A **tool** is any external component invoked by the model during inference that performs a specific function. This includes: - **Learned tools:** Separately trained models such as diarization systems, speaker identification models, retrieval encoders, or medical NER models. All learned tools' parameters are summed toward the track budget. - **Knowledge lookups:** Queries against medical ontologies, drug databases, clinical guidelines, or PubMed. These do not count toward the parameter budget, but must not return linguistic content derived from the conversation audio. - **Code execution:** Running code (e.g., for structured data manipulation, formatting, or computation). This does not count toward parameters. **Constraints on tools:** - No tool may take conversation audio as input and return linguistic content (text derived from speech). Tools may take audio as input only if they return non-linguistic output (timestamps, speaker IDs, acoustic features, etc.). - All tool calls and their inputs/outputs must appear in the submission trace (see §4). - Deterministic code (regex, format conversion, lookup tables) is always allowed and does not count toward parameters, but must be included in the extraction script or documented in the system description. ## 6. Evaluation - **Primary:** Open Medical Concept F1 (MeSH + scispaCy NER) - **Secondary:** ROUGE F1 (R-2, R-3, R-L) - **Post-competition (top 5/track):** LLM-as-a-Judge + OOD generalization on real OSCE interviews ## 7. Proposals Additional datasets or models may be proposed for inclusion by **May 4, 2026**. Contact **[betrac@googlegroups.com](mailto:betrac@googlegroups.com)**. ## 8. Model Eligibility Any model is eligible provided it meets **all four** of the following criteria: 1. **Publicly available weights** — the checkpoint must be downloadable without a gated access request or commercial agreement at the time of submission. 2. **Reproducible checkpoint** — the exact weights used must remain accessible (e.g., a pinned HuggingFace revision or a DOI-backed release) through at least the post-competition evaluation phase. 3. **Published model card or technical report** — a publicly available document describing architecture, training data, and intended use. 4. **Parameter budget** — forward-pass parameter count (see counting rules below) fits within the track limit. **Parameter counting rules:** The budget counts all learned parameters that participate in at least one forward pass during inference. Teams may **strip** sub-modules that are not used for the task (e.g., a vision encoder or speech synthesis decoder in an audio-only text-output challenge). Stripped modules must not be loaded into memory during inference, and the system description paper must document what was removed. - **MoE models:** All expert parameters count toward the budget (total, not active-per-token), since every expert may be routed to during a forward pass. - **Omni-models:** Only sub-modules that participate in inference count. For example, if a model ships with a vision encoder, a talker, and a codec decoder that are unused, those may be stripped and excluded from the count. - **Named sizes** (e.g., "7B") typically refer to the LLM core only and should not be used as the budget figure without verification. - **LoRA** and other low-rank adaptation methods whose parameters are merged into existing weight matrices at inference time do not add to the count. However, any **new modules** added on top of a base model (projection layers, additional encoder heads, cross-attention layers, etc.) do count. - **Heavyweight pipelines:** All learned components across the entire pipeline — including tool models — that participate in a forward pass are summed toward the budget. Unused sub-modules in tool models may also be stripped. **Ineligible models** include: any model requiring a proprietary license, a non-disclosure agreement, or API-only access; models whose weights are removed or made inaccessible before the competition concludes; and models with no public documentation of their training data or architecture. ## 9. Compliance Review The organizers will review all submission traces (raw model outputs and tool logs) for compliance with the no-transcription rule and parameter limits. System description papers must include sufficient detail to reproduce the inference pipeline. The organizers reserve the right to request inference code from any team. **CoT training attestation.** System descriptions must attest that no transcript of the input audio (verbatim, paraphrased, or in rich/structured form encoding the spoken word sequence) was used as a CoT target, prefix, or intermediate supervision signal during training, and that the system was not distilled from a teacher whose CoT is transcript-aligned. Where submission traces show sustained transcript-like patterns in CoT, organizers may request training code, training data manifests, or loss configurations to verify compliance. Submissions found to be non-compliant will be excluded from rankings. If non-compliance is discovered after results are announced, rankings will be revised. --- ## Eligible Models The models listed below are approved for use, provided they meet the criteria in §8. Additional models may be proposed for inclusion by **May 4, 2026** — contact **[betrac@googlegroups.com](mailto:betrac@googlegroups.com)** (see §7). **All eligible models must use versions uploaded no later than May 12, 2026.** > ⚠️ **Parameter counting reminder:** The budget counts all parameters that participate in a forward pass. Unused sub-modules (e.g., vision encoders, speech decoders) may be stripped — see §8. For MoE models, all expert parameters count (total, not active-per-token). Named sizes (e.g., "7B") typically refer to the LLM core only — always verify against the model card. ### Audio-Language / Omni | Model | Total params | Notes | Links | | ------------------ | -------------------------------- | ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | | Qwen2.5-Omni-3B | ~5–6B (est.) | Dense | [HF](https://huggingface.co/Qwen/Qwen2.5-Omni-3B) · [arXiv](https://arxiv.org/abs/2503.20215) | | Qwen2.5-Omni-7B | **10.73B** ⚠️ | Dense; "7B" refers to LLM core only | [HF](https://huggingface.co/Qwen/Qwen2.5-Omni-7B) · [arXiv](https://arxiv.org/abs/2503.20215) | | Qwen2-Audio-7B | **8.2B** | Dense | [HF](https://huggingface.co/Qwen/Qwen2-Audio-7B) · [arXiv](https://arxiv.org/abs/2407.10759) | | Qwen3-Omni-30B-A3B | **~35B total** / ~31B forward-pass | MoE; ~31B after stripping unused vision/talker/codec | [HF](https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct) · [arXiv](https://arxiv.org/abs/2509.17765) | | SALMONN-7B / 13B | ~8B / ~14B (est.) | Whisper-L + BEATs + Vicuna backbone | [HF](https://huggingface.co/tsinghua-ee/SALMONN-7B) · [arXiv](https://arxiv.org/abs/2310.13289) | | LLaMA-Omni | ~9.1B (est.) | Llama-3.1-8B backbone | [HF](https://huggingface.co/ICTNLP/Llama-3.1-8B-Omni) · [arXiv](https://arxiv.org/abs/2409.06666) | | Gemma-3n-E2B | 6B raw / **~2B effective** ⚠️ | PLE offloading; confirm counting rule with organizers | [HF](https://huggingface.co/google/gemma-3n-E2B-it) · [Docs](https://ai.google.dev/gemma/docs/gemma-3n) | | Phi-4-multimodal-instruct | **~5.6B** | Dense | [HF](https://huggingface.co/microsoft/Phi-4-multimodal-instruct) · [arXiv](https://arxiv.org/abs/2503.01743) | | Family | HuggingFace / Download | Paper / Doc | | ----------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | Voxtral | [Voxtral](https://huggingface.co/collections/mistralai/voxtral) | [arxiv main](https://arxiv.org/abs/2507.13264) · [Docs](https://mistral.ai/news/voxtral)| | MOSS-Audio | [MossAudio](https://huggingface.co/collections/OpenMOSS-Team/moss-audio) | [Docs](https://github.com/OpenMOSS/MOSS-Audio) | | Gemma 4 | [Gemma 4](https://huggingface.co/collections/google/gemma-4) | [Docs](https://ai.google.dev/gemma/docs/core/model_card_4) | | Omnilingual ASR | [Omnilingual ASR](https://github.com/facebookresearch/omnilingual-asr) | [Docs](https://ai.meta.com/research/publications/omnilingual-asr-open-source-multilingual-speech-recognition-for-1600-languages/) | | Granite Speech | [Granite Speech](https://huggingface.co/collections/ibm-granite/granite-speech) | [arxiv](https://arxiv.org/abs/2505.08699) | ### Self-Supervised Audio Encoders | Model | Params | Links | | ------------------------------------ | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | Whisper (tiny → large-v3) | 39M – 1,550M | [HF](https://huggingface.co/openai/whisper-large-v3) · [GitHub](https://github.com/openai/whisper) · [arXiv](https://arxiv.org/abs/2212.04356) | | Whisper large-v3-turbo | 809M | [HF](https://huggingface.co/openai/whisper-large-v3-turbo) | | OWSM v3 / v3.1 (ESPnet) | 889M / 101M–1.02B | [HF](https://huggingface.co/espnet/owsm_v3) · [Project](https://www.wavlab.org/activities/2024/owsm/) · [arXiv](https://arxiv.org/abs/2401.16658) | | wav2vec 2.0 (base / large / XLSR-53) | 95M / 317M / 317M | [HF](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) · [arXiv](https://arxiv.org/abs/2006.11477) | | Wav2Vec-BERT 2.0 (Seamless) | ~580M | [HF](https://huggingface.co/facebook/w2v-bert-2.0) · [arXiv](https://arxiv.org/abs/2312.05187) | | HuBERT (base / large / xlarge) | 95M / 316M / 964M | [HF](https://huggingface.co/facebook/hubert-large-ll60k) · [arXiv](https://arxiv.org/abs/2106.07447) | | WavLM (base / base+ / large) | 94.7M / 94.7M / 316.6M | [HF](https://huggingface.co/microsoft/wavlm-large) · [arXiv](https://arxiv.org/abs/2110.13900) | | BEATs | ~90M | [GitHub](https://github.com/microsoft/unilm/tree/master/beats) · [arXiv](https://arxiv.org/abs/2212.09058) | ### ASR Models (encoder features only — no text decoding at inference) | Model | Params | Links | | --------------------------------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- | | Whisper (tiny → large-v3-turbo) | 39M – 1,550M | [HF](https://huggingface.co/openai/whisper-large-v3) · [GitHub](https://github.com/openai/whisper) · [arXiv](https://arxiv.org/abs/2212.04356) | | NeMo Parakeet family (CTC / RNNT / TDT) | 600M – 1.1B | [HF collection](https://huggingface.co/collections/nvidia/parakeet) · [arXiv](https://arxiv.org/abs/2509.14128) | | NeMo Canary-1B / Flash / v2 | 883M – 1B | [HF](https://huggingface.co/nvidia/canary-1b) · [arXiv](https://arxiv.org/abs/2406.19674) | | ESPnet OWSM-CTC v3.1 | ~1B | [HF](https://huggingface.co/espnet/owsm_ctc_v3.1_1B) · [arXiv](https://arxiv.org/abs/2402.12654) | ### Diarization / VAD / Audio Processing | Model | Params | Type | Links | | ------------------------------------------------------ | --------------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | | Pyannote Audio (segmentation-3.0 / diarization-3.1) | ~1.2M | Segmentation + full pipeline | [HF](https://huggingface.co/pyannote/speaker-diarization-3.1) · [GitHub](https://github.com/pyannote/pyannote-audio) | | NeMo (MarbleNet / MSDD / TitaNet / SpeakerNet / ECAPA) | 91.5K – 23M | VAD, diarization, speaker embeddings | [NGC](https://catalog.ngc.nvidia.com/orgs/nvidia/teams/nemo) · [GitHub](https://github.com/NVIDIA-NeMo/NeMo) | | SpeechBrain ECAPA-TDNN | ~14.7M | Speaker embeddings | [HF](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb) · [GitHub](https://github.com/speechbrain/speechbrain) | | Silero VAD | ~260K | VAD | [GitHub](https://github.com/snakers4/silero-vad) | | Brouhaha | ~1–2M | VAD + SNR + C50 | [GitHub](https://github.com/marianne-m/brouhaha-vad) · [arXiv](https://arxiv.org/abs/2210.13248) | | X-vector / VBx (Kaldi ResNet101) | ~4.2M / ~40–45M | Speaker embeddings + clustering | [GitHub](https://github.com/BUTSpeechFIT/VBx) · [arXiv](https://arxiv.org/abs/2012.14952) | | DiariZen-Large / Large-s80 | ~320M+ / 63.3M | EEND-VC diarization | [GitHub](https://github.com/BUTSpeechFIT/DiariZen) · [arXiv](https://arxiv.org/abs/2409.09408) | > ℹ️ Diarization, VAD, and speaker embedding models used as **separate inference steps** are heavyweight track only and their forward-pass parameters count toward the track budget. In the lightweight track, such functionality may only appear as integrated (end-to-end optimizable) submodules within the single model. ### TTS | Model | Params | Type | Links | | ------------------------------------------------------ | --------------- | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------- | | Qwen3-TTS-12Hz-1.7B-Base | 1.7B | TTS | [HF](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-1.7B-Base) · [arXiv](https://arxiv.org/abs/2601.15621) | | Kokoro-82M | 82M | TTS | [HF](https://huggingface.co/hexgrad/Kokoro-82M) > ℹ️ TTS models used as an **augmentation step**, independent of inference, do not count towards the parameter budget. If they are used as a tool during inference, they are heavyweight track only and their forward-pass parameters count toward the track budget. In the lightweight track, such functionality may only appear as integrated (end-to-end optimizable) submodules within the single model. ### Text LLMs Approved LLM families (see §7 to propose others): | Family | HuggingFace | Paper | | ----------------- | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | | Qwen2.5 / Qwen3 / Qwen3.5 (except Omni) | [Qwen org](https://huggingface.co/Qwen) | [arXiv:2412.15115](https://arxiv.org/abs/2412.15115) · [arXiv:2505.09388](https://arxiv.org/abs/2505.09388) | | Llama 2 / 3.x | [meta-llama](https://huggingface.co/meta-llama) | [arXiv:2307.09288](https://arxiv.org/abs/2307.09288) · [arXiv:2407.21783](https://arxiv.org/abs/2407.21783) | | DeepSeek V2/V3/V4/R1 | [deepseek-ai](https://huggingface.co/deepseek-ai) | [arXiv:2412.19437](https://arxiv.org/abs/2412.19437) · [arXiv:2501.12948](https://arxiv.org/abs/2501.12948) | | Phi-3 / 3.5 / 4 | [microsoft](https://huggingface.co/microsoft) | [arXiv:2404.14219](https://arxiv.org/abs/2404.14219) · [arXiv:2412.08905](https://arxiv.org/abs/2412.08905) | | Gemma 2 / 3 | [google](https://huggingface.co/google) | [arXiv:2408.00118](https://arxiv.org/abs/2408.00118) · [Report](https://goo.gle/Gemma3Report) | | Mistral / Mixtral | [mistralai](https://huggingface.co/mistralai) | [arXiv:2310.06825](https://arxiv.org/abs/2310.06825) · [arXiv:2401.04088](https://arxiv.org/abs/2401.04088) | | OLMo | [allenai](https://huggingface.co/allenai) | [arXiv:2501.00656](https://arxiv.org/abs/2501.00656) | | TinyLlama | [TinyLlama](https://huggingface.co/TinyLlama) | [arXiv:2401.02385](https://arxiv.org/abs/2401.02385) | Approved LLMs: | Model | Total params | Notes | Links | | ------------------ | -------------------------------- | ----------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | | Llama3-Med42-8B | ~8B | Fine-tuned Llama3-8B | [HF](https://huggingface.co/m42-health/Llama3-Med42-8B) · [arXiv](https://arxiv.org/html/2408.06142v1) | | Llama3-OpenBioLLM-8B| ~8B | Fine-tuned Llama3-8B | [HF](https://huggingface.co/aaditya/Llama3-OpenBioLLM-8B) · [arXiv](https://arxiv.org/abs/2408.13833) | | II-Medical-8B | ~8B | Fine-tuned Qwen3-8B | [HF](https://huggingface.co/Intelligent-Internet/II-Medical-8B) · [Doc](https://ii.inc/web/blog/post/ii-medical)| | meditron-7b | ~7B | Fine-tuned Llama2-7B | [HF](https://huggingface.co/epfl-llm/meditron-7b) · [arXiv](https://arxiv.org/abs/2311.16079) | | BioMistral-7B | ~7B | Fine-tuned Mistral 7B | [HF](https://huggingface.co/BioMistral/BioMistral-7B) · [arXiv](https://arxiv.org/abs/2402.10373) | | omi-health/sum-small| ~4B | Fine-tuned Phi-3-mini-3k-instruct; Dataset Availible | [HF](https://huggingface.co/omi-health/sum-small) · [Docs](https://omi.health/research/omi-sum) | --- ## Approved External Datasets ### Clinical / Medical | Dataset | Size | Content | Links | | ---------------------------- | ----------- | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Synth-DoPaCo** (provided) | 8,800 conv. | Synthetic doctor-patient, audio + SOAP | Provided by organizers | | **PubMed / PMC Open Access** | — | Biomedical literature (RAG) | [PubMed](https://pubmed.ncbi.nlm.nih.gov/) · [PMC FTP](https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_bulk/) | | **ACI-Bench** | 207 conv. | Doctor-patient dialogue + clinical visit notes | [GitHub](https://github.com/wyim/aci-bench) · [Figshare](https://figshare.com/articles/dataset/aci-bench-corpus_zip/22494601) · [Paper](https://www.nature.com/articles/s41597-023-02487-3) | | **MTS-Dialog** | 1,701 conv. | Doctor-patient dialogue + clinical note sections (20 section types) | [GitHub](https://github.com/abachaa/MTS-Dialog) · [Paper](https://aclanthology.org/2023.eacl-main.168) | | **PriMock57** | 57 conv. | Mock primary care consultations, audio + transcripts + clinician notes | [GitHub](https://github.com/babylonhealth/primock57) · [arXiv](https://arxiv.org/abs/2204.00333) | | **medical-dialogue-to-soap-summary **| 10,000 conv. | Synthetic doctor-patient, SOAP | [GitHub](https://huggingface.co/datasets/omi-health/medical-dialogue-to-soap-summary) · [arXiv](https://arxiv.org/abs/2310.15959v2) · [Docs](https://omi.health/research/omi-sum) | ### Speech / Audio | Dataset | Size | Links | Paper | | -------------------------- | ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | | **LibriSpeech** | 960h | [OpenSLR/12](https://www.openslr.org/12/) · [HF](https://huggingface.co/datasets/openslr/librispeech_asr) | Panayotov et al., ICASSP 2015 | | **Scalable SpontaneousSpeech Dataset** | 727h | [cmu.flintbox.com](https://cmu.flintbox.com/technologies/b435b5ba-62a0-442d-857a-bced6070f621) | [isca-archive.org](https://www.isca-archive.org/interspeech_2025/sheikh25_interspeech.pdf) | | **CANDOR** | 850h | [https://convokit.cornell.edu/documentation/candor.html](https://convokit.cornell.edu/documentation/candor.html) | [https://www.science.org/doi/10.1126/sciadv.adf3197](https://www.science.org/doi/10.1126/sciadv.adf3197) | | **CASPER** | 200h | [https://huggingface.co/datasets/CASPER-SSSD/CASPER](https://huggingface.co/datasets/CASPER-SSSD/CASPER) | [https://arxiv.org/html/2506.00267v1](https://arxiv.org/html/2506.00267v1) | | **GigaSpeech** | 10,000h | [GitHub](https://github.com/SpeechColab/GigaSpeech) · [HF](https://huggingface.co/datasets/speechcolab/gigaspeech) | [arXiv:2106.06909](https://arxiv.org/abs/2106.06909) | | **Common Voice** | 30,000h+ | [Mozilla](https://commonvoice.mozilla.org/en/datasets) · [HF](https://huggingface.co/datasets/mozilla-foundation/common_voice_13_0) | [ACL 2020](https://aclanthology.org/2020.lrec-1.520/) | | **SPGISpeech** | 5,000h | [Kensho](https://datasets.kensho.com/datasets/spgispeech) · [HF](https://huggingface.co/datasets/kensho/spgispeech) | [arXiv:2104.02014](https://arxiv.org/abs/2104.02014) | | **VoxCeleb 1** | 1,251 speakers | [Oxford VGG](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/vox1.html) | [arXiv:1706.08612](https://arxiv.org/abs/1706.08612) | | **VoxCeleb 2** | 6,112 speakers | [Oxford VGG](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/vox2.html) | [arXiv:1806.05622](https://arxiv.org/abs/1806.05622) | | **MUSAN** | Music, speech, noise | [OpenSLR/17](https://www.openslr.org/17/) | [arXiv:1510.08484](https://arxiv.org/abs/1510.08484) | | **LibriCSS** | 10h (far-field multi-speaker) | [GitHub](https://github.com/chenzhuo1011/libri_css) | [arXiv:2001.11482](https://arxiv.org/abs/2001.11482) | | **AMI** | 100h meetings | [Edinburgh](https://groups.inf.ed.ac.uk/ami/corpus/) · [HF](https://huggingface.co/datasets/edinburghcstr/ami) | Carletta, LRE 2007 | | **DNS Challenge noises** | — | [GitHub](https://github.com/microsoft/DNS-Challenge) | [arXiv:2005.13981](https://arxiv.org/abs/2005.13981) | | **REVERB Challenge RIRs** | — | [reverb2014.dereverberation.com](http://reverb2014.dereverberation.com/) | Kinoshita et al., WASPAA 2013 | | **Aachen AIR** | 215 RIRs | [RWTH Aachen](https://www.iks.rwth-aachen.de/en/research/tools-downloads/databases/aachen-impulse-response-database/) · [OpenSLR/20](https://www.openslr.org/20/) | Jeub et al., DSP 2009 | | **BUT ReverbDB** | 1,300+ RIRs | [BUT Speech@FIT](https://speech.fit.vut.cz/software/but-speech-fit-reverb-database) | [arXiv:1811.06795](https://arxiv.org/abs/1811.06795) | | **SLR28 RIR and Noise DB** | — | [OpenSLR/28](https://www.openslr.org/28/) | Ko et al., ICASSP 2017 | | **RWCP Sound Scene DB** | — | [NII Japan](http://research.nii.ac.jp/src/en/RWCP-SSD.html) · [OpenSLR/13](https://www.openslr.org/13/) | Nakamura et al., 2002 | | **FSD50k** | 51k clips | [Zenodo](https://zenodo.org/records/4060432) · [HF](https://huggingface.co/datasets/Fhrozen/FSD50k) | [arXiv:2010.00475](https://arxiv.org/abs/2010.00475) | | **WSJ / WSJ0 / WSJ1** | ~80h / 15h / 80h | [LDC WSJ0](https://catalog.ldc.upenn.edu/LDC93S6A) · [LDC WSJ1](https://catalog.ldc.upenn.edu/LDC94S13A) | Paul & Baker, DARPA SLS 1992 | | **WSJ0-2mix** | Generated (requires WSJ0 from LDC) | [Script](https://github.com/mpariente/pyscaffold) | [arXiv:1508.04306](https://arxiv.org/abs/1508.04306) | | **WHAM!** | — | [wham.whisper.ai](http://wham.whisper.ai/) | [arXiv:1907.01160](https://arxiv.org/abs/1907.01160) |