# ASD Benchmark ## Overview This repository provides a unified benchmark for anomalous sound detection (ASD) representation evaluation on **DCASE 2020–2025 Task 2**. The benchmark is designed for fair and reproducible comparison of pre-trained audio representations under a common pipeline, while preserving the official evaluation rules of each DCASE year. It covers multiple ASD settings, including conventional unsupervised ASD, domain-shifted ASD, domain-generalization ASD, and first-shot unsupervised ASD. We evaluate representations under two complementary protocols: - **Embedding-based evaluation**: frozen embeddings are directly used for ASD via memory-bank-based nearest-neighbor scoring, measuring the intrinsic discriminability of the representation. - **Adaptation-based evaluation**: a lightweight adaptation module is trained on top of frozen embeddings, and the adapted features are then evaluated with the same ASD pipeline, measuring the transferability and adaptability of the representation. The benchmark follows the **official DCASE splits** and reports **AUC** and **pAUC** for all years, while preserving the **year-aware scoring logic** required by different DCASE editions. ## Dataset Download You could manually download the datasets via the challenge websites ([DCASE 2020 Task 2](https://dcase.community/challenge2020/task-unsupervised-detection-of-anomalous-sounds#download), [DCASE 2021 Task 2](https://dcase.community/challenge2021/task-unsupervised-detection-of-anomalous-sounds#download), [DCASE 2022 Task 2](https://dcase.community/challenge2022/task-unsupervised-anomalous-sound-detection-for-machine-condition-monitoring#download), [DCASE 2023 Task 2](https://dcase.community/challenge2023/task-first-shot-unsupervised-anomalous-sound-detection-for-machine-condition-monitoring#download), [DCASE 2024 Task 2](https://dcase.community/challenge2024/task-first-shot-unsupervised-anomalous-sound-detection-for-machine-condition-monitoring#download), [DCASE 2025 Task 2](https://dcase.community/challenge2025/task-first-shot-unsupervised-anomalous-sound-detection-for-machine-condition-monitoring#download)) After download, please re-arange the datasets to follow the tree structure shown below: ```text DCASE_DATASET_ROOT/ dcase2020_t2/ development/ / train/*.wav test/*.wav evaluation/ eval_data_list_2020_converted.csv / train/*.wav test/*.wav dcase2021_t2/ development/ / train/*.wav source_test/*.wav target_test/*.wav evaluation/ eval_data_list_2021_converted.csv / train/*.wav source_test/*.wav target_test/*.wav dcase2022_t2/ development/ / train/*.wav test/*.wav evaluation/ eval_data_list_2022_converted.csv / train/*.wav test/*.wav dcase2023_t2/ development/ / train/*.wav test/*.wav evaluation/ eval_data_list_2023_converted.csv / train/*.wav test/*.wav dcase2024_t2/ development/ / train/*.wav test/*.wav evaluation/ eval_data_list_2024_converted.csv / train/*.wav test/*.wav dcase2025_t2/ development/ / train/*.wav test/*.wav evaluation/ eval_data_list_2025_converted.csv / train/*.wav test/*.wav ``` Important: - `eval_data_list_202X_converted.csv` should be placed at `ROOT/dcase202X_t2/evaluation/`. You could find `eval_data_list_202X_converted.csv` in the `supplements/eval_lists`. For quick starts, we also provide pre-processed datasets for download. You could download them at [here](https://drive.google.com/drive/folders/1P-h66C_aCi7c2MK-GKBlS5HWpVYlhAow?usp=sharing). ## Run Run from inside this folder: ```bash cd ASD_benchmark ``` ### 1) Without adaptation ```bash ENV_DATASET_ROOT=/path/to/dcase_dataset_root \ python evaluate_dcase.py \ --extractor_path examples/melspec_extractor.py \ --dcase_year all ``` ### 2) With adaptation ```bash ENV_DATASET_ROOT=/path/to/dcase_dataset_root \ python evaluate_dcase.py \ --extractor_path examples/melspec_extractor.py \ --dcase_year all \ --enable_adaptation ``` ## Outputs Outputs are written under this folder: - `test_results//...` - `feature_cache//...` We also provide a script to show all the benchmark results. By simply running: ```bash python result_extractor.py ``` The benchmark results for each year (dev/eval) will be output to the terminal. ## Acknowledgements We sincerely thank the DCASE Task 2 organizers for their open-sourced data. If you use this benchmark, please consider to cite the following dataset papers: ```bibtex @inproceedings{dcase2020, author = "Koizumi, Yuma and Kawaguchi, Yohei and Imoto, Keisuke and Nakamura, Toshiki and Nikaido, Yuki and Tanabe, Ryo and others ", title = "Description and Discussion on DCASE2020 Challenge Task2: Unsupervised Anomalous Sound Detection for Machine Condition Monitoring", booktitle = "Proc. DCASE", year = "2020", pages = "81--85", } @inproceedings{dcase2021, author = "Kawaguchi, Yohei and Imoto, Keisuke and Koizumi, Yuma and Harada, Noboru and Niizumi, Daisuke and Dohi, Kota and others ", title = "Description and Discussion on DCASE 2021 Challenge Task 2: Unsupervised Anomalous Detection for Machine Condition Monitoring Under Domain Shifted Conditions", booktitle = "Proc. DCASE", year = "2021", pages = "186--190", } @inproceedings{dcase2022, author = "Dohi, Kota and Imoto, Keisuke and Harada, Noboru and Niizumi, Daisuke and Koizumi, Yuma and Nishida, Tomoya and others ", title = "Description and Discussion on DCASE 2022 Challenge Task 2: Unsupervised Anomalous Sound Detection for Machine Condition Monitoring Applying Domain Generalization Techniques", booktitle = "Proc. DCASE", year = "2022", } @inproceedings{dcase2023, author = "Dohi, Kota and Imoto, Keisuke and Harada, Noboru and Niizumi, Daisuke and Koizumi, Yuma and Nishida, Tomoya and others ", title = "Description and Discussion on {DCASE} 2023 Challenge Task 2: First-Shot Unsupervised Anomalous Sound Detection for Machine Condition Monitoring", booktitle = "Proc. DCASE", year = "2023", pages = "31--35", } @inproceedings{dcase2024, author = "Nishida, Tomoya and Harada, Noboru and Niizumi, Daisuke and Albertini, Davide and Sannino, Roberto and Pradolini, Simone and others ", title = "Description and Discussion on DCASE 2024 Challenge Task 2: First-Shot Unsupervised Anomalous Sound Detection for Machine Condition Monitoring", booktitle = "Proc. DCASE", year = "2024", pages = "111--115", } @inproceedings{dcase2025, title={Description and Discussion on DCASE 2025 Challenge Task 2: First-shot Unsupervised Anomalous Sound Detection for Machine Condition Monitoring}, author={Nishida, Tomoya and Harada, Noboru and Niizumi, Daisuke and Albertini, Davide and Sannino, Roberto and Pradolini, Simone and others }, booktitle={Proc. DCASE}, year={2025} } ``` If you find this benchmark useful, please consider to cite our work: ```bibtex @inproceedings{ECHO, title={ECHO: Frequency-aware Hierarchical Encoding for Variable-length Signals}, author={Zhang, Yucong and Liu, Juan and Li, Ming}, booktitle={Proc. ICASSP}, year={2026} } ```