# CH-SIMS v2.0

Official codes for paper "Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module" (ICMI 2022)

- [Dataset HomePage](https://thuiar.github.io/sims.github.io/chsims)

- [ICMI 2022 Paper](https://arxiv.org/abs/2209.02604)

## 1. Illustration of CH-SIMS v2.0 Data
<p align="center">
  <img width="800" src="show/CH-SIMSv2.0.png">
</p>

## 2. Data Download

1. CH-SIMS v2(s) - Supervised data:
  - [Google Drive](https://drive.google.com/drive/folders/1wFvGS0ebKRvT3q6Xolot-sDtCNfz7HRA?usp=sharing)
  - [Baiduyun Drive](https://pan.baidu.com/s/13Ds2_XDIGUqMHt4lXNLQSQ) Code: icmi

2. CH-SIMS v2(u) - Unsupervised data:
  - [Google Drive](https://drive.google.com/drive/folders/1llIbm3gwyJRwwk58RUQHWBNKjHI9vGGB?usp=sharing)
  - [Baiduyun Drive](https://pan.baidu.com/s/1tezEDR3Y23hJ6Mp5fmcp-w) Code: icmi 

## 3. Baselines Results
<p align="center">
  <img width="800" src="show/ModelResults.png">
</p>

## 4. Run Experiments

1. Download dataset and set correct path in:

```text
config/config.py --> line 32  --> "root_dataset_dir"
```

2. If you want to run the AV-MC framework: 

```shell
python run.py --is_tune Flase --modelName v1
```

3. If you want to run the AV-MC(Semi) framework 

```shell
python run.py --is_tune Flase --modelName v1_semi
```

## 5. Citation

If you find this paper or dataset useful, please cite us at: 

```bib
@misc{liu2022make,
      title={Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module}, 
      author={Yihe Liu and Ziqi Yuan and Huisheng Mao and Zhiyun Liang and Wanqiuyue Yang and Yuanzhe Qiu and Tie Cheng and Xiaoteng Li and Hua Xu and Kai Gao},
      year={2022},
      eprint={2209.02604},
      archivePrefix={arXiv},
      primaryClass={cs.MM}
}
```

## 6. Contact Us

For any questions, please email [Yihe Liu](mailto:512796310@qq.com) or [Ziqi Yuan](mailto:yzq21@mails.tsinghua.edu.cn)