# Model Training
1. [Installation](#installation)
2. [Video Database Download](#video-database-download)
3. [VLM Training](#vlm-training)

## Installation
Clone the repository and install the required packages.
```bash
git clone https://github.com/TencentARC/DSR_Suite
cd model
conda create -n model python=3.11
conda activate model
pip install -r requirements.txt
```
## Video Database Download
Download the videos and captions of [Koala-36M](https://github.com/KlingTeam/Koala-36M). Download our constructed [DSR-Train](https://huggingface.co/datasets/TencentARC/DSR_Suite-Data) as training QAs (optional).

## VLM Training
While our current model is based on Qwen2.5-VL-7B, you can replace it will any other VLMs. With Qwen2.5-VL-7B as the example, first convert the training QAs into the format required for training:
```bash
cd qwen-vl-finetune
python qa_json_gen.py --qa_path ../../qa_pairs.json
```
where `--qa_path` is the path of json file containing QAs generated by our pipeline. The results will be saved in `train_qas.json`. Modify the training data path `'PATH_TO_QA_JSON'` and `'PATH_TO_VIDEO_ROOT'` in `qwen/data/__init__.py` into the path to `train_qas.json` and the video root.
```
SPATIAL_REASONING = {
"annotation_path": "PATH_TO_QA_JSON",
"data_path": "PATH_TO_VIDEO_ROOT",
}
```

Change `'PATH_TO_Pi3'` in `train.sh` to the checkpoint of π^3 then conduct the training of Qwen2.5-VL-7B integrated with our GSM with the following command:
```bash
sh train.sh
```