# Qwen/Qwen2-72B-Instruct-GPTQ-Int4 Cog Model

This is an implementation of [Qwen/Qwen2-72B-Instruct-GPTQ-Int4](https://huggingface.co/Qwen/Qwen2-72B-Instruct-GPTQ-Int4) as a [Cog](https://github.com/replicate/cog) model.

## Development

This is a fork of [replicate/cog-vllm](https://github.com/replicate/cog-vllm).

Follow the [model pushing guide](https://replicate.com/docs/guides/push-a-model) to push your own model to [Replicate](https://replicate.com).

## Basic Usage

To run a prediction:

    cog predict -i prompt="Give me a short introduction to large language model."