Instructions to use hanchaow/QTuneVL1_5-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hanchaow/QTuneVL1_5-3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="hanchaow/QTuneVL1_5-3B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("hanchaow/QTuneVL1_5-3B")
model = AutoModelForImageTextToText.from_pretrained("hanchaow/QTuneVL1_5-3B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use hanchaow/QTuneVL1_5-3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "hanchaow/QTuneVL1_5-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hanchaow/QTuneVL1_5-3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/hanchaow/QTuneVL1_5-3B

SGLang

How to use hanchaow/QTuneVL1_5-3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "hanchaow/QTuneVL1_5-3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hanchaow/QTuneVL1_5-3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "hanchaow/QTuneVL1_5-3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "hanchaow/QTuneVL1_5-3B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use hanchaow/QTuneVL1_5-3B with Docker Model Runner:
```
docker model run hf.co/hanchaow/QTuneVL1_5-3B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

QTuneVL1.5-3B developed by the Reconova AI Lab (Leader: Jia Baozhi; Team members: Wang Hanchao, Chen Mingmu, Lin Bingqi, et al.) && BDAA-Lab

Introduction

We are pleased to introduce QTuneVL1.5-3B, the latest addition to Reconova AI Lab's series of multimodal large language models. Built upon Qwen2.5-VL-3B, the model's capabilities have been further enhanced through RLVR training using the latest GSPO algorithm.

The model is mainly trained on reasoning datasets, but still maintains proficiency in various general tasks, achieving an overall performance superior to the base model.

Architecture:

ViT: QwenViT
Projector: 2-layer MLP
LLM: Qwen2.5-3B

Evaluation

We evaluate on eight benchmarks specified in the OpenCompass leaderboard using VLMEvalKit, including:

MMBench_TEST_EN/CN_V11, MMStar, MMMU_VAL, MathVista_MINI, HallusionBench, AI2D_TEST, OCRBench, MMVet. The results are shown below:

	Avg	MMBench v1.1	MMStar	MMMU	MathVista	HallusionBench	AI2D	OCRBench	MMVet
Qwen2.5-VL-3B	64.8	77.1	55.3	51.2	60.1	48.6	81.5	83.2	61.4
QTuneVL1-3B	66.1(+1.3)	77.3(+0.2)	57.3(+2.0)	53.6(+2.4)	63.7(+3.6)	49.4(+0.8)	81.3	83.8(0.6)	62.5(+1.1)

The reported results are based on our local implementations and may slightly differ from the official ones.

Copyright

We welcome suggestions to help us improve the QTuneVL. For any query, please contact HanChao Wang: wanghanchao@reconova.com. If you find something interesting, please also feel free to share with us through email or open an issue.