Qwen3-VL-4B-Instruct-LoRA-AX650

Qwen3-VL-4B-Instruct LoRA on AXERA AX650

Ready-to-run package for Qwen/Qwen3-VL-4B-Instruct on AX650 / NPU3. It includes an AX650 aarch64 axllm server, 36 compiled text layers, a fixed-shape image encoder, and two runtime-selectable LoRA adapters. This release supports text chat and single-image requests through the OpenAI-compatible chat API.

Supported Platform and Configuration

• AX650 / AX650N aarch64; the results below were measured on an AX650 / NPU3 board.
• Text prefill limit: 1,536 input tokens, in 128-token chunks. The compiled context is 2,048 positions; the runtime reports max_token_len: 2047. Leave room for the generated response.
• Packaged image profile: 384 × 384. The runtime resizes input images to this fixed profile.
• Maximum server concurrency: one request. Send requests serially, especially when changing adapters.

The image encoder contributes 144 visual soft tokens per image: (384 / 16 / 2)² = 144, using the packaged 16-pixel patch size and spatial merge size of 2. The total input-token count also includes the text prompt and chat-format tokens. Keep the full request, including visual tokens, within the prefill and context limits.

Performance

These are single-request measurements on AX650 / NPU3 with requests sent serially. TTFT means time to first token; the image row includes image preparation and encoding.

The 575- and 995-token prompts used a repeated word list and asked for a one-word response; both returned the requested word. The first request after an adapter change includes adapter loading and rebinding. The image request crosses the 128-token first prefill chunk because its 144 visual soft tokens are added to the prompt.

Startup Runtime Footprint

Measured on AX650 / NPU3 with this package.

The package includes 39 AXModels, embedding weights, both LoRA adapters, and runtime files. CMM use may vary with the runtime environment.

Image Profile and Accuracy

Qwen3-VL-4B-Instruct_vision.axmodel is the image encoder selected by config.json. The package also contains Qwen3-VL-4B-Instruct_vision_u8.axmodel; the startup script does not select that file. The fixed 384 × 384 profile limits fine-detail chart reading.

At this fixed resolution, chart reading is unreliable: this build answered 2 of 24 questions correctly in a ChartQA check. Check image-based answers before relying on them.

LoRA Adapters

Set task_id on each chat request:

The package stores both adapters as BF16 matrix-input payloads. Adapter selection is process-global, so send requests serially when switching tasks.

These examples demonstrate dynamic LoRA loading and switching; their task accuracy may be insufficient for practical use.

Package Layout

.
├── README.md
├── bin/axllm
├── start_axllm.sh
├── axllm.version.json
├── config.json
├── post_config.json
├── qwen3_tokenizer.txt
├── model.embed_tokens.weight.bfloat16.bin
├── qwen3_vl_text_p128_l0_together.axmodel ... qwen3_vl_text_p128_l35_together.axmodel
├── qwen3_vl_text_post.axmodel
├── Qwen3-VL-4B-Instruct_vision.axmodel
├── Qwen3-VL-4B-Instruct_vision_u8.axmodel
├── lora/
│   ├── qwen3-vl-lora-chartqa/   (layer_00.bf16.bin ... layer_35.bf16.bin, manifest)
│   └── qwen3-vl-lora-design/    (layer_00.bf16.bin ... layer_35.bf16.bin, manifest)
├── assets/chartqa_00.png
└── runtime/lib/libax_engine.so

The original Hugging Face weights are not included. The startup script resolves all runtime files relative to this package.

Download and Run

Download the package with the Hugging Face CLI on a network-connected machine:

mkdir -p Qwen3-VL-4B-Instruct-LoRA-AX650
cd Qwen3-VL-4B-Instruct-LoRA-AX650
hf download AXERA-TECH/Qwen3-VL-4B-Instruct-LoRA-AX650 --local-dir .

Transfer this directory to an AX650 board if it was downloaded elsewhere. From the package directory on the board, start the bundled server:

chmod +x ./bin/axllm ./start_axllm.sh
./start_axllm.sh 8000

start_axllm.sh sets the bundled runtime library path and runs bin/axllm serve. In another terminal on the board, check the server:

curl -fsS http://127.0.0.1:8000/health
curl -fsS http://127.0.0.1:8000/v1/models

A healthy server reports "status": "healthy" and "max_concurrency": 1. The model list reported this ID:

AXERA-TECH/Qwen3-VL-4B-Instruct-LoRA-VLM-AX650-P1536-C2048-Chunk128

Use that exact ID in the model field of every request.

Text Request

This request selects the Design adapter:

curl -sS http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"AXERA-TECH/Qwen3-VL-4B-Instruct-LoRA-VLM-AX650-P1536-C2048-Chunk128","task_id":"qwen3-vl-lora-design","messages":[{"role":"user","content":"Name three colors that work well with navy blue. Be brief."}],"max_tokens":64,"temperature":0}'

Example response content (choices[0].message.content): "White, beige, gold".

Image Request

The packaged sample is a bar chart at assets/chartqa_00.png:

From the package directory on the board, send it as a base64 data URL. The base64 -w 0 option is available on the board's GNU coreutils.

IMAGE_DATA=$(base64 -w 0 assets/chartqa_00.png)
curl -sS http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  --data-binary @- <<JSON
{"model":"AXERA-TECH/Qwen3-VL-4B-Instruct-LoRA-VLM-AX650-P1536-C2048-Chunk128","task_id":"qwen3-vl-lora-chartqa","messages":[{"role":"user","content":[{"type":"text","text":"What is the index value of Coffee?"},{"type":"image_url","image_url":{"url":"data:image/png;base64,$IMAGE_DATA"}}]}],"max_tokens":32,"temperature":0}
JSON

For this sample, the package returned "1" in choices[0].message.content, although the chart shows 82.2 for Coffee. This illustrates the image accuracy limitation above.

Conversion References

If you need the original model files or want to rebuild the deployment artifacts, start with:

• Original Hugging Face model: Qwen/Qwen3-VL-4B-Instruct
• AXERA conversion and deployment workflow: Pulsar2 LLM build and ax-llm runtime

Discussion

• GitHub Issues
• QQ group: 139953715