Qwen3.5-KETI-HAECHI-27B

Qwen3.5-KETI-HAECHI-27B is a multimodal model derived from Qwen/Qwen3.5-27B. It was developed for two primary purposes: improving Korean cultural-heritage understanding and Korean OCR, and improving tool calling and multi-step, stateful agent execution. Alongside these goals, the model preserves broad multimodal, language, and coding capabilities from the base model.

Core capability profile

• Korean cultural heritage and OCR: identifies the official names of heritage objects, answers questions grounded in heritage images, and reads Korean text from signs, scenes, rendered text, and public documents.
• Tool calling and long-horizon task execution: selects and calls tools, carries information across multiple turns, tracks changing state, and works toward an end-to-end goal over several steps.
• General multimodal understanding: interprets images and text together, follows Korean and English instructions, and performs visual reasoning beyond the specialized heritage domain.

Performance overview

The chart highlights representative metrics from each major capability area. Detailed benchmark tables appear in the Evaluation section.

Quickstart

Qwen3.5 support and AutoModelForMultimodalLM may be unavailable in older Transformers releases. The commands below reproduce the clean environment used to verify this repository. torchvision is required when AutoProcessor initializes the bundled video processor, including for image-only inference.

pip install "torch==2.9.1" "torchvision==0.24.1" accelerate pillow safetensors
pip install "transformers @ git+https://github.com/huggingface/transformers.git@c93057d4835cd31752bb56f59989dd27696eb45b"

When the model files are stored at the root of a Hugging Face repository:

import torch
from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "KETI-AIR/Qwen3.5-KETI-HAECHI-27B"
image_url = (
    f"https://huggingface.co/{model_id}/resolve/main/"
    "samples/qualitative/06_cheomseongdae.jpg"
)

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
).eval()

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "url": image_url,
            },
            {
                "type": "text",
                "text": "사진 속 국가유산의 정확한 공식 명칭만 답하세요. 설명은 쓰지 마세요.",
            },
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
    enable_thinking=False,
).to(model.device)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=32,
        do_sample=False,
    )

generated_ids = output_ids[:, inputs["input_ids"].shape[1]:]
answer = processor.batch_decode(generated_ids, skip_special_tokens=True)[0].strip()
print(answer)

# Example output from this checkpoint:
# 경주 첨성대

For the packaged directory in this release, use model_id = "./model" and image_url = "samples/qualitative/06_cheomseongdae.jpg". For a text-only request, omit the image item and retain only a {"type": "text", "text": ...} item in the message content.

Thinking mode is enabled by default in the bundled chat template. The example disables it for concise answers. Set enable_thinking=True for tasks that benefit from explicit reasoning, and adjust the generation budget accordingly.

Evaluation

All reported deltas are calculated as Qwen3.5-KETI-HAECHI-27B minus the untouched official Qwen/Qwen3.5-27B checkpoint. Percentage deltas are absolute percentage points. Both models used matched API, chat-template, thinking, stop-token, parser, and dataset settings.

Qualitative comparison with the base model

Korean cultural-heritage recognition

Korean OCR

Font

Outdoor

Public exec

Quantitative benchmarks

Korean cultural-heritage performance

H400 is an internal evaluation set built around 400 Korean cultural-heritage items. It evaluates canonical-name recognition, fine-grained discrimination among visually similar heritage items, and heritage knowledge grounded in either images or text. H400 direct exact measures open-ended canonical-name retrieval, whereas H400 hard is closed-set selection. Their large gap indicates that recognition and discrimination are substantially stronger than exact free-form naming.

Korean (Hangul) OCR performance

These Mammoth OCR results use strict exact match, where additional prose or differences in spacing and normalization can cause an otherwise useful answer to be marked incorrect.

Tool calling and long-horizon task performance

Tau2

Tau2 measures end-to-end success across multi-step airline, retail, and telecom workflows. It improves overall, especially in airline and retail, although the telecom domain declines.

BFCL V4

BFCL V4 measures structured function-call generation and multi-turn recovery from missing functions or parameters. The small overall regression indicates that tool calling and long-horizon performance remain sensitive to the environment and task protocol.

General multimodal and language retention

See the detailed score report for all 148 metrics, the benchmark guide for definitions and caveats, and the machine-readable results for downstream analysis.

Intended use

The model is intended for research and prototyping involving:

• Korean cultural-heritage image identification and visual question answering;
• Korean scene, sign, font, and public-document OCR;
• image-grounded heritage knowledge retrieval;
• multi-step, stateful tool-use workflows with explicit monitoring and recovery;
• general image understanding and text generation; and
• analysis of domain specialization and capability retention in multimodal models.

Responsible use

Verify cultural-property names and factual claims against authoritative catalogs or domain experts. Preserve human review for public descriptions, education, archival metadata, and research outputs. Do not treat model output as evidence of authenticity or provenance. Avoid submitting sensitive or personal documents for OCR unless the deployment provides appropriate privacy controls.

Limitations

• OCR outputs may omit text, normalize spelling incorrectly, or add unsupported text.
• The model can hallucinate names, dates, designations, provenance, and historical claims. Similar-looking artifacts and uncommon viewpoints are especially challenging.
• General multilingual, video, robustness, demographic-bias, privacy, and safety behavior were not comprehensively evaluated for this release.
• As with the base model, generated content may be inaccurate, biased, unsafe, or unsuitable for the user's context.

Core contributors

• San Kim ([email protected]) — Led long-context agent task performance and development/management of the main training loop.
• Byunggill Joe ([email protected]) — Secured GPU compute resources through the support program and led Korean cultural-heritage recognition/OCR performance.

Acknowledgements

Qwen3.5-KETI-HAECHI-27B was developed using compute resources provided through the 첨단 GPU 활용 지원 사업 of the National IT Industry Promotion Agency (NIPA; 정보통신산업진흥원).

License and attribution

This derivative checkpoint is released under the Apache License 2.0, following the upstream Qwen/Qwen3.5-27B license. Use of third-party data and generated outputs may be subject to additional terms. Please acknowledge the Qwen team and cite the upstream model when using this checkpoint in published work.

Citation

@misc{qwen35_keti_haechi_27b,
  title        = {Qwen3.5-KETI-HAECHI-27B},
  author       = {Kim, San and Joe, Byunggill},
  year         = {2026},
  howpublished = {Hugging Face model repository},
  url          = {https://huggingface.co/KETI-NLP/Qwen3.5-KETI-HAECHI-27B}
}

한국어 요약

이 모델은 Qwen/Qwen3.5-27B를 기반으로 두 가지 목적을 위해 개발한 멀티모달 모델입니다. 첫 번째 목적은 한국 문화유산의 정확한 명칭 식별, 이미지 기반 문화유산 질의응답, 한글 OCR 능력을 향상하는 것입니다. 두 번째 목적은 적절한 도구를 호출하고 여러 단계에 걸쳐 정보를 기억하며 변화하는 상태를 추적하는 장기작업 수행 능력을 향상하는 것입니다.

공식 베이스 모델과 동일한 조건의 내부 평가에서 H400 직접 식별 정확도는 0.92%에서 38.07%로, H400 유사 문화재 선택 정확도는 26.45%에서 80.43%로 향상되었습니다. 장기작업 평가인 Tau2 가중 점수는 69.79%에서 71.88%로 향상되었습니다. 다만 세부 환경에 따라 성능 차이가 있으므로, 실제 에이전트 작업에서는 단계별 상태 확인과 실패 복구 절차를 함께 적용하는 것을 권장합니다.