Haidass-Translate-143M-Instruction

English | 中文

The instruction-tolerant sibling of DALabCommunity/Haidass-Translate-143M: same 143M zh⇄en translation training, plus 9.1% cleaned general-domain data (STEP_FUN ShareGPT) mixed in. Translation scores are within 0.3 BLEU of the pure-translation version, and the model retains limited general instruction-following ability that the pure-translation version does not have.

Live demo: Haidass Translate Instruction on Hugging Face Spaces.

FLORES-200 dev

*Drafter-143M: a control model with identical configuration, data and training recipe, except that it starts from random initialization instead of the pretrained base — used to quantify the contribution of base-model pretraining.

OPUS-MT models are single-directional — one independent 78M model per direction; "-" marks directions a model does not serve.

FLORES+ devtest

The same models re-evaluated on FLORES+ devtest (released 2026; zero overlap with dev):

Instruction following

Mixing 9.1% general-domain data into the translation SFT gives this variant a limited ability to respond to general instructions (short poems, simple summaries, paraphrases) that the pure-translation sibling essentially lacks. This ability is fragile: prompts outside the translation domain may produce echoes of the input, partial answers, or drifting continuations. The model is still primarily a translator; treat general instruction following as a bonus, not a guarantee.

Training recipe

• Base: Haidass1.5-143M (Qwen3 architecture: 30 layers, hidden 576, GQA 9/3, vocab 64,000)
• Data: 7.837M cleaned zh↔en parallel sentence pairs (15.67M samples bidirectional) + 157,593 cleaned STEP_FUN ShareGPT conversations (9.1% of training tokens)
• Packing: official MindSpeed-LLM --pack --neat-pack (683,227 full 2048-token sequences with inter-document attention-mask isolation)
• Training: 16×Ascend 910C, GBS=256, lr 3e-5 cosine, 2 epochs = 5,337 steps (~2.8B tokens), final loss 1.788
• Framework: MindSpeed-LLM v2.3.0 + Megatron-LM core_v0.12.1 (NPU)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("umeiko/Haidass-Translate-143M-Instruction", torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained("umeiko/Haidass-Translate-143M-Instruction")

msgs = [{"role": "user", "content": "将以下文本翻译为英文：光子甚至比构成原子的物质还要小！"}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
# Even photons are smaller than the stuff that makes up atoms!

Note: the training data follows the qwen3 chat template (an empty <think></think> block precedes the assistant turn). Always use the model's built-in chat_template at inference; do not hand-craft prompts.

Known limitations

• zh→en BLEU plateaus at ~14 across data scales; more same-distribution parallel data does not push this direction further
• Typical residual errors: entity mix-ups, occasional omission of numeric details; rare-entity-dense zh→en inputs can trigger repetitive looping under greedy decoding (use repetition_penalty / no_repeat_ngram_size as mitigation)
• General instruction following is limited and unstable (see above)
• Optimized for zh⇄en; not a general chat model

Evaluation

• Metrics: sacreBLEU corpus BLEU (tokenize=zh for Chinese targets, tokenize=13a for English) + chrF++ (word_order=2); prompts byte-identical to the training chat template
• Decontamination: all 15.83M training samples audited against both benchmarks by n-gram overlap (8-word English / 10-character Chinese fragments): 1,147 hits (0.0072%) vs FLORES-200 dev, 1,788 (0.0113%) vs FLORES+ devtest; the STEP_FUN-only subset shows 6 hits (0.0038%) vs devtest. Manual inspection shows common-phrase-level overlap, not full-sentence leakage
• Raw predictions, audit reports and evaluation scripts for every model in the tables above: umeiko/Haidass-Translate-143M-eval

_{Note: all scores are measured on the FLORES Chinese–English subset (eng_Latn ↔ zho_Hans), bidirectional (997 sentences for dev, 1,012 for devtest), with greedy decoding.}