Qwen3.5-2B-DNA

Qwen3.5-2B-DNA (preliminary)

A Qwen3.5-2B backbone extended with DNA reading. Paste a gene sequence between <dna_start> and <dna_end> tags and ask about it in plain English. It can report gene type, organism domain, length and GC content, and describe what a sequence shows. It also continues DNA spans when asked.

Two extra tokens (<dna_start>, <dna_end>) bracket the sequence. Anything outside the tags is normal chat, and the model answers that normally.

Model Architecture

This modified version of Qwen3.5-2B includes an additional small custom DNA model that's been trained separately, and then inserted into the model's architecture, with projection layers in between the two to get a representation of the latter's tokens into the former's token space, and the vision tower has been removed, so the model retains no vision capabilities. The final model has then been finetuned to use these more naturally. The DNA tokeniser works in a slide-1 overlapping 3-mer, and the model's been trained on at most 2048 tokens, so roughly ~2046 base pairs.

Capabilities

This is only meant to be a prototype as of now, so don't expect it to work flawlessly. The inserted layers are tiny in comparison to even the vision tower they replace.

What it can mostly do as of now is classifying short genes, and continuing a span base by base.

Some things I didn't target for this first version include designing sequences to spec (GC, length and motif control are unreliable) and precise edits. Those are open problems for a future version.

This is a research preview, not a tool for medical or clinical use.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained('snowyautumn/Qwen3.5-2B-DNA')
model = AutoModelForCausalLM.from_pretrained(
    'snowyautumn/Qwen3.5-2B-DNA', trust_remote_code=True,
    dtype='bfloat16', device_map='cuda')
model.eval()

batch, info = model.build_inputs(tok, 'Look at <dna_start>REPLACE WITH SOME FASTA SEQUENCE<dna_end> and state its gene type.')
out = model.generate(**batch, max_new_tokens=256, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0], skip_special_tokens=True)) # The final response from the model.
print(info['domain_pred']) # This comes directly from the prediction head, which is usually more accurate than what the model might report in text.