MiniCPM5-2B-pjev-LoRA

pjev-2b as a LoRA for MiniCPM5-2B

A one-token decision model you can switch on per request, on the MiniCPM5-2B server you already run.

Kailune-AI/pjev-2b is a strong typed-decision model: give it a state, a question and lettered options, and it returns a calibrated probability for every option in a single forward pass. It ships as a full merged model (about 2.7 GB in Q8_0). If your agent already serves openbmb/MiniCPM5-2B for chat, a second full model costs VRAM you probably do not have.

This repository re-expresses pjev-2b as a rank-64 LoRA over the stock MiniCPM5-2B weights. Load it once next to the chat model, turn it on only for decision requests, and pay about 200 MB of VRAM instead of 2.7 GB.

Results

Measured on a private held-out set of 62 English decisions taken from a real local agent harness: which read-only tool to call now (24), which UI element to act on (22), what state a web page is in (16). The set was written before any of these models were tried and was never used for tuning. One question is worth 1.6 points.

The LoRA picks the same option as the merged pjev-2b on 60 of 62 questions. All wrong answers of the merged model had low confidence (0.34 to 0.61), so a confidence threshold is a natural place to hand off to a slower second stage.

Setup: llama.cpp b10549 (b2e5e9b28), MiniCPM5-2B Q8_0 (official GGUF), KV cache q8_0, GTX 1080 8 GB (Pascal). Latency includes prompt processing of about 340 tokens.

Italian, for the record: 69.4% with the LoRA, 64.5% without. pjev-2b was trained on English; use it in English.

How it was made

1. Same base, verified. All 87 non-linear tensors of pjev-2b (embeddings, norms, output head) are bit-identical to openbmb/MiniCPM5-2B (revision f97400052a43). The fine-tune lives entirely in the seven linear projections of each layer (q k v o gate up down).
2. Low rank, measured. The singular values of W_pjev - W_base fall off a cliff at rank 64: 99.9% of the energy of an attention projection sits in the first 64 directions; what is left on the MLP projections is flat, which is bf16 rounding of the merge, not signal.
3. Truncated SVD per matrix (294 matrices, rank 64, B = U sqrt(S), A = sqrt(S) V^T), saved in PEFT format with lora_alpha = r = 64, then converted with llama.cpp convert_lora_to_gguf.py.

The extraction is an approximation of the merged model, which is why the two rows above differ by one question.

Use it with llama.cpp

Start the server once with the adapter loaded but off by default:

llama-server -m MiniCPM5-2B-Q8_0.gguf \
  --lora pjev-lora-r64-f16.gguf --lora-init-without-apply \
  -c 32768 -ngl 999 -fa on --jinja

Then send decision requests with the adapter at scale 1, and every other request with scale 0:

{"prompt": "<rendered prompt, see below>", "n_predict": 1, "n_probs": 50, "temperature": 0,
 "grammar": "root ::= [A-G]", "lora": [{"id": 0, "scale": 1.0}]}

Watch out (llama.cpp b10549): the LoRA scale set by a request stays on that slot. A later chat request on the same slot without a lora field runs with the adapter still on. Either pass "lora": [{"id": 0, "scale": 0}] on every chat request, or give the decider its own slot (id_slot).

Read the answer from the probabilities of the letter tokens at the first generated position (the n_probs list), not from sampled text. Softmax over the letters in play gives the per-option probability.

Prompt format

The adapter only works with the format pjev-2b was trained on (the "semif" style of its decision_core.py): a fixed system message, a JSON user turn, and the chat template with thinking disabled.

System message, verbatim:

Apply the supplied criterion to the supplied evidence. Choose exactly one listed option. Respond with only its uppercase letter, with no explanation or reasoning.

User message (one line of JSON):

{"evidence": "User message: \"will it rain in Lisbon tomorrow morning?\"\nTool results in this turn: none.",
 "criterion": "Which read-only tool must be called NOW, before answering the user?",
 "options": [{"letter": "A", "description": "weather: forecast for a named place, next 7 days"},
             {"letter": "B", "description": "news_search: news about a subject within a window of days"},
             {"letter": "C", "description": "none: no tool is needed now, answer directly"}]}

Render with the MiniCPM5 chat template and enable_thinking=False (the assistant turn opens with an empty <think>\n\n</think>\n\n block); the next token is the letter.

Use it with transformers + PEFT

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

tok = AutoTokenizer.from_pretrained("openbmb/MiniCPM5-2B")
model = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-2B", torch_dtype="auto")
model = PeftModel.from_pretrained(model, "adriandj3/MiniCPM5-2B-pjev-LoRA")

Build the messages as above, call tok.apply_chat_template(..., add_generation_prompt=True, enable_thinking=False), run one forward pass and softmax the logits of the option letters at the last position.

Limitations

• English in practice. Inherited from pjev-2b.
• Small benchmark. 62 held-out questions from one harness. The difference with the merged model (one question) and with LFM2.5 full reasoning is inside the noise; the difference with the base MiniCPM5 prompts is not.
• Approximation. Rank-64 SVD of the merged delta: very close to pjev-2b, not identical.
• No abstention. Like pjev-2b, it always picks one of the listed options. Add a "none" option when that is a valid answer, and use the confidence.
• Not evaluated on JevBench in this form. See the pjev-2b card for its JevBench numbers.

License and credits

• Weights: Apache-2.0. This is a derivative of Kailune-AI/pjev-2b (Apache-2.0), itself a fine-tune of openbmb/MiniCPM5-2B (Apache-2.0). Change made here: the merged fine-tune was re-expressed as a rank-64 LoRA by truncated SVD of the weight delta. See NOTICE.
• The decision format and letter readout come from the Eikos project (MIT, Caio Vicentino), as used by pjev-2b.
• Not affiliated with OpenBMB, Kailune-AI or the Eikos project. Made while building Vergilius, a local assistant for modest machines.