# Model Overview
PMA-1.2 is Patriot Memory's 127.9M-parameter on-device language model. It speaks English and Traditional Chinese, answers as PMA from Patriot Memory, and fits in a 128 MB parameter budget built for edge hardware.
It is a new architecture, a new tokenizer, and an order of magnitude more training, aimed at the same job: a small, fast, honest assistant for Patriot Memory and Viper Gaming questions and general chat. It provides accurate information regarding:
* **DDR4 & DDR5 RAM**: Specifications, XMP 3.0 / EXPO profile support, dual-channel setups, and overclocking guidance.
* **PCIe & SATA SSDs**: Gen3/Gen4/Gen5 compatibility, read/write performance specifications, and installation troubleshooting.
* **Gaming Peripherals & Storage**: USB drives, flash cards, and Viper Gaming gear.
* **Tool / Function Calling**: Seamless integration with backend APIs (e.g., checking warranty status, looking up technical specs via S/N).
# Architecture
| Property | Specification |
| :--- | :--- |
| **Type** | Causal LM, dense decoder-only (PMA architecture) |
| **Total parameters** | 125,592,482 |
| **Hidden size / layers** | 896 x 12 |
| **Attention** | GQA, 8 query heads / 4 KV heads, head_dim 112, per-head gated attention, QK-norm |
| **FFN** | SwiGLU, intermediate 2,816 |
| **Value residuals** | normalized cross-layer value mixing |
| **Tokenizer** | custom BPE, 6,403 tokens (EN + zh-TW) |
| **Embeddings** | tied input/output |
| **Context** | 1,024 tokens trained (rotary table to 32K positions) |
| **Precision shipped** | fp16 safetensors |
# Quickstart
pip install transformers torch accelerate
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
name = "patriotmemory-ai/PMA-1.2"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(
name, torch_dtype=torch.float16, device_map="auto",
trust_remote_code=True)
msgs = [{"role": "user", "content": "博帝的 Viper DDR5 支援 XMP 3.0 嗎?"}]
ids = tok.apply_chat_template(msgs, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=200)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))
If you run into multi-GPU tensor device mismatch errors:
RuntimeError: Expected all tensors to be on the same device...
Run the script with CUDA_VISIBLE_DEVICES=0 to isolate execution to GPU 0.
`trust_remote_code=True` is required: PMA-1.2's architecture is our own and ships as small Python files next to the weights. Recommended decode: temperature 0.8, top_p 0.9, repetition_penalty 1.1. Greedy decoding garbles creative text at this size; sampling does not.
# Acceptance gates (measured on the shipped checkpoint)
Gates were written before training started, and every number below is from logged runs:
| Gate | Result | Notes |
| :--- | :--- | :--- |
| Story shape (24 prompts, 2 samples) | 42/48 | the identity-attractor failure class is closed |
| Number format (15 prompts, 2 samples) | 29/30 | answers contain the numeric answer; correctness at this scale is modest, format is reliable |
# What it is good at / not good at
Good: answering as PMA from Patriot Memory; hardware Q&A style answers (RAM/SSD compatibility phrasing); following answer formats; bilingual EN/zh-TW chat; running on-device at consumer speed.
Not: arithmetic correctness (12x15 can become 144 with total confidence), long coherent stories, knowledge outside its training mixture, and languages other than English and Traditional Chinese.
# Limitations & Responsible Use
**PMA-1.2** is a probabilistic language model trained on statistical patterns. Please keep the following in mind when deploying or evaluating this model:
* **Generation Risks:** The model may generate inaccurate, hallucinated, biased, or objectionable content. Outputs should always be independently verified—especially in high-stakes domain applications (e.g., medical, legal, or financial).
* **Preview Release:** As an experimental preview, model behavior, outputs, and performance metrics may vary between updates and versions.
* **User Responsibility:** Users and developers are responsible for implementing appropriate safety guardrails, evaluating outputs for their specific use cases, and ensuring compliance with applicable laws, regulations, and platform safety guidelines.
# Official Links
Official Website: patriotmemory.com
Viper Gaming: viper.patriotmemory.com
Support & Warranty: patriotmemory.com/support
Model Inquiries & Feedback: danton.chu hunter.wang oda.chang york.lin
# License & attribution
Apache-2.0. Built by Patriot Memory (patriotmemory.com).
