Meridian.AI — Continual-Learning Finance LLM
Meridian.AI is a finance-specialized language model that continuously fine-tunes a
Qwen2.5-0.5B backbone every hour on 25+ finance and math datasets, using Elastic
Weight Consolidation (EWC) to prevent catastrophic forgetting across training sessions.
The entire pipeline runs unattended on free GitHub Actions infrastructure — no GPUs.
- Base model:
Qwen/Qwen2.5-0.5B (~494M params, Qwen2 architecture)
- Continual learning: Elastic Weight Consolidation (diagonal Fisher)
- Training cadence: hourly GitHub Actions CI on CPU runners
- Source code & full docs: github.com/MeridianAlgo/FinAI
Usage
The deployed checkpoint is a standard Qwen2 model — trust_remote_code=True is not
required.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "meridianal/FinAI"
tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="checkpoint")
model = AutoModelForCausalLM.from_pretrained(
repo_id,
subfolder="checkpoint",
torch_dtype=torch.float32,
low_cpu_mem_usage=True,
)
model.eval()
prompt = """### Instruction:
Explain the difference between a bond's yield to maturity and its coupon rate.
### Response:
"""
inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
output = model.generate(
**inputs,
max_new_tokens=200,
do_sample=True,
temperature=0.8,
top_p=0.92,
repetition_penalty=1.3,
no_repeat_ngram_size=3,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(output[0], skip_special_tokens=True))
Inputs are formatted with the ### Instruction: / ### Response: template used during training.
Model details
| Specification |
Value |
| Base model |
Qwen2.5-0.5B |
| Architecture |
Qwen2ForCausalLM |
| Parameters |
~494M |
| Context window |
32,768 tokens (Qwen2.5 default) |
| Training dtype |
bfloat16 |
| Continual learning |
Elastic Weight Consolidation (EWC) |
Training data
A weighted streaming mix of 25+ finance and instruction datasets, including
gbharti/finance-alpaca, sujet-ai/Sujet-Finance-Instruct-177k,
nvidia/OpenMathInstruct-2, HuggingFaceFW/fineweb-edu, yahma/alpaca-cleaned, and the
FinanceMTEB suite. See the
repository README for the full
curriculum and weights.
Limitations & disclaimer
This is an experimental research project on continual learning for financial NLP. Outputs
may contain factual errors and are intended for academic and research purposes only.
Nothing generated by this model constitutes financial advice. Do not use outputs to make
real financial decisions or execute trades.
