MANN-GPT

Overview / 模型简介

MANN-GPT v6 is a memory-augmented autoregressive language-model architecture built around causal MANN layers, a formally qualified Global Memory v2 path, and a Dynamic Thinking Network (DTN). This repository is the frozen architecture and code archive for the first formally qualified bilingual continuation line.

MANN-GPT v6 是一种记忆增强型自回归语言模型架构，其核心由 因果 MANN 层（causal MANN layers）、已完成正式资格验证的 Global Memory v2 全局记忆路径，以及 动态思考网络（Dynamic Thinking Network, DTN） 构成。本仓库保存首条完成正式资格验证的中英双语继续训练线路所对应的冻结架构与代码归档。

[!IMPORTANT] Stable architecture freeze / 稳定架构冻结版本: v6-mature-20260910
Canonical qualified parent / 当前正式合格父模型: Formal Bilingual Continuation Phase-1 · Seq512 · 20260909T133242Z
Long-context experiment / 长上下文实验: Phase-2 · Seq1024 · HOLD / not promoted（未晋升）

Core design / 核心设计

Release Status / 当前状态

The canonical Phase-1 checkpoint passed value qualification, Global-Memory causal-identity retention, exact Q2 preservation, fixed-D3 retention, strict autoregressive causality, protected-state exactness, optimizer-resume lineage, and exact checkpoint loadback.

当前标准 Phase-1 checkpoint 已通过价值验证、Global Memory 因果身份保持、Q2 精确保留、固定 D3 保持、严格自回归因果性、受保护状态精确性、优化器续训谱系，以及 checkpoint 精确回载等资格检查。

Architecture at a Glance / 架构概览

flowchart LR
    A[Token IDs<br/>词元 ID] --> B[Embedding + Semantic Adapter<br/>嵌入 + 语义适配器]
    B --> C[26 Causal MANN Layers<br/>26 层因果 MANN]
    C --> D[Fixed Cumulative D3 DTN<br/>固定累计 D3 DTN]
    D --> E[RMSNorm]
    E --> F[Tied LM Head<br/>权重绑定 LM Head]

    subgraph L[MANN Layer / MANN 层]
      L1[Gated Causal Depthwise Local Conv<br/>门控因果深度局部卷积] --> L2[MemorySelector]
      L2 --> L3[q/k/v Causal Relation Network<br/>q/k/v 因果关系网络]
      L3 --> L4[ReLU² Normalized Connections<br/>ReLU² 归一化连接]
      L4 --> L5[Iterative Activation Propagation<br/>迭代激活传播]
      L5 --> L6[SwiGLU FFN]
    end

    subgraph GM[Global Memory v2 @ L24 / L24 全局记忆 v2]
      G1[SourceRetention Writer<br/>SourceRetention 写入器] --> G2[16-token Segment Slots<br/>16-token 分段槽位]
      G2 --> G3[Causal Remote q/k Reader<br/>因果远程 q/k 读取器]
      G3 --> G4[Local + β·Remote Propagation<br/>局部 + β·远程传播]
    end

The relation path is causal and uses q/k geometry with RoPE, ReLU² non-negative relation scores, and explicit normalization rather than a softmax attention router. Global Memory v2 adds source-retention-weighted segment slots and a remote causal reader outside the 256-token local relation window.

关系路径保持严格因果性，并采用带 RoPE 的 q/k 几何结构、ReLU² 非负关系分数以及显式归一化，而不是使用 softmax attention router。Global Memory v2 进一步加入由 SourceRetention 加权的分段记忆槽位，并通过远程因果读取器访问 256-token 局部关系窗口之外的信息。

Frozen Model Geometry / 冻结模型几何参数

Mature Continuation Contract / 成熟继续训练契约

During the formally qualified continuation line, the protected architecture remains fixed while only the L23–L25 Tail-MANN scope is adapted. The Global Memory reader, SourceRetention writer, bilingual protected embeddings/rows, historical v_proj, and DTN remain frozen. Causal-retention replay is used to prevent target-memory identity drift during continued learning.

在正式合格的继续训练线路中，受保护架构保持冻结，仅适配 L23–L25 的 Tail-MANN 范围。Global Memory reader、SourceRetention writer、中英双语受保护 embedding/rows、历史 v_proj 与 DTN 均保持冻结；继续学习过程中使用 causal-retention replay（因果保持重放），以避免目标记忆身份发生漂移。

Canonical Phase-1 contract / Phase-1 标准训练契约

Qualification Evidence / 资格验证证据

Canonical stable run / 标准稳定运行

Run: 20260909T133242Z
Verdict: FORMAL_BILINGUAL_CONTINUATION_PHASE1_SEQ512_QUALIFIED_PROMOTE_PHASE2_CONTEXT_RAMP

Checkpoint integrity / Checkpoint 完整性

formal_phase1_state.pt
ce1d9e7abf5d04e78f63bfa2a23bf3bee4bd1a1f3c940ddf4af18cb78e0928b8

optimizer_state.pt
bae2f679f27a0af7bdccdb016b25f48b4e762dd942a1ac84845e4e5294a365f2

Hugging Face checkpoint path / HF checkpoint 路径

checkpoint/formal_bilingual_continuation/phase1/20260909T133242Z/qualified

Repository Layout / 仓库结构

MANN-GPT-v6/
├── src/
│   ├── base_architecture.py
│   │   └── Qualified base MANN-GPT v6 architecture
│   │       已通过资格验证的 MANN-GPT v6 基础架构
│   │
│   ├── global_memory_components.py
│   │   └── SourceRetention writer, segment slots & Global Memory v2 reader
│   │       SourceRetention 写入器、分段槽位与 Global Memory v2 reader
│   │
│   ├── global_memory_install.py
│   │   └── L24 installation path & causal intervention hooks
│   │       L24 安装路径与因果干预 hooks
│   │
│   └── dtn_policy.py
│       └── Qualified fixed cumulative D3 policy
│           已通过资格验证的固定累计 D3 策略
│
├── docs/
│   ├── MATURE_ARCHITECTURE_FREEZE_20260910.md
│   │   └── Design freeze, responsibilities, invariants & stable/experimental boundary
│   │       架构冻结、职责、不变量及稳定/实验边界
│   │
│   └── FORMAL_TRAINING_STATUS_20260910.md
│       └── Qualification state & Phase-2 HOLD diagnosis
│           资格状态与 Phase-2 HOLD 诊断
│
├── manifests/
│   └── v6_mature_20260910.json
│       └── Machine-readable release/checkpoint lineage
│           机器可读的版本与 checkpoint 谱系
│
├── experiments/
│   ├── formal_phase1_20260909/README.md
│   │   └── Qualified Phase-1 evidence / Phase-1 合格证据
│   └── formal_phase2_20260910/README.md
│       └── Seq1024 HOLD evidence; not a promoted parent
│           Seq1024 HOLD 证据；未晋升为父模型
│
└── scripts/
    └── publish_to_hubs.py
        └── Hugging Face & ModelScope publication helper
            Hugging Face 与 ModelScope 发布辅助脚本

Stable vs Experimental / 稳定版本与实验版本

[!NOTE]

Stable / 稳定版本

MANN-GPT v6 + Global Memory v2 + frozen-reader continuation policy + fixed cumulative D3 + qualified Phase-1 Seq512 checkpoint
MANN-GPT v6 + Global Memory v2 + 冻结 reader 继续训练策略 + 固定累计 D3 + 已合格 Phase-1 Seq512 checkpoint

[!WARNING]

Experimental — not promoted / 实验版本——未晋升

Seq1024 Phase-2 context ramp retained memory utility and strict causality, but Q2 target attribution fell below the pre-registered floor and optimization gain did not reach the qualification floor.
Seq1024 Phase-2 上下文扩展实验 保持了记忆效用与严格因果性，但 Q2 目标归因低于预注册门槛，同时优化增益未达到资格门槛，因此不作为新的稳定父模型。

Repositories / 项目仓库

License / 许可证

This project is released under the MIT License. See LICENSE for details.

本项目采用 MIT License 开源。详细条款请参阅 LICENSE。