DrosophiLLM-0.5B

--- license: mit library_name: transformers tags: - neuromorphic - connectomics - drosophila - flywire - sparse-computation - causal-lm - qwen datasets: - flywire-codex-2024 pipeline_tag: text-generation

DrosophiLLM: Connectome-Constrained Autoregressive Language Routing

Architect & Author: Robin-Kevin Vettik (@Robillionair)
Organization: Robillionair OÜ
Base Architecture: Qwen/Qwen2.5-0.5B-Instruct
Biological Substrate: Adult Drosophila melanogaster Whole-Brain Connectome (FlyWire / Codex, Nature 2024)
License: MIT (Copyright © 2026 Robillionair OÜ / Robin-Kevin Vettik)

The Core Thesis: Beyond Dense Matrix Projections

Contemporary Large Language Models rely universally on dense, rectangular projections ($O(N^2)$). Every token representation is forced through uniform, all-to-all matrix transformations that treat every latent channel with identical geometric indifference.

Biological nervous systems operate under radically different evolutionary constraints. The adult fruit fly (Drosophila melanogaster) coordinates complex 3D aerobatics, active olfactory search, spatial dead reckoning, and visual threat evasion across ~139,000 neurons and ~54.5 million synapses—operating on a biological power budget under 1 milliwatt.

DrosophiLLM tests what happens when an autoregressive language model's latent residual stream is topologically constrained by an authentic biological connectome:

• Real Biological Wiring: An anatomically isolated 2,048-neuron, 59,963-synapse sub-circuit spanning the Antennal Lobe (AL), Optic Lobe (OPT), Mushroom Body (MB), Central Complex (CX), and Descending Neurons (DN).
• Dale's Principle Invariance: Synapses enforce biological neurotransmitter polarities—Acetylcholine remains strictly excitatory ($\ge 0$), while GABA and Glutamate (via GluCl channels) remain strictly inhibitory ($\le 0$).
• Pure $O(E)$ Sparse CSR Compute: Matrix-vector passes are executed via compressed sparse row representations (torch.sparse_csr_tensor), eliminating dense $O(N^2)$ memory footprint.
• Recurrent Settling ($k=3$): Representations settle over three recurrent iterations with membrane leak integration before projecting back to the transformer residual stream.

Architecture & Mathematical Formulation

At layer $l = 12$ of Qwen2.5-0.5B-Instruct, the residual vector $x_l \in \mathbb{R}^{B \times S \times D}$ ($D = 896$) is intercepted and projected into biological coordinate space:

$$u = x_l W_{\text{sensory}} \quad \text{where } W_{\text{sensory}} \in \mathbb{R}^{D \times N}, \; N = 2048$$

The recurrent settling state evolves across steps $t \in {0, 1, 2}$:

$$h^{(t+1)} = (1 - \sigma(\lambda)) \odot h^{(t)} + \sigma(\lambda) \odot \text{ReLU}\left( W_{\text{bio}} h^{(t)} + u \right)$$

Where $W_{\text{bio}}$ enforces biological Dale's law:

$$W_{\text{bio}} = M_{\text{sign}} \odot \text{softplus}(\Theta_{\text{gain}}) \odot \log(1 + C_{\text{syn}}) \cdot \frac{1}{\bar{d}}$$

• $M_{\text{sign}} \in {-1, +1}^{N \times N}$ is the fixed neurotransmitter polarity matrix from the FlyWire consensus.
• $C_{\text{syn}}$ is the verified physical synapse count from electron microscopy reconstruction.
• $\Theta_{\text{gain}}$ is a learnable synaptic plasticity parameter.
• $\lambda \in \mathbb{R}^N$ is the biological membrane leak parameter.

The settled activity $h^{(3)}$ is projected back into token space via the descending motor readout:

$$x_{l,\text{out}} = x_l + \text{LayerNorm}(h^{(3)} W_{\text{motor}})$$

The base language model parameters remain 100% frozen; only the sensory projector, motor readout, and biological synaptic gains are optimized.

5-Way Empirical Ablation Benchmark

To determine whether the biological connectome acts as a meaningful structural prior or merely an arbitrary sparse mask, DrosophiLLM was evaluated on 150 procedural Spatial Vector Navigation tasks alongside 150 WikiText-2 retention tasks against four null-hypothesis configurations:

Hardware Profiling: Sparse CSR vs. Dense Baseline

Benchmarked across 6 batch $\times$ sequence length combinations on equivalent $2048 \times 2048$ dimensions:

• Weight Memory Compression: Sparse CSR allocates 0.70 MB vs. 16.00 MB for Dense FP32 (22.8× compression factor).
• VRAM Efficiency: Consistently saves $\sim 14.6\text{ MB}$ of active working set per forward pass.

Anatomical Telemetry & Functional Localization

By measuring regional sub-circuit activations during generation, DrosophiLLM demonstrates clear functional routing:

• Motor Output Localization: Descending Neurons (DN) exhibit statistically significant activation variance on directional/motion tokens compared to neutral text ($t = 8.85, p = 1.29 \times 10^{-9}$, Welch's $t$-test).
• Kenyon Cell Associative Hubs: High-degree Kenyon cells (KC_0742, KC_0964) act as primary associative firing clusters, driving transient cholinergic spikes during decision sequences.

Quickstart

git clone https://github.com/Rob-bio4/DrosophiLLM.git
cd DrosophiLLM
pip install -r requirements.txt

1. Run Comprehensive Unit Tests

pytest tests/test_neuromorphic.py -v

2. Run 5-Way Ablation Benchmark

python -m fly_neuro.train_eval --neurons 1024 --epochs 2 --samples 150

3. Run Hardware Memory & Latency Profiler

python -m fly_neuro.benchmark_efficiency

4. Run Anatomical Telemetry Probing

python -m fly_neuro.probing --samples 25

5. Launch Live Hugging Face Gradio Space (Interactive 3D Connectome)

python app.py

Navigate to http://localhost:7860 to query DrosophiLLM with spatial decision prompts and observe real-time 3D synaptic activation patterns.

Scientific Citations

@article{dorkenwald2024neuronal,
  title={Neuronal wiring diagram of an adult brain},
  author={Dorkenwald, Sven and McKellar, Claire E and Macrina, Thomas and others},
  journal={Nature},
  volume={634},
  pages={124--138},
  year={2024},
  publisher={Nature Publishing Group},
  doi={10.1038/s41586-024-07558-y}
}

@article{schlegel2024whole,
  title={Whole-brain annotation and multi-connectome marker atlas for Drosophila},
  author={Schlegel, Philipp and Yin, Yijie and Bates, Alexander S and others},
  journal={Nature},
  volume={634},
  pages={139--152},
  year={2024},
  publisher={Nature Publishing Group},
  doi={10.1038/s41586-024-07686-5}
}

Copyright © 2026 Robillionair OÜ / Robin-Kevin Vettik. Released under the MIT License.