RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS-B searches full-prefix and norm-filtered forests of chunk mean keys, reranks their candidates, and reads four chunks. Its attention is superlinear and subquadratic. This 743M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without…
Independent publisher
Zekun Shi
zekun-shi
Models in Library6
Datasets in Library0
Models on Hugging Face6
Followers—
Compare the published evidence
This profile connects the 6 model and dataset records shown below to their upstream cards and sources. The library counts describe SAVRN's coverage, rather than this publisher's entire output.
Published tasks: Text generation.
Reported licenses: mit. Open a record to review its specific license and conditions.
Compare model specifications, source dates, and hardware estimates on the individual records. Publisher identity verification is separate from a model's quality, deployment readiness, or suitability for your work.
Models
RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS-L retains a bounded pool of chunks by key norm and recency, then reads four selected chunks. Its attention scales linearly with sequence length. This 743M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without rotating their queries…
RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS-B searches full-prefix and norm-filtered forests of chunk mean keys, reranks their candidates, and reads four chunks. Its attention is superlinear and subquadratic. This 341M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without…
RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS scores all completed chunk mean keys and reads four selected chunks per query and head. This 743M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without rotating their queries and keys. 12 tasks at 2k/4k; 13 at 8k/16k, where QA-2 is…
RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS-L retains a bounded pool of chunks by key norm and recency, then reads four selected chunks. Its attention scales linearly with sequence length. This 341M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without rotating their queries…
RoPE-free long-range attention, trained at 4k with zero-shot length extrapolation. FS scores all completed chunk mean keys and reads four selected chunks per query and head. This 341M research base model was trained on FineWeb-Edu at a 4,096-token context. The short retrieval example asks for the hidden number 5437923. Start with CUDA-enabled PyTorch; Usage gives the tested installation and speed-replication command. FreeSolo learns simple chunk mean keys that support search, norm pruning, and hierarchical summarization. RoPE stays in the local window; long-range attention reads selected original tokens without rotating their queries and keys. 12 tasks at 2k/4k; 13 at 8k/16k, where QA-2 is…