Research

Research

Technical reports, methods and evaluation code. Each report ships with the code and the results of every run, under Apache-2.0.

Technical reports

Figure 1. Three local layers precede each global layer.
Technical reportNovember 2026Leon Lehmann

Quark

128K Context and RL-Only Agentic Post-Training at 165M Parameters for $4,200 of Compute

A 165M-parameter hybrid model that reads 128K tokens and learned simple agentic tasks from reinforcement learning alone, with no supervised fine-tuning stage. The whole project cost about $4,200 of compute.

Figure 1. The complete stack fits the 1 to 1.5M target with the vocoder included.
Technical reportNovember 2026Leon Lehmann

PiVo

Two English Voices in 1.4 Million Inference Parameters

How many parameters does a voice need? PiVo speaks English in two synthetic voices with 1,401,562 learned parameters, vocoder included, and runs on a plain CPU. Its words come through clearly; some sentences still sound metallic.

Figure 1. Table layouts.
Technical reportOctober 2026Leon Lehmann

PageGram

Multi-Head N-gram Memory at the Read Cost of a Flat Table

Hashed n-gram tables add billions of parameters for little compute, but their heads scatter reads across flash. PageGram places every head of an n-gram in one page: two reads per token instead of sixteen, at equal or better loss.

Figure 1. One MGVR layer.
Technical reportOctober 2026Leon Lehmann

Identity on Demand

Multi-head Gated Value Residuals

Each attention head decides, per token, how much of the first layer's values to draw on. In a controlled 1B-token comparison, MGVR reaches the same loss in 16 to 22% fewer steps for 0.2% more parameters.

Correspondence

Questions about a report, or a result you could not reproduce: [email protected]. Corrections are welcome.