Koinophobia LabsLab · KOI · Research · a language model from nothing
Teaching a model to read, from random weights, on a laptop.
KOI is not a fine-tune and not a wrapper. The tokenizer, the transformer, the training loop, and the local inference server are all the studio's own code, and the weights started as noise. The corpus is one the studio owns. The question being asked is small and exact: what does a model this size actually learn, and how would you know?
What it has not learned
Most experiments end in a verdict of REJECTED, and every rejection is a receipt.
Long-range agent closure: 0.12 against a 0.10 baseline. Not learned. Free-generation format validity: 0.00. Not learned. Verbatim echo: failed; the copy wall is characterised.
“It is not useful yet. That is the next milestone, not this one.”
How it's run
Eighteen lanes, one baseline, one GPU.
Eighteen experiment lanes as git worktrees, one certified integration baseline, one heavy job at a time because the machine has one GPU. Seventy-six receipt files for the third epoch. A falsifier that reverts a change when the bar isn't cleared: the latest commit reads “MORPH-1 VERDICT: bar FAILED at diagnose 5/8, reverted per the rule.”
Why it's in the lab
A falsifier that reverts the change when the bar isn't cleared.
“A research log where every experiment succeeds is a diary. The rule is written down, the bar is written down, and the commit that fails it reverts itself.”
Status
Local development. Verified 2026-09-13.
A from-scratch pretraining program running on one Apple M4. The largest documented runs are a few million parameters; the README says it is not useful yet.
- It cannot generate useful text. The README says so and this page says so.
- No model has been published, served to anyone but Blake, or compared against an external benchmark.
- The 30-million-parameter variant is referenced in the log but its results are not summarised here.
- Weights began random, on a corpus the studio owns; no external weightskoi-model README and run manifests (initialization: random, external_weights_used: false), read 2026-09-13
- koi-0.1 is 869,504 parameters, trained 1,500 steps in 75 s, held-out perplexity 2.11koi-model README run record for koi-0.1 (4 layers, 4 heads, 128 dims, block 128, vocab 512), read 2026-09-13
- Most versioned experiments end in rejection, and the latest reverted itselfkoi-model EXPERIMENTS.md (~25 versioned verdicts, e.g. 'koi-0.10 — REJECTED for promotion, falsifier fired') and the 2026-08-24 commit 'MORPH-1 VERDICT: bar FAILED at diagnose 5/8 — reverted per the rule'
- The receipts exist on disk~20 *-receipt.json files at the koi-model root and 76 receipt files in koi-epoch3-receipts; 18 experiment worktrees governed by koi-lanes/LANES.md, listed 2026-09-13
Next. Characterise the 30-million-parameter variant on the same falsifier, and publish the first run whose free generation is well-formed, if one appears.