# sup computer

> A small language model studio: train small GPTs, write up the research, and show the results.

Every report, model card, and page has a markdown twin at the same path with `.md` in place of the trailing slash — the links below point at those twins, so any of them can be read directly. The instrument pages are HTML: each model's playable interface, running in the browser. For everything in one file, see [llms-full.txt](https://www.supcpu.com/llms-full.txt).

## Pages

- [Train a small model](https://www.supcpu.com/train.md): The studio's method as one prompt. Paste it into a coding agent and it trains a ~10M-parameter character-level GPT on TinyStories, end to end, on your machine.

## Instruments

- [daydream-chess-nanogpt](https://www.supcpu.com/daydream/): play — Plays chess without ever knowing the rules — learned move by move from games, not a rulebook, across three board sizes.
- [gatsby-nanogpt](https://www.supcpu.com/gatsby/): dial — Bends any story toward the green light — obsession you can dial from 1 to 5.
- [glyph-nanogpt](https://www.supcpu.com/glyph/): draw — Draws the lowercase alphabet one letter at a time — a single evolving model, released chasing the twenty-six specialists that out-drew it.
- [kenosha-kid-nanogpt](https://www.supcpu.com/kenosha-kid/): dream — Dreams endlessly on just six words.
- [pona-nanogpt](https://www.supcpu.com/pona/): talk — Speaks Toki Pona, and its whole vocabulary is the keyboard — about 130 words is a language small enough to get right.
- [shakespeare-nanogpt](https://www.supcpu.com/shakespeare/): write — Writes Shakespeare from scratch — and gets sharper every research round.

## Essays

- [1 month and 60 models later](https://www.supcpu.com/research/1-month-and-60-models-later.md): A month after the first model, I've trained ~60 and released 11, each starting from a question. These models are knowledge-light and shaped like their data, too small to chat with. The long-term bet is inventing harnesses that turn each one into an instrument.

## Lab notes

- [A language small enough to get right](https://www.supcpu.com/research/a-language-small-enough-to-get-right.md): Three 2.7M-param GPTs trained on 6.9M characters of Toki Pona and scored sentence-by-sentence by the community's grammar checker: the word-tokenized arms clear the span-thesis null, and the released chat model writes more grammatically than its own corpus (96.9% vs 96.0%). Mixing 4.6% oracle-filtered synthetic dialogue into the same recipe turns it into something you can talk to — 160/160 grammatical replies at temperature 0.8.
- [You can't parameterize a disagreement](https://www.supcpu.com/research/cant-parameterize-a-disagreement.md): Three thousand fonts generated from Knuth's own Computer Modern source cover no more of real type design than five hundred did — the ceiling is set by the program, not by the sample count. A third to a half of independently-set parameter combinations won't compile at all, and one boolean, the single-storey g, is worth more coverage than every continuous dial combined.
- [Three predictions from a font chapter](https://www.supcpu.com/research/three-predictions-from-a-font-chapter.md): A round-2 training plan built from a type-design chapter makes three predictions a model run would bet on — so all three were measured first, with zero training. One died (the chapter's craft axes explain 2.1% of what the generalist finds hard; outline complexity explains 17.9%), one passed (quantization binarized overshoot instead of erasing it), and one survived after the corpus corrected the plan itself (u is a round letter, whatever the shape grouping says).
- [One model or twenty-six?](https://www.supcpu.com/research/one-model-or-twenty-six.md): Twenty-six 1.8M-param GPTs, one per lowercase letter, against one letter-conditioned generalist at two sizes — all trained on 82k glyph outlines from 759 open-licensed sans-serifs. The 47.8M generalist wins mean bits-per-char by 2% but fails to draw a well-formed glyph 29% of the time where the specialists fail 8% — and it ships anyway, on purpose: the studio releases one evolving instrument, with the case's numbers frozen as the yardstick every future version has to overtake.
- [An instrument anything can play: why the studio ships a CLI](https://www.supcpu.com/research/an-instrument-anything-can-play.md): The studio's small models are instruments — single-purpose, played rather than prompted — and sup, the studio CLI, is the accessibility argument: one greeting downloads a release and streams its voice to stdout. A handle that simple works in a shell pipe, which means it works for another model.
- [Can a model's own likelihood hear register?](https://www.supcpu.com/research/the-likeliest-line-is-a-footnote.md): The shakespeare model's own likelihood is register-blind: fluent Gutenberg editorial prose scores inside any NLL band that admits verse, and the model's most inevitable text is the junk — footnotes, [Illustration] tags, speaker lists at 1.2–1.8 NLL — so the band's raised floor, not its ceiling, is the load-bearing edge. An LLM judge riding the same steer layer held verse register where the band drifted into publication history.
- [A borrowed cadence: where the house style comes from](https://www.supcpu.com/research/a-borrowed-cadence.md): The studio writes to a rulebook — twelve editing operations encoded as a skill, distilled from Anthropic's research posts, Thoughtful Lab, and Ramp Labs, piloted on two files and then swept across the studio: nine model cards, ten docs, and the website's visitor copy, 130-odd edits in all. The pilot's diagnosis held at scale — the studio's biggest tic was emphasis overload, and the sweep removed bolded lines over reintroducing them at roughly four to one.
- [Can a token budget buy a finished chess game?](https://www.supcpu.com/research/budget-cant-buy-the-midgame.md): Five rounds of Token Chess — a benchmark where LLMs may not author chess moves, only steer Daydream's sampler under a token budget — asked what a token actually buys. Round one: plies, at a worsening rate (every game forfeits; legality collapses 49% → 14% out of the opening book). Round two: remove death and nothing discriminates. Round three: price batches and candidate-picking with engine adjudication, and the benchmark finally separates players — then ministral, the worst sampler on the board, beats olmo 3–1 on pure tempo. Rounds four and five close on memory: assigned a free notepad, olmo writes zero notes in 24 games; offered the choice, it picks the notepad 24 seats out of 24, explains why every time, and still writes nothing.
- [A pass over the studio: one research loop across four models](https://www.supcpu.com/research/a-pass-over-the-studio.md): A single afternoon spent improving all four sup computer models at once — a larger model planned a per-model optimization, small runs executed it. Two new releases (shakespeare-nanogpt-3, kenosha-kid-nanogpt-2), one migration, one eval-only characterization, and a handful of findings that only show up when you look across projects side by side.
- [Can a chess model's illegal moves be the point?](https://www.supcpu.com/research/illegal-moves-are-the-point.md): A three-tier chess-move GPT family (5x5, 8x8, and a custom 12x10 board) built around a single inversion: illegal moves are rendered as dim near-misses instead of being masked away by the sampler. All three tiers land in a tight band of legal-move rate (35-39% on a raw, unresampled first try) despite very different board sizes, vocabularies, and corpus sources -- and two separate facts in the original design plan turned out to be wrong when checked against the live engine instead of trusted from web research.
- [The twenty-second training run: a bigger model cleans a smaller model's house](https://www.supcpu.com/research/twenty-second-training-run.md): A repo-wide audit by a larger model found the small-model studio's engine had two advertised code paths that crashed on use, a metric that quietly flattered char models, and a resume that restarted. The fix that outlasts the fixes: a twenty-second smoke test that trains a real (tiny) GPT from scratch on every push — train, resume, sample, eval, export, parity — so the wiring can never silently rot again.
- [Can four borrowed models write one obsession?](https://www.supcpu.com/research/mixture-of-models.md): gatsby's first corpus cost ~$6 of Claude API to write. This round throws that out and has a mixture of four local open models — Olmo, Ministral, Gemma, Granite — write the corpus instead: free, unlimited, and in four different voices. The model that results matches the paid baseline's behaviour at $0. The catch, and the finding: the blend is a designed object. A granite-heavy first round broke the green-light dial; rebalancing off it and doubling the data brought the dial back.
- [Can a model dream a single phrase?](https://www.supcpu.com/research/dream-a-single-phrase.md): The smallest obsession in the studio: a char-level model whose entire corpus is punctuated permutations of six words. A bot enumerates that space exactly; a learned model can't — and the blur it produces instead is the artifact. The finding: dreaminess is governed by two knobs, training progress and sampling temperature.
- [The logits oracle: running small models in the browser](https://www.supcpu.com/research/logits-oracle.md): Don't serve a model — export only its forward pass as a static ONNX graph (tokens in, last-position logits out) and keep the autoregressive loop, sampling, and tokenization in JS, so a small model becomes a static asset that runs client-side with no server.
- [Can you put an obsession on a dial?](https://www.supcpu.com/research/obsession-on-a-dial.md): A char-level model built to compulsively reach for Gatsby's green light — and the $0, fully-controlled ablation that found the dial's real bottleneck: signal loudness, not corpus shape.
- [Can a big model improve a small one?](https://www.supcpu.com/research/improve-a-small-model.md): An LLM-assisted experiment: four rounds took held-out BPC from 2.395 to 1.919. More data was the win; regularization was the dead end.

## Optional

- [daydream-chess-nanogpt-1](https://www.supcpu.com/models/daydream-chess-nanogpt-1.md): daydream GPT, char tokenizer, 2,660,000 params
- [daydream-chess-nanogpt-grand-1](https://www.supcpu.com/models/daydream-chess-nanogpt-grand-1.md): daydream GPT, char tokenizer, 4,730,000 params
- [daydream-chess-nanogpt-micro-1](https://www.supcpu.com/models/daydream-chess-nanogpt-micro-1.md): daydream GPT, char tokenizer, 790,000 params
- [gatsby-nanogpt-1](https://www.supcpu.com/models/gatsby-nanogpt-1.md): gatsby GPT, char tokenizer, 10,650,000 params
- [gatsby-nanogpt-2](https://www.supcpu.com/models/gatsby-nanogpt-2.md): gatsby GPT, char tokenizer, 10,650,000 params
- [glyph-nanogpt-1](https://www.supcpu.com/models/glyph-nanogpt-1.md): glyph GPT, char tokenizer, 47,860,000 params
- [kenosha-kid-nanogpt-1](https://www.supcpu.com/models/kenosha-kid-nanogpt-1.md): kenosha-kid GPT, char tokenizer, 790,000 params
- [kenosha-kid-nanogpt-2](https://www.supcpu.com/models/kenosha-kid-nanogpt-2.md): kenosha-kid GPT, char tokenizer, 792,576 params
- [pona-nanogpt-1](https://www.supcpu.com/models/pona-nanogpt-1.md): pona GPT, word tokenizer, 2,730,000 params
- [shakespeare-nanogpt-1](https://www.supcpu.com/models/shakespeare-nanogpt-1.md): shakespeare GPT, char tokenizer, 10,700,000 params
- [shakespeare-nanogpt-2](https://www.supcpu.com/models/shakespeare-nanogpt-2.md): shakespeare GPT, gpt2-bpe tokenizer, 29,900,000 params
- [shakespeare-nanogpt-3](https://www.supcpu.com/models/shakespeare-nanogpt-3.md): shakespeare GPT, bpe tokenizer, 11,015,040 params