nanochat

Rent a single eight-GPU machine, run one script, and a little under four hours and $92.40 later you have trained a ChatGPT clone you can open in a browser and talk to. That is the speedrun Karpathy clocked for nanochat: a self-contained repo of roughly 8,300 lines that walks through the entire modern language-model pipeline — building the tokenizer, pretraining on web text, fine-tuning it to chat, an optional round of reinforcement learning, and the inference server and web UI to use it.

"Our model isn't even sure about the color of the sky so we're probably safe on the biohazard side of things for now." — Andrej Karpathy

His earlier teaching repo, nanoGPT, only covered the first step. nanochat is the first to carry a reader all the way to a working assistant, and the point is less the model than the readability: the whole thing is meant to be forked and understood, not configured. The price is the other half of the story. Training a comparable model — GPT-2 — cost around $43,000 in 2019; the same shape of thing now fits on a single afternoon's GPU rental.

The catch, stated plainly by its author, is that $100 buys a deliberately feeble model — it scores under five percent on grade-school math and gets basic facts wrong. The achievement is not the chatbot but the map: the complete path from text to talkable model, cheap enough that any student can run the whole thing once and watch every stage happen.