Karpathy's small-model thesis meets its test: an 8.8B model at 93% of frontier coding

Andrej Karpathy's argument about small models is not that smaller is better. It is that most of what current models spend capacity on is not intelligence.

"A 1B model can be much more effective than most current LLMs," he wrote. "90% of current models waste a ton of capacity and money on remembering stuff that doesn't matter."

Stated plainly, the claim is that a large share of what a modern model learns is memorised text, and that the part which actually reasons is much smaller than the model carrying it. Karpathy has repeated the argument in different forms through the year, and this week an 8.8B open-weight model from webAI gave it a measurable test.

The five claims

Karpathy's position breaks into five parts.

Intelligence was never the knowledge. Strip the memorised internet and keep the reasoning; what remains is a cognitive core, and it is small.

The core might fit in about 1B parameters, maybe less. His phrasing borrows Feynman's title: plenty of room at the bottom. In his framing, the industry has been paying for a library when what it needed was a brain.

Size stopped being the bottleneck. What makes an agent work is the tools it can call and how tight the feedback loop is, not the parameter count.

Intelligence lives in the system, not the weights. A raw model is a CPU with no operating system; a small core inside a good harness beats a giant model on its own.

You can own it. A model on your own hardware is one nobody can rate-limit, price-hike or deprecate.

What the test shows

webAI released TwIL-8B, an 8.79-billion-parameter reasoning model post-trained from IBM's Granite 4.2 8B with full-parameter supervised fine-tuning and GRPO reinforcement learning, then a further length-aware RL stage that rewards shorter reasoning traces.

On its published comparisons, TwIL-8B reaches 93% of the best frontier score on LiveCodeBench coding, 92% on IFBench instruction following, 90% on HMMT November 2025 and 88% on AIME 2026. It scores 98.2 on MATH-500 and 81.6 on LiveCodeBench, within 2.9 to 6.2 points of GPT-5.5, Claude Opus 4.8 and Grok 4.3, and it beats Claude Opus 4.8 on IFBench by 12.2 points.

It also runs locally: 8.79 billion parameters in bfloat16, or a 4.98 GiB Q4_K_M build for a CPU or a modest GPU.

The catch is exactly the one Karpathy's framing predicts. TwIL-8B is not a knowledge model. GPQA Diamond comes in at 70.2, roughly 20 to 23 points behind the frontier; HLE is 12.6, or about 26% of the best frontier score; MMLU-Pro is 76.5. On the hardest contest set, HMMT February 2026, it drops to 69% of frontier.

That split is the argument in miniature. Where a benchmark rewards stored knowledge, the small model loses. Where it rewards reasoning and instruction following, it closes to within a few points at a fraction of the size.

The caveats are real

The model card is unusually candid about what the numbers do not show.

Competitor scores were not re-run; they come from vendor cards and third-party evaluations with their own prompts, sampling and token budgets. AIME and HMMT sets hold 30 to 33 problems, so a single problem is worth about three points and gaps that size sit inside noise. Several of TwIL-8B's own rows were measured on its verbose variant and have not been run on the balanced one. SWE-bench, tool use and agentic tasks were not evaluated at all. The licence is webAI's non-commercial licence, not an open-source one.

And it is verbose: about 19,500 output tokens per AIME 2026 problem, roughly four times what GPT-5.5 spends.

What it changes

Karpathy's fourth and fifth claims are the ones this release actually tests. A model that runs in five gigabytes is a model a team can put on its own hardware, call without a per-token meter, and keep after a vendor changes its lineup. That is the ownership argument, and it does not depend on beating the frontier on every row.

His prediction is that in twelve months the gap between teams shipping on-device and teams paying per-token frontier bills will not be about capability. If TwIL-8B is the shape of what comes next, the split will show up where the model card already shows it: reasoning travels well at small size, and memory does not.

Source: Andrej Karpathy; webAI TwIL-8B model card on Hugging Face.

Contact: [email protected]