Scaling laws
The empirical finding that a model's loss falls predictably as you add compute, data, and parameters — turning "make it bigger" from a hunch into a forecastable curve.
Scaling laws are the observation that, across orders of magnitude, a language model's error drops smoothly and predictably with more compute, more data, and more parameters. Because the curve is regular, you can forecast how good a bigger model will be before training it — which is why labs pour capital into scale. The open question is where, or whether, the curve finally bends.
Scaling laws reframed AI progress as an engineering forecast rather than a series of lucky architectures. If loss is a predictable function of compute, the rational move is to buy more compute — which is much of what the frontier labs have done.
The debate they fuel: optimists read the unbroken curve as a runway to far more capable systems; sceptics argue the benchmarks that keep improving aren't the ones that matter, and that something other than scale is missing. Both camps are watching the same graph.
Where it came from
In megatrends
Related players
Related concepts
Finds citing this concept
Overall = breadth + depth + substance + ½·novelty + freshness (a small boost that halves each week)
nanochat
Andrej Karpathy packed the whole ChatGPT recipe — from raw text to a chatbot you talk to in a browser — into one readable codebase that trains for about the cost of a nice dinner.
The Jevons bill comes due
AI inference keeps getting cheaper per token, yet the bills are climbing fast — because every price cut invites buyers to spend the savings on more tokens, and then some.
The terafab ceiling
At his GTC keynote, Jensen Huang said NVIDIA's order book for its next two chip generations will hit $1 trillion through 2027 — double last year's figure — and that the only thing capping it now is how fast fabs can be built.
A flagship on a gaming card
Alibaba's new open-weight model fires only 3 billion parameters per word it writes, yet matches last year's flagship that fired more than seven times as many — and it runs on an $800 gaming card.
How this connects
Tap a node to open it