The New Scaling Law

Max Cook

The next scaling law: learning velocity "on the job".

AI has been defined by pre-training: spend more upfront, get a smarter model. But a shift is underway, from pre-training to continual, on-the-job learning.

A new study measured whether models improve when given time, feedback, and room to work. The data shows the newest models learn on the job almost twice as fast as models from just three months earlier, and the gains compound with iteration.

It turns out models may not be so different from us: given room to practice, they get better.

For more on this C:\Take, watch Max:

The New Scaling Law

Latest Takes