Scaling Laws

Scaling laws are the empirical finding that a model's performance improves in a smooth, predictable way as you increase three things together: parameters, training data, and compute. Plotted on log axes, loss falls in a near-straight line, which lets labs forecast how good a bigger model will be before training it, and decide how to spend a fixed compute budget. The influential "Chinchilla" result showed many early models were oversized and undertrained: for a given compute budget you often want a smaller model trained on far more data. SaaS builders will never fit these curves themselves, but scaling laws explain the industry you're buying into — why capabilities keep climbing on schedule, why "just make it bigger" worked for years, and why the frontier is now shifting toward data quality and test-time compute as raw scaling hits practical limits. The takeaway for planning: assume the model tier you use will get cheaper and more capable on a predictable cadence, and design so you can swap it in without a rewrite.