Large Language Models: A New Moore's Law?

A newly released analysis on AI and compute says that since 2012, the amount of compute used in the largest AI training runs has increased exponentially, with a 3.4-month doubling time. For comparison, Moore’s Law had a 2-year doubling period. The analysis frames this trend through large models and large language models, raising the question of whether the rapid growth of AI training compute represents a new Moore’s Law.

For readers less familiar with the terminology, compute in this context usually refers to the aggregate computational effort required to train a machine-learning model, often measured through a combination of hardware capability, the number of operations performed, and the time spent executing those operations. Large models are systems designed with many parameters and trained on broad, high-volume data, while large language models are a prominent category of such systems that learn statistical patterns in text in order to generate, summarize, translate, or reason over language. As these models become larger and more capable, the training process can demand substantially more compute than earlier approaches, especially when researchers seek to improve performance through scale, longer training runs, or more complex architectures.

The comparison to Moore’s Law is meaningful because Moore’s Law was historically used as a shorthand for a sustained pattern of improvement in computing resources over time. If AI training compute were to follow a similarly steady doubling rhythm, it would suggest that progress in the field is not merely a sequence of isolated breakthroughs, but part of a broader scaling dynamic. The shorter doubling period described in the analysis implies an especially rapid expansion of demand for specialized hardware, energy infrastructure, data processing pipelines, and research organization.

This trend also has practical implications. Training very large models can concentrate resources in settings where substantial capital and infrastructure are available, which may shape who can develop frontier systems and how they are deployed. At the same time, faster growth in compute demand may encourage efforts to improve efficiency, such as more effective model architectures, better parallelization, quantization, sparsity, or other techniques that aim to obtain capability from fewer resources. The analysis therefore highlights not only how much compute is being used, but also how the pace of scaling could influence the future economics and engineering of AI research.

In summary, the central point is that AI training compute appears to be expanding at a rate that invites comparison with a foundational idea from computing history. Whether this pattern will persist, flatten, or evolve remains an open question, but the framing helps readers understand why growth in model scale and training effort is often discussed as a defining feature of the development of large AI systems.