Superfact 127: The Intel 4004 processor chip created in 1971 had 2,308 transistors. Modern cutting-edge processors and AI accelerators exceed 50 billion to over 200 billion transistors, which is 25 million to nearly 100 million times as many transistors per chip. This follows Moore’s law, which states that the number of transistors on a microchip doubles roughly every two years, while the cost of computers drops. As a result, computer chips have gotten millions of times faster and the cost of memory (RAM) has dropped by four trillion times. This has made the modern AI / LLM possible.
The transistor below is a large transistor that you can hold between your fingers. Transistors inside a computer chip are very small. They are microscopic. In the diagram, whenever a small voltage ( typically 0.7 volts) is applied to the base (B) it allows a much larger current to flow through the main terminals from the collector (C) to the emitter (E). Since electrons are negative this means that the electrons are flowing from the emitter to the collector. If there is no voltage or current at the base there is no current flowing from the collector to the emitter either. This is how you control the transistor and achieve more complex behavior if you have many connected transistors. Also, note that the transistor is a one way street.
Moore’s law, which states that the number of transistors on a microchip doubles roughly every two years, is not a physical law, it is a purely observational law that has held for well over half a century. It is just a reflection of how good we are at improving processors. No one knows whether Moore’s law will continue to hold but if it does, then today’s computers and today’s AI are very bad compared to tomorrows computers and AI, and they are super bad compared to what we will have 20 years from now.
The diagram below shows 100+ microprocessors (CPUs), their number of processors, and the year they were introduced going from 1970 to 2020. The microprocessor data comes from this list. Intel 4004 is located in the lower left corner of the diagram and as mentioned it has 2,308 transistors. The AMD Epyc Rome processor in the upper right corner has 39.54 billion processors. Notice that the number of processors is on a logarithmic scale (not 1,2,3,4,5….but 1,10,100,1000,10000, etc.) A curve that grows exponentially becomes like a line in such a diagram. This is another way of saying that the technological progress for microprocessors is exponential. Every six and a half years the number of transistors is 10-doubled.
However, the processor that currently has the most transistors is the Cerebras Wafer Scale Engine 3 (WSE-3) with 4 trillion transistors. It is not in the diagram. A comparison between Intel 4004 and Cerebras Wafer Scale Engine 3 (WSE-3) may not be entirely fair since it is a wafer sized CPU specifically made for AI and much bigger than Intel 4004. However, the standard consumer or workstation CPU with currently the most transistors is Apple’s M3 Ultra (normal sized) with 184 billion transistors. However, 184 billion transistors that is more than 79 million times as many transistors as the Intel 4004.
This is a super fact because it is an important fact that is a surprise to you if you did not know about Moore’s law and mind-blowing even if you did.
Computational Capacity / Speed
One benefit of transistors and other electronic components becoming much smaller is increased speed. The graph below shows that the fastest supercomputers in 1993 had a processing speed of 124 Gigaflops and in 2025 a processing speed of 1.81 billion Gigaflops, which is 14.6 million times faster. One gigaflop equals one billion floating-point operations per second. Multiplying 3.1415926536 times 2.7182818284 = 8.5397342225 is an example of a floating point operation. Note that 1.81 billion Gigaflops is 1.81 quintillion floating-point operations per second.
Cost of Memory
Much smaller and faster and transistors and other kinds of electronic components also come with additional benefits such as an extreme reduction in the price of memory, including the extreme reduction in the price of random access memory, RAM. In the diagram below (taken from this page ) the blue line shows that the price for one Terabyte of memory (RAM) in 1957 was 3,79 quadrillion dollars and now in 2023 it is only 1,088 dollars. 3,79 quadrillion dollars is a lot of money.
You may object that and point out that 3,79 quadrillion dollars is even more than our national deficit. So how can that be possible? The answer is that one terabyte of memory did not exist in 1957. 3,79 quadrillion dollars per terabyte is the same as 3.79 million dollars per kilobyte, but those are the numbers that make sense for 1957. The gigantic computers back then only had a few kilobytes of memory. You just need to do the conversion. Anyway, 3,79 quadrillion dollars versus 1,088 dollars corresponds to a reduction in price by 3,48 trillion times. For disk memory the price per terabyte went down from 87.5 billion in 1956 to 11 dollars in 2023, a reduction of 7.95 billion times.
Increased Computational Power and Artificial Intelligence
My super fact 88 states that “Artificial Intelligence is Not New” It goes back to at least 1943 with the creation of the first artificial neural network model, the first trainable (able to learn) neural network in 1957, and the foundation of the field of “Artificial Intelligence Research” in 1956. In 1986 a landmark paper was published by David Rumelhart, Geoffrey Hinton, and Ronald Williams which introduced the Rumelhart backpropagation algorithm, which is used today by modern AI and Large Language Models, such as ChatGPT. Geoffrey Hinton, who is said to be the father of Artificial Intelligence, received the Nobel Prize in physics in 2024. David Rumelhart and Ronald Williams were both dead and could therefore not receive the Nobel Prize.
So why did it take so long for today’s commercial LLMs to appear? The answer is, for the most part, that the computational power and huge memory required did not exist until recently. Sure, training these large AI systems require enormous amounts of data that comes from outside of the processors (internet) but you need enormous amounts of memory to store this data and an enormous computational processing power to run the algorithms such as Rumelhart’s backpropagation algorithm. For example, ChatGPT 3.5 and 4.0 feature 96 versus 120 hidden layers of neurons with hundreds of billions and trillions of parameters/neurons in total. The enormous increase in computer processing power and memory is an essential aspect of scaling up AI. Our World in Data has an article about this here.
Conclusion
Moore’s law, a purely observational law, states that the number of transistors on a microchip doubles roughly every two years. This also means that the computational processing power of processors has greatly increased and the cost of memory has dropped by a lot. This in turn has made the existence of modern Large Language Models such as ChatGPT possible.
Transistors per Microchip Increased by More than 50 million Times in 50 Years. Processing speed has increased 14.6 million times in 32 years. The reduction in price per Tera Byte has gone down 3,48 trillion times in 67 years.
If Moore’s law holds, we can expect that in twenty years the number of transistors per Microchip will increase by more than 1,000 times, that processing speed will increase roughly 29,000 times, and that memory RAM will be more than 6,000 times cheaper. These are my calculations based on the graphs above.
