AI Parameters Increased Billions of Times

Super Fact 148 : The number of parameters in AI systems has increased by tens of billions of times since the 1950’s. Transistors per Microchip have increased by more than 50 million Times in 50 Years. The size of AI training data sets increased trillions of times. The number of computations for training has increased hundreds of sextillion times. The number of parameters, transistors per microchips, and computations for training are growing exponentially. This is one reason many AI experts believe there is a real chance that human-level artificial intelligence will be developed within the next decades.

The y-axis of this graph shows the number of parameters, and the x-axis is the timeline starting in 1950 and ending in 2025. The graph shows a straight line corresponding to an increase of several thousand times from 1950 to 2010 and then a much more rapid rate from 2010 and on.
Exponential growth of parameters in notable AI systems. Parameters are variables in an AI system whose values are adjusted to establish how input data gets transformed into the desired output; for example, the connection weights in an artificial network. Data Source: Epoch AI (2026). Note: estimates are based on AI literature with uncertainty up to a factor of 10. OurWorldinData.org/artificial-intelligence | CC BY

The Number of Parameters above are plotted on a logarithmic axis. This means that exponential growth becomes a straight line and each tick-step is a 100 doubling. The regression lines show a rise in parameters since 2010, driven by the success of deep learning methods that leverage neural networks and massive datasets.

Theseus an AI system from 1950 had 40 parameters. The perceptron an early neural network (Artificial Intelligence) built in 1960 had 1,000 parameters. Deep Blue, the AI supercomputer that beat chess world champion Garry Kasparov in 1997 had 8,000 parameters. It could evaluate roughly 200 million chess positions per second, which is hard to beat. In 2012 AlexNet a neural network with multiple hidden layers, so called deep learning, had 60 million parameters. Switch from built in 2021 has 1.57 trillion parameters and GPT-4 (recent ChatGPT) has 1.8 trillion parameters. The number of parameters from Theseus to GPT-4 increased 45 billion times.

This diagram shows similar data to the diagram above and it is also a logarithmic scale like the diagram above. However, it has more data points and no lines to fit the data.
Parameters in notable artificial intelligence systems. Parameters are variables in an AI system whose values are adjusted to establish how input data gets transformed into the desired output; for example, the connection weights in an artificial network. Data Source: Epoch AI (2026). Note: estimates are based on AI literature with uncertainty up to a factor of 10. OurWorldinData.org/artificial-intelligence | CC BY

The exponential growth in the number of computations used for training is even more impressive. The exponential growth took a sharp upturn in 2010. As can be seen in the graph below the computation used to train notable AI systems has doubled every 6 months since 2010. This means a doubling by million times in 10 years, a doubling by a trillion times in 20 years, a doubling by a quintillion times in 30 years, a doubling by a septillion times in 40 years, and a doubling of nonillion times in 50 years. Naturally, we don’t know if the rapid exponential growth will continue but also don’t forget Moore’s Law. You can read more about the rapid exponential growth in training computation here.

The graph shows the number computations of training on the y-axis and the time starting in 1950 on the x-axis.
The computation used to train notable AI systems has doubled every 6 months since 2010. Training computation is measured in total floating point operations (FLOP). Each FLOP represents a single arithmetic calculation, such as multiplication. Shown on a logarithmic scale. Data source: Epoch (2024) CC BY

As mentioned, the rapid exponential growth in parameters, transistors per chip, computations, etc., is one reason many AI experts believe there is a real chance that human-level artificial intelligence will be developed within the next decades, and some believe that it will exist much sooner. You can explore more on this topic here.

I consider this a super fact because it is important information and the exponential growth of parameters in AI, computations and the size of training sets, is shocking and should surprise many of us.

What Are AI Parameters ?

Parameters are variables in an AI system whose values are adjusted to establish how input data gets transformed into the desired output; for example, the connection weights in an artificial network and the biases associated with each neuron. Biases are offset values added to the weighted sum at each neuron to help the model fit the data better. Below is a very simple illustration of a small neural network (drawn by me) with three hidden layers (multiple hidden layers is called deep learning), an input and output layer. There are 19 neurons and 64 connections with weights yielding a total of 83 parameters.

Three blue circles connected to four yellow circles connected to four green circles connected to six blue circles connected to two red circles all via lines representing weights.
My drawing of a deep learning neural network (deep learning AI). There are three hidden layers (yellow, green and light blue). There are 19 neurons and 64 connections with weights yielding a total of 83 parameters.

I should add that in other types of AI systems parameters might be something different. For example, at work I used an AI algorithm called C5, a type of decision tree algorithm, and in that algorithm the tree parameters controlled how the model split the data. You listed most of the parameters you wanted in a text file.

The human brain has an estimated 86 billion neurons (plus/minus around 15 billion) The human brain also has an estimated 100 trillion to 1,000 trillion synaptic connections. That is a lot more than the estimated 1.8 trillion parameters of GPT 4. However, keep in mind 1.8 trillion will pretty soon be 100’s of trillions, or quadrillions of parameters and then quintillions of parameters, then sextillions of parameters, etc.

The Rummelhart Backpropagation Algorithm for Training

There are many kinds of neural networks, but modern Large Language Models are feedforward neural networks using backpropagation for training, specifically modernized versions of the 1986 Rumelhart, Hinton, and Williams algorithm. In 1985/1986, Geoffrey Hinton, David Rumelhart and Ronald J. Williams presented an algorithm applied to a neural network featuring a hidden layer that was very successful. It was effective and guaranteed to learn patterns that were possible to learn. It set off a revolution in Neural Networks.  Geoffrey Hinton received the Nobel Prize in physics in 2024 for his inventions enabling machine learning with artificial neural networks. David Rumelhart and Ronald J. Williams were dead and could therefore not receive the Nobel prize. You can read more about backpropagation here and read the original Rumelhart backpropagation paper here..

I can add that the next year, in 1987, when I was a college student, I used this algorithm on a neural network featuring a hidden layer to do simple OCR (optical character recognition). Note that a computer reading an image with a letter is very different from someone typing it on a keyboard. In the case of the image, you must use OCR, a complicated and smart algorithm for the computer to know which letter it is.

Other Super Facts Related to Artificial Intelligence

The diagram shows a deep learning neural network featuring hidden layers, a couple of input/output layers, and large computer.
The diagram shows a deep learning neural network featuring hidden layers, a couple of input/output layers, and a large computer. The dots in the diagram are neurons. With permission from kwholley63.



To see the other Super Facts click here

I would also like to mention that today November 15 is “I Love to Write Day”

Global Warming Causes Sea Level Rise – Culprit is not Sea Ice

Superfact 147: Global Warming causes sea level rise via Thermal Expansion and the Melting of Land Ice and to a lesser degree from changes in land water storage, such as groundwater pumping and dam construction. However, almost none of the sea level rise is due to the Melting of Sea Ice.

This graph from NASA shows a global rise in the sea level of 10.1 centimeters in 11 years (1993 to 2024). The graph is jagged and not smooth. | Global Warming Causes Sea Level Rise - Culprit is not Sea Ice
This graph shows global mean sea level (in blue) since 1993 as measured by a series of five satellites. The solid red line indicates the trajectory of this increase, which has more than doubled over the past three decades. The dotted red line projects future sea level rise. Credit: NASA/JPL-Caltech

As ocean water warms, it expands, which also raises sea level. Water expands with the temperature as long the temperature is above 4 Celsius. This is referred to as thermal expansion. In recent years, thermal expansion accounts for about 30% to 33% of global sea level rise. The melting of land ice (glaciers, Greenland, Antarctic land ice) accounts for roughly 60% to 66% and 3% to 10% of global sea level rise comes from changes in land water storage, such as groundwater pumping and dam construction. You can read more here and here.

The graph shows an accelerating increase in global sea level rise. Since 1880 the global sea level has increased by 8-9 inches.
This graph is taken from climate.gov This graph goes from 1880 to 2024 and up to 1993 it is based on analysis. The satellite records begin in 1993.

A very small amount, less than 2%, comes from the melting of sea ice (floating Arctic sea ice or ice shelves). Sea ice melt itself does not cause sea levels to rise because the ice is already in the water. When it melts, it displaces the same volume of water it already took up. This is Archimedes principle. It does not add new mass to the ocean. On the other hand, the melting of ice on a land surface, like Greenland, does increase the sea level because new water is flowing into the ocean. It should be noted that the minor contribution of global sea level rise that comes from sea ice stem from the fact that sea ice is mostly fresh water while sea water is saltwater and fresh water and saltwater have different densities. Click here for an explanation.

Global Warming Causes Sea Level Rise - Culprit is not Sea Ice
Ice melting in a glass of water does not change the density, while ice melting in the ocean decreases the density of the salt water, raising the sea level slightly. Note: magnitude has been exaggerated for artistic effect. Credit NASA taken from this page.

I consider this a super fact because it is an important fact surrounded by a lot of confusion and surprises. Many people do not know that sea ice contributes almost nothing to sea level rise and would not have contributed any at all if it weren’t for the different salt content of sea ice and sea water. However, the melting of the arctic is a big problem for other reasons.

Other people, often from the climate denier camp, think that they are the only ones who understand Archimedes principle and that sea ice not contributing to sea level rise somehow refutes what climate scientists (or activists) say (it doesn’t of course). Yet others who understand this better still do not know that sea ice does contribute to global sea level rise even though it is very little.

But Global Sea Levels Have Changed Much More in the Past

It may seem like a sea level rise significantly less than a meter cannot be that bad. After all the sea level has changed by much more in the past, even 100’s of meters. However, there are a few things to keep in mind.

First this is happening in an extremely short time. 100 years is a 100 times shorter time interval than 10,000 years. Secondly there are 8 billion people on the planet and hundreds of millions of people are living in areas that are very close to the sea level. Our civilization developed during a period of unusually stable climate and relatively stable sea levels. The CoastalDEM study indicates that up to 630 million people live on land below projected annual flood levels for 2100. In addition, a lot of that land is used for food production. There is also erosion, habitat loss, storms having greater impacts, etc. So, sea level rise is a significant problem.

Global Warming is Chiefly Caused by Burning Fossil Fuels

Another thing to keep in mind is that not only do we know that global warming is happening and that it is happening very fast. We also know that it is caused by us primarily as a result of our burning of fossil fuels. There is a long-standing scientific consensus on these two facts because the evidence is conclusive. We also know that global warming and global sea level rise will continue long after we have halted our carbon emissions.

The graph shows the temperature since the last ice age. There is a maximum temperature about 6,000 years ago, then a cooling and after that there is a sharp uptick in temperature during the last 100 years.
Global temperature from 20,000 BCE until now. Notice the relatively stable temperature during the last 10,000 years, coinciding with the development of human civilization. Also, notice the slight cooling followed by the very sharp increase in average global temperature at the end.

The sharp spike in the global average temperatures towards the right side of the graph above is caused by us humans, chiefly by our burning of fossil fuels. A very common objection to this claim is “Climate has always changed”, or “I think it is a natural cycle”,  etc. Such statements dismiss out of hand the expertise of the scientists who are the experts on past climate and natural causes for climate change. They are nearly unanimously stating that natural causes cannot explain the current rapid global warming, and it is obvious why if you look at the evidence.

To quote Katherine Hayhoe, a well-known climate scientist : “Hi, I’m a climate scientist. You may know me from my greatest hits including, “No, it’s not a natural cycle,” “Yes, I know it’s been warmer before (and the only reason YOU know this is because we scientists told you so)”

Other super facts related to global warming




To see the other Super Facts click here

Engineering Photosynthesis to Save the Planet

It was a great pleasure to read the article by retired scientist, pathologist, and blogger, Craigavad miscellany, “Can We Engineer Photosynthesis to Save the Planet?” Note, he has written 240 peer reviewed papers.

The article describes how the evolutionary invention of Photosynthesis transformed the planet. He states that “atmospheric CO₂ continues to rise because of our uncontrolled use of hydrocarbon fuels.” He asked a very interesting question. “Could we use photosynthesis to remove enough carbon dioxide from the atmosphere to make a significant difference to climate change?” The article is very professionally written and full of interesting information and I urge you to read it.

The Unfolding Driverless Car Revolution

Super Fact 146 : Research on self-driving and driverless cars has been going on for many decades but only during the last few years have driverless cars began to show up on our roads and they are quickly becoming more common.

Waymo One Jaguar driverless autonomous car equipped with lidar sensors and cameras drives on a multi-lane city road. Front side view. Selective focus - San Francisco, California, USA - March 8, 2025 | The Unfolding Driverless Car Revolution
Waymo One Jaguar driverless or autonomous car. Shutterstock asset id:  2597125675 by Michael Vi

When I was working on my PhD in Applied Physics & Electrical Engineering, specialty Robotics, in the 1990’s I used Artificial Intelligence algorithms, and I also read research papers on self-driving cars and autonomous cars, and I watched demonstrations of self-driving cars on videos. Self-driving and even autonomous cars existed but they weren’t driving around on public roads. It is not until the last few years that driverless cars have become common on our roads. I’ve recently seen several Waymo cars driving around in traffic here in Dallas.

Note, a fully autonomous or driverless car can drive without any human supervision whilst a self-driving car still requires some human supervision. In this post I am referring to both for the same purpose. Self-driving cars and driverless cars are both gaining a lot of traction.

When we took our daughter to the car dealership last month to buy her a car there were many cars that came with a self-parking option and that included parallel parking. You drove up to the open space and pressed a button, and the car parked itself perfectly in seconds. That is not really what we mean by a driverless car, but it shows how the technology is changing. When we bought our last car a few years ago that option did not exist. As the graphs below show, during the last few years driverless cars are rapidly becoming common.

Total monthly distance traveled by passengers in California’s driverless taxis. Monthly passenger-kilometers traveled in California’s paid driverless taxis, summing the kilometers actually traveled by individual passengers across all trips. Data source: California Public Utilities Commission (2026) – Learn more about this data – OurWorldinData.org/artificial-intelligence | CC BY
Global Autonomous Vehicles Market Revenue in USD billions. This graph is from this site.

Technology may be developing slowly and then suddenly everything happens very quickly. I remember people complaining in the early 1990’s about how Artificial Intelligence was making such slow and meager progress and that it was a disappointment. It is true that for decades it was just for engineers and scientists. Then a few years ago the powerful Large Language Models showed up in everyone’s browsers. Electrical Vehicles were rare and restricted just a bit more than a decade ago but now they are replacing gasoline cars around the world. See my post “EVs are Replacing the Internal Combustion Engine”. In Norway, 98% of all new cars sold are Electrical Vehicles. Clean energy is similarly beginning to replace fossil fuels, see “The Unfolding Clean Energy Revolution”.

I believe that the rapid adoption of autonomous cars comes as a surprise to many and therefore I consider this a super fact.

SAE Classification

The SAE classification defines the level of automation for autonomous cars.

  • Level 0: No automation. The human does all the driving.
  • Level 1: The human controls most of the driving tasks, must be fully attentive, but get some assistance from the car. Examples are lane-keep assistance and cruise control.
  • Level 2: The human must supervise the system and keep the hands nearby the steering wheel. Self-parking is considered level 2.
  • Level 3: The human can take their eyes off the road under certain conditions but must remain in standby.
  • Level 4: High automation. The vehicle drives itself completely but is restricted to a specific weather or route condition.
  • Level 5:  Full automation. Passengers are purely riders and no steering wheel, pedals or driver seat is needed.

You can read more here or here

A Brief History of Autonomous Cars

The history of driverless cars goes back 100 years. In 1925, a radio-controlled car named the “American Wonder” was driven through New York City traffic. In the 1950’s there was serious scientific research on driverless cars. In 1977 Japan’s Tsukuba Mechanical Engineering Laboratory built the first autonomous car capable of independently reading painted road lines without external wires. In January 2014, Induct Technology’s Navia shuttle became the first self-driving vehicle available for commercial sale. In 2015–2017 early commercial driverless taxi and robotaxi services (such as Waymo’s early rides) began rolling out on public roads. The last few years autonomous vehicles have become increasingly common around the world.

Other Super Facts Related to Artificial Intelligence

The diagram shows a deep learning neural network featuring hidden layers, a couple of input/output layers, and large computer. | The Unfolding Driverless Car Revolution
The diagram shows a deep learning neural network featuring hidden layers, a couple of input/output layers, and a large computer. The dots in the diagram are neurons. With permission from kwholley63.



To see the other Super Facts click here

Texas the Snake State

Esther’s writing prompt: November 4th: Snake

Click here to join in

For Esther’s prompt today I am posting about a Texas Snake book that I like. I am also adding some additional Texas snake information. There are 105 species of snake in Texas. More than any other state. 16 of them are venomous. Below is a picture of four venomous Texas snakes.

The four snakes are on a beige sofa. | Texas the Snake State
Western Diamond Back, Copperhead, Cottonmouth, Texas Coral Snake. I generated this picture with the help of ChatGPT.

The Western Diamond Back Rattle Snake is a very common large Rattle Snake. The Copperhead is another common venomous snake. The Cottonmouth is a common venomous snake which is white inside its mouth. It is also called the Water Moccasin because it is a semi-aquatic snake that hunts fish and frogs. The colorful snake on the right, the Texas Coral Snake, is very venomous but luckily not very aggressive. A bite from the Texas Coral Snake has enough venom to kill seven adult men. There is a harmless snake, the Milk Snake, which has similar colors to the Texas Coral Snake. Since both Milk Snakes and the Texas Coral Snake have similar coloring, how do you tell them apart? This is the rule you follow:

Red touches yellow kill a fellow red touches black friend of jack

Can you see this in the picture below? The snake on the left you can hold and cuddle. The snake on the right can kill you.

The Milk Snake on the left is red, black and yellow with the black bands separating red and yellow. The coral snake on the right is also red, black and yellow but with the yellow bands separating black and red.
Milk snake and Coral snake. I generated this picture with the help of ChatGPT.

The Green Mamba Between my Feet

When our kids were young, we frequently visited Dinosaur Valley State Park south of Glen Rose Texas. It is famous for its fossilized dinosaur footprints. As I was walking around with my four year old son (our younger son) on my shoulders, he suddenly called out saying “Dad you almost stepped on that Green Mamba”. I looked down and between my feet there was a Green Grass Snake. It did not look like a Green Mamba, so we were safe. However, I am still glad I did not step on it.

Green curled up small snake.
Rough green snake (Opheodrys aestivus) MH Herpetology, CC BY-SA 4.0, via Wikimedia Commons.

The Whip Snake Inside my Shirt

When our daughter was young, she was part of the scout like organization called Adventure Princesses. Sometimes the Adventure Princesses went camping with their dads. On one occasion we visited a state park that had a building featuring lots of different kinds of Texas snakes. None of the snakes were venomous and the dads were allowed to pick them up and show them to the girls.

My daughter and her friends were sitting on chairs in a classroom like setting. I made the mistake of picking up a coachwhip snake or a so called whip snake. These snakes are long and thin but very muscular. They can easily raise their head above half their body. The whip snake was wild, and it snuck itself under my long shirt sleeve and started crawling up my arm. It was much stronger than I anticipated so I struggled with the snake before I could pull it out of my shirt sleeve. The girls were very interested in the spectacle. However, nothing bad happened. The snake was just very good at slithering where I did not want it to.

Snake crawling inside a man’s shirt sleeve. | Texas the Snake State
Coachwhip going inside my shirt sleeve.
A long curled up dark brown and black snake.
Eastern coachwhip (Masticophis flagellum flagellum) MH Herpetology, CC BY-SA 4.0, via Wikimedia Commons

The Rat Snake in the Sink

This happened more recently. We were visiting my wife’s parents when my wife went to the bathroom. As she was washing her hands, she noticed that a piece of what she thought was the faucet moved. She realized that there was a more than a yard long snake right behind the faucet. She had washed her hands just an inch away from the snake.

It turned out to be a harmless rat snake. However, since the Rat snake looks a bit like a Cottonmouth / Water Moccasin, we were afraid of handling it and we called animal control. I can add that we had also read that Water Moccasin Snakes could crawl through sewage pipes and come up through the toilet and bite your behind. As turned out, that is just a common myth.

Big black snake in the grass looking into the camera.
Central ratsnake, Pantherophis alleghaniensis. Stephen Lody Photography, CC BY-SA 3.0, via Wikimedia Commons.

Now finally, the review of the book.

Lone Star Field Guide to Texas Snakes (Lone Star Guides) by Alan Tennant

Below I am listing the two editions of this book. I bought and read the paperback edition.

  • Paperback –  Publisher : Random House Large Print (November 13, 2018), ISBN-10 : 1984887262, ISBN-13 : 978-1984887269, 352 pages, item weight : 10 ounces, dimensions : 5.3 x 0.82 x 7.8 inches. It costs $27.00 on US Amazon. Click here to order it from Amazon.com.
  • Kindle –  Publisher : Taylor Trade Publishing (January 27, 2006), ASIN : B00IUFI1K8, ISBN-13 : 978-1461635642, 658 pages, it costs $10.99 on US Amazon. Click here to order it from Amazon.com.
The front cover of the book features the title and author of the book and picture of the Gray Banded King Snake. | The front cover of Lone Star Field Guide to Texas Snakes (Lone Star Guides) by Alan Tennant.
The front cover of Lone Star Field Guide to Texas Snakes (Lone Star Guides) by Alan Tennant. Click on the image to go to the Amazon page for the paperback edition of the book.

Amazon’s Description of the Book

The State of Texas not only boasts the largest snakes in North America, but also the largest number of species—105. This guide describes in detail each and every one of them, from the benign Texas long-nosed snake, to the venomous Western Cottonmouth. Facts on the biology and behavior are given, plus the latest findings on abundance, reproduction, prey, sizes, and habitat. In addition, introductory chapters describe the physiology and diet of snakes, and an all-important section on “Venom and Evenomation” debunks the many myths surrounding what to do when bit by a venomous snake. 128 color plates of species and subspecies aid in identification.

This is my five star review for the book Lone Star Field Guide to Texas Snakes

You almost stepped on that green mamba

“Dad, you almost stepped on that Green Mamba”. That is what my 4-year old (sitting on my shoulders) told me when we were walking in Dinosaur Valley state park. Using the second edition of this book I was able to identify the snake as the “Rough Green Snake” when I got home. I am glad there are no Green Mambas in Texas (in the wild), but there are still some very poisonous snakes to watch out for. Seven species of rattle snake, two species of Massasauga, and a pigmy rattle snake, cottonmouth (water moccasin), three species of copperhead, and the Texas Coral snake. The Texas Coral snake is the most poisonous of the snakes in Texas (but it is not aggressive) and the poison from the Mojave Rattle Snake can kill six adults (lives only in West Texas). There are 105 species of snakes all in all in Texas.

This edition (third edition) has essentially the same pictures and information as the second edition, but the third edition is organized better, is a little bit bigger, and more updated. What I really like about this book is that for each of the snakes it has a map of Texas showing the distribution of the snake. The book is also well organized and the photos are good. It is also an interesting book to read, it contains a lot of facts about the various snakes. Once you start reading about the various snakes, and looking at the pictures and the distribution maps, you just can’t put it down. The book tickles your curiosity. Compared to other Texas snake books that I have seen this one is the most interesting and the most well organized of them all. If you quickly need to identify a snake this is the right book.

The back cover of Lone Star Field Guide to Texas Snakes (Lone Star Guides) by Alan Tennant.
The back cover of Lone Star Field Guide to Texas Snakes (Lone Star Guides) by Alan Tennant. Click on the image to go to the Amazon page for the Kindle edition of the book.

My Other Responses to Esther’s Prompts




To see the Super Facts click here