Renormalization: The Fundamental Mechanism of Intelligence

Renormalization

In software design, developers often encounter similar code snippets that address specific needs. Over the years, the collection of these code snippets has become known as "design patterns." It makes sense to question whether Mother Nature also relies on "design patterns" in creating the world that we live in today. In particular, we may wonder which specific design patterns have helped transform humans into a form of intelligent life.

"Renormalization" embodies one such design pattern. Although renormalization has its origin in the obscure field of theoretical physics, its presence in natural phenomena is ubiquitous. It's just that we have not paid enough attention to connect all these phenomena to recognize its presence. This article aims to point out some of these phenomena.

Dewdrops

As temperatures cool throughout the night, moisture starts to condense and deposit on leaves, forming a dense layer of tiny water drops. As the condensation process continues, the original water drops start to touch and coalesce with their neighbors, forming larger and larger dewdrops. By early dawn, often only a few large drops remain on the leaves. This example captures the essential aspects of the renormalization process:

1. Iteration: Smaller entities interact and form successively larger entities.
2. Enhanced contrast: The initial distribution of tiny dewdrops is uniform. However, by the end of the process, only very few places on the leaf contain giant dewdrops, with the rest of the leaf largely dry.

Microphone-loudspeaker audio feedback

When a microphone is placed near a loudspeaker, it produces a high-pitched sound due to audio feedback. As with the formation of dewdrops, audio feedback proceeds in an iterative fashion. The initial sound is uniformly distributed across a wide range of frequencies. Through the renormalization process, the final distribution of acoustic power centers on one single resonance frequency, depriving all other frequencies of such power. That is, the contrast in power distribution is maximized: zero (quiet) everywhere, except for one single frequency, where it overwhelms (loud).

Traffic lights

When traffic lights fail in a busy intersection, they often display blinking red lights. When this happens, the traffic lights are interpreted as equivalent to all-way stop signs, and cars may cross the intersection on a first-come, first-served basis.

After the traffic lights are properly restored, cars may cross the intersection only when the lights in their direction turn green. Effectively, instead of a single car crossing the intersection, we now have packs of cars crossing the intersection. In a sense, individual cars have been "renormalized" into packs of cars. For busy intersections, traffic lights actually help make transportation more efficient, as compared to all-way stop signs.

Superconductivity

Electric charges inside conductors are determined to go straight to their destinations. However, they are constantly bombarded by thermal noise packets known as "phonons." Phonons are adept at knocking electric charges off course, causing electric resistance. For certain materials at low temperatures, the electric charges actually interlock to form larger blocks. One may wonder why these charges can attract each other when they have the same sign of electric charge and, in principle, should repel each other. The answer is that the underlying molecular lattice carries an opposite charge to the moving charges which creates a screening effect and neutralizes the electric charges of the moving particles. In this situation, any other minor source of attraction (such as the statistical attraction of bosons or particles with integer spins) is sufficient to hold the moving particles together.

As the temperature decreases, the moving electric charges may form larger and larger blocks, and charges within the same block will move in unison. If the temperature drops below a critical value, all the moving charges in the conductor may form one single block. Previously, it was easy for phonons to knock a single charge off course. However, if all charges form one single block, it becomes impossible for a phonon to knock a single charge off course, as other nearby charges firmly pull the victim charge together with them in their journey. In other words, there's strength in numbers! Phonons cease to interact with the moving charges, and the latter can now move freely without impediment. Macroscopically, this translates into zero electric resistance.

Giraffe's neck

Many eons ago, giraffes (or more precisely, giraffids) had short necks, as supported by fossil evidence. However, due to their need to find nourishment from taller and taller trees, giraffes evolved and adapted by developing the long necks that are so familiar to us today. That is, renormalization has driven giraffes toward their final shape.

Diamonds

Diamonds are essentially carbon atoms lined up on a crystal (face-centered cubic) lattice. They are formed under the high pressure and temperature of Earth's upper mantle. Aside from carbon, other elements such as oxygen, magnesium, silicon, and iron are also found in the upper mantle. This raises the question as to why carbon atoms would only hook up with other carbon atoms to neatly form crystals of diamonds. The answer is that atoms and molecules are like LEGO blocks of different shapes, and two elements would "hook up" and have strong bonding only if they have matching shapes of holes and protrusions. It just so happens that carbon atoms have the right holes and protrusions to form a face-centered cubic lattice, which enables the renormalization of carbon atoms into diamond crystals.

The initial distribution of carbon atoms and other elements in the upper mantle is essentially random and uniform. However, in a diamond crystal, the presence of carbon atoms is 100%. In other words, the contrast in the density of carbon atoms has been enhanced.

Deep learning and ChatGPT

Modern AI relies on deep neural networks. The input data to deep neural networks often comes from domains of extremely high dimensionality. For instance, a standard HD TV screen has 1920×1080, or roughly two million pixels. An HD-quality movie has a file size ranging from two to four gigabytes. Neither computers nor human brains can digest such a vast amount of data efficiently to make decisions unless a dimensional reduction is performed to bring the amount of information down to a more manageable size.

Renormalization offers just such a mechanism, akin to coalescing fine dewdrops into large dewdrops. The successive layers in deep neural networks implement the iterative steps of renormalization, which engender higher and higher concepts (also known as feature variables). The deep learning models are trained to create the weights of the neural networks, which actually create the conditions to detect the right combinations of "holes and protrusions" as with LEGO blocks. That is, only intermediate concepts with the right holes and protrusions will couple to form higher concepts.

ChatGPT relies on a particular type of deep neural network known as a transformer. The cool thing about transformers is that each word (or more precisely, token) is processed independently, with the effect from other words entering as various types of "shift forces." This is akin to the application of Newton's equations of motion to individual objects in an approach known as a "Free Body Diagram." 

The transformer algorithm employs a large number of different "forces" that are only activated when two tokens have the right match of "holes and protrusions," and lower-level concepts iteratively couple to form higher and higher concepts, very much like how small molecules join to form macro-molecules such as proteins and enzymes. (The technical jargon for these "shift forces" is known as "attention units.")

Final thoughts

Quite a few practical examples of renormalization have been presented here, spanning various domains. However, we have discovered a common pattern in all these examples. In a sense, these examples have the right "holes and protrusions" for us to link them up in this article, so this article itself is performing a renormalization process to obtain a unified view of their shared mechanism. We have found commonality across very different phenomena, much like how Archimedes was inspired by a bathtub to solve the gold purity problem of a crown. The application of renormalization extends far beyond the examples presented here and surely brings profound consequences in other disciplines as well.

Renormalization is ultimately the mechanism responsible for our intelligence and creativity. A proper understanding of the mathematical properties of renormalization allows us to truly explore the mechanisms of deep learning and the human brain. By removing the shroud of mystery from these "black boxes," we can start to operate with full confidence, knowing what we are doing and where we are going.