When I started at UC Berkeley in January 1986, the first thing I did was compile a history of theories of intelligence and brain function. I read hundreds of papers by anatomists, physiologists, philosophers, linguists, computer scientists, and psychologists. Numerous people from many fields had written extensively about thinking and intelligence. Each field had its own set of journals and each used its own terminology. I found their descriptions inconsistent and incomplete. Linguists talked of intelligence in terms such as “syntax” and “semantics.” To them, the brain and intelligence was all about language. Vision scientists referred to 2D, 2½D, and 3D sketches. To them, the brain and intelligence was all about visual pattern recognition. Computer scientists talked of schemas and frames, new terms they made up to represent knowledge. None of these people talked about the structure of the brain and how it would implement any of their theories. On the other hand, anatomists and neurophysiologists wrote extensively about the structure of the brain and how neurons behave, but they mostly avoided any attempt at large-scale theory. It was difficult and frustrating trying to make sense of these various approaches and the mountain of experimental data that accompanied them.

Around this time, a new and promising approach to thinking about intelligent machines burst onto the scene. Neural networks had been around since the late 1960s in one form or another, but neural networks and the AI movement were competitors, for both the dollars and the mind share of the agencies that fund research. AI, the 800-pound gorilla in those days, actively squelched neural network research. Neural network researchers were essentially blacklisted from getting funding for several years. A few people continued to think about them though, and in the mid-1980s their day in the sun had finally arrived. It is hard to know exactly why there was a sudden interest in neural networks, but undoubtedly one contributing factor was the continuing failure of artificial intelligence. People were casting about for alternatives to AI and found one in artificial neural networks.

Neural networks were a genuine improvement over the AI approach because their architecture is based, though very loosely, on real nervous systems. Instead of programming computers, neural network researchers, also known as connectionists, were interested in learning what kinds of behaviors could be exhibited by hooking a bunch of neurons together. Brains are made of neurons; therefore, the brain is a neural network. That is a fact. The hope of connectionists was that the elusive properties of intelligence would become clear by studying how neurons interact, and that some of the problems that were unsolvable with AI could be solved by replicating the correct connections between populations of neurons. A neural network is unlike a computer in that it has no CPU and doesn’t store information in a centralized memory. The network’s knowledge and memories are distributed throughout its connectivity—just like real brains.

On the surface, neural networks seemed to be a great fit with my own interests. But I quickly became disillusioned with the field. By this time I had formed an opinion that three things were essential to understanding the brain. My first criterion was the inclusion of time in brain function. Real brains process rapidly changing streams of information. There is nothing static about the flow of information into and out of the brain.

The second criterion was the importance of feedback. Neuroanatomists have known for a long time that the brain is saturated with feedback connections. For example, in the circuit between the neocortex and a lower structure called the thalamus, connections going backward (toward the input) exceed the connections going forward by almost a factor often! That is, for every fiber feeding information forward into the neocortex, there are ten fibers feeding information back toward the senses. Feedback dominates most connections throughout the neocortex as well. No one understood the precise role of this feedback, but it was clear from published research that it existed everywhere. I figured it must be important.

The third criterion was that any theory or model of the brain should account for the physical architecture of the brain. The neocortex is not a simple structure. As we will see later, it is organized as a repeating hierarchy. Any neural network that didn’t acknowledge this structure was certainly not going to work like a brain.

But as the neural network phenomenon exploded on the scene, it mostly settled on a class of ultrasimple models that didn’t meet any of these criteria. Most neural networks consisted of a small number of neurons connected in three rows. A pattern (the input) is presented to the first row. These input neurons are connected to the next row of neurons, the so-called hidden units. The hidden units then connect to the final row of neurons, the output units. The connections between neurons have variable strengths, meaning the activity in one neuron might increase the activity in another and decrease the activity in a third neuron depending on the connection strengths. By changing these strengths, the network learns to map input patterns to output patterns.

These simple neural networks only processed static patterns, did not use feedback, and didn’t look anything like brains. The most common type of neural network, called a “back propagation” network, learned by broadcasting an error from the output units back toward the input units. You might think this is a form of feedback, but it isn’t really. The backward propagation of errors only occurred during the learning phase. When the neural network was working normally, after being trained, the information flowed only one way. There was no feedback from outputs to inputs. And the models had no sense of time. A static input pattern got converted into a static output pattern. Then another input pattern was presented. There was no history or record in the network of what happened even a short time earlier. And finally the architecture of these neural networks was trivial compared to the complicated and hierarchical structure of the brain.

I thought the field would quickly move on to more realistic networks, but it didn’t. Because these simple neural networks were able to do interesting things, research seemed to stop right there, for years. They had found a new and interesting tool, and overnight thousands of scientists, engineers, and students were getting grants, earning PhDs, and writing books about neural networks. Companies were formed to use neural networks to predict the stock market, process loan applications, verify signatures, and perform hundreds of other pattern classification applications. Although the intent of the founders of the field might have been more general, the field became dominated by people who weren’t interested in understanding how the brain works, or understanding what intelligence is.

The popular press didn’t understand this distinction well.

Newspapers, magazines, and TV science programs presented neural networks as being “brainlike” or working on the “same principles as the brain.” Unlike AI, where everything had to be programmed, neural nets learned by example, which seemed, well, somehow more intelligent. One prominent demonstration was NetTalk. This neural network learned to map sequences of letters onto spoken sounds. As the network was trained on printed text, it started sounding like a computer voice reading the words. It was easy to imagine that, with a little more time, neural networks would be conversing with humans. NetTalk was incorrectly heralded on national news as a machine learning to read. NetTalk was a great exhibition, but what it was actually doing bordered on the trivial. It didn’t read, it didn’t understand, and was of little practical value. It just matched letter combinations to predefined sound patterns.

Let me give you an analogy to show how far neural networks were from real brains. Imagine that instead of trying to figure out how a brain worked we were trying to figure out how a digital computer worked. After years of study, we discover that everything in the computer is made of transistors. There are hundreds of millions of transistors in a computer and they are connected together in precise and complex ways. But we don’t understand how the computer works or why the transistors are connected the way they are. So one day we decide to connect just a few transistors together to see what happens. Lo and behold we find that as few as three transistors, when connected together in a certain way, become an amplifier. A small signal put into one end is magnified on the other end. (Amplifiers in radios and televisions are made using transistors in this fashion.) This is an important discovery, and overnight an industry springs up making transistor radios, televisions, and other electronic appliances using transistor amplifiers. This is all well and good, but it doesn’t tell us anything about how the computer works. Even though an amplifier and a computer are both made of transistors, they have almost nothing else in common. In the same way, a real brain and a three-row neural network are built with neurons, but have almost nothing else in common.

During the summer of 1987, I had an experience that threw more cold water on my already low enthusiasm for neural nets. I went to a neural network conference where I saw a presentation by a company called Nestor. Nestor was trying to sell a neural network application for recognizing handwriting on a tablet. It was offering to license the program for one million dollars. That got my attention. Although Nestor was promoting the sophistication of its neural network algorithm and touting it as yet another major breakthrough, I felt the problem of handwriting recognition could be solved in a simpler, more traditional way. I went home that night, thought about the problem, and in two days had designed a handwriting recognizer that was fast, small, and flexible. My solution didn’t use a neural network and it didn’t work at all like a brain. Although that conference sparked my interest in designing computers with a stylus interface (eventually leading to the PalmPilot ten years later), it also convinced me that neural networks were not much of an improvement over traditional methods. The handwriting recognizer I created ultimately became the basis for the text entry system, called Graffiti, used in the first series of Palm products. I think Nestor went out of business.

So much for simple neural networks. Most of their capabilities were easily handled by other methods and eventually the media hoopla subsided. At least neural network researchers did not claim their models were intelligent. After all, they were extremely simple networks and did less than AI programs. I don’t want to leave you with the impression that all neural networks are of the simple three-layer variety. Some researchers have continued to study neural networks of different designs. Today the term neural network is used to describe a diverse set of models, some of which are more biologically accurate and some of which are not. But almost none of them attempt to capture the overall function or architecture of the neocortex.

In my opinion, the most fundamental problem with most neural networks is a trait they share with AI programs. Both are fatally burdened by their focus on behavior. Whether they are calling these behaviors “answers,” “patterns,” or “outputs,” both AI and neural networks assume intelligence lies in the behavior that a program or a neural network produces after processing a given input. The most important attribute of a computer program or a neural network is whether it gives the correct or desired output. As inspired by Alan Turing, intelligence equals behavior.

But intelligence is not just a matter of acting or behaving intelligently. Behavior is a manifestation of intelligence, but not the central characteristic or primary definition of being intelligent. A moment’s reflection proves this: You can be intelligent just lying in the dark, thinking and understanding. Ignoring what goes on in your head and focusing instead on behavior has been a large impediment to understanding intelligence and building intelligent machines.

Before we explore a new definition of intelligence, I want to tell you about one other connectionist approach that came much closer to describing how real brains work. Trouble is, few people seem to have realized the importance of this research.

While neural nets grabbed the limelight, a small splinter group of neural network theorists built networks that didn’t focus on behavior. Called auto-associative memories, they were also built out of simple “neurons” that connected to each other and fired when they reached a certain threshold. But they were interconnected differently, using lots of feedback. Instead of only passing information forward, as in a back propagation network, auto-associative memories fed the output of each neuron back into the input—sort of like calling yourself on the phone. This feedback loop led to some interesting features. When a pattern of activity was imposed on the artificial neurons, they formed a memory of this pattern. The auto-associative network associated patterns with themselves, hence the term auto-associative memory.

The result of this wiring may at first seem ridiculous. To retrieve a pattern stored in such a memory, you must provide the pattern you want to retrieve. It would be like going to the grocer and asking to buy a bunch of bananas. When the grocer asks you how you will pay, you offer to pay with bananas. What good is that? you might ask. But an auto-associative memory has a few important properties that are found in real brains.

The most important property is that you don’t have to have the entire pattern you want to retrieve in order to retrieve it. You might have only part of the pattern, or you might have a somewhat messed-up pattern. The auto-associative memory can retrieve the correct pattern, as it was originally stored, even though you start with a messy version of it. It would be like going to the grocer with half eaten brown bananas and getting whole green bananas in return. Or going to the bank with a ripped and unreadable bill and the banker says, “I think this is a messed-up $100 bill. Give me that one, and I will give you this new, crisp $100 bill.”

Second, unlike most other neural networks, an autoassociative memory can be designed to store sequences of patterns, or temporal patterns. This feature is accomplished by adding a time delay to the feedback. With this delay, you can present an auto-associative memory with a sequence of patterns, similar to a melody, and it can remember the sequence. I might feed in the first few notes of “Twinkle Twinkle Little Star” and the memory returns the whole song. When presented with part of the sequence, the memory can recall the rest. As we will see later, this is how people learn practically everything, as a sequence of patterns. And I propose the brain uses circuits similar to an auto-associative memory to do so.

Auto-associative memories hinted at the potential importance of feedback and time-changing inputs. But the vast majority of AI, neural network, and cognitive scientists ignored time and feedback.

Neuroscientists as a whole have not done much better. They too know about feedback—they are the people who discovered it—but most have no theory (beyond vague talk of “phases” and “modulation”) to account for why the brain needs to have so much of it. And time has little or no central role in most of their ideas on overall brain function. They tend to chart the brain in terms of where things happen, not when or how neural firing patterns interact over time. Part of this bias comes from the limits of our current experimental techniques. One of the favorite technologies of the 1990s, aka the Decade of the Brain, was functional imaging. Functional imaging machines can take pictures of brain activity in humans. However, they cannot see rapid changes. So scientists ask subjects to concentrate on a single task over and over again as if they were being asked to stand still for an optical photograph, except this is a mental photograph. The result is we have lots of data on where in the brain certain tasks occur, but little data on how realistic, time-varying inputs flow through the brain. Functional imaging lends insight into where things are happening at a given moment but cannot easily capture how brain activity changes over time. Scientists would like to collect this data, but there are few good techniques for doing so. Thus many mainstream cognitive neuroscientists continue to buy into the input-output fallacy. You present a fixed input and see what output you get. Wiring diagrams of the cortex tend to show flowcharts that start in the primary sensory areas where sights, sounds, and touch come in, flow up through higher analytical, planning, and motor areas, and then feed instructions down to the muscles. You sense, then you act.

I don’t want to imply that everyone has ignored time and feedback. This is such a big field that virtually every idea has its adherents. In recent years, belief in the importance of feedback, time, and prediction has been on the rise. But the thunder of AI and classical neural networks kept other approaches subdued and underappreciated for many years.

It’s not difficult to understand why people—laymen and experts alike—have thought that behavior defines intelligence. For at least a couple of centuries people have likened the brain’s abilities to clockworks, then pumps and pipes, then steam engines and, later, to computers. Decades of science fiction have been awash in AI ideas, from Isaac Asimov’s laws of robotics to Star Wars’ C3PO. The idea of intelligent machines doing things is engrained in our imagination. All machines, whether made by humans or imagined by humans, are designed to do something. We don’t have machines that think, we have machines that do. Even as we observe our fellow humans, we focus on their behavior and not on their hidden thoughts. Therefore, it seems intuitively obvious that intelligent behavior should be the metric of an intelligent system.

However, looking across the history of science, we see our intuition is often the biggest obstacle to discovering the truth. Scientific frameworks are often difficult to discover, not because they are complex, but because intuitive but incorrect assumptions keep us from seeing the correct answer. Astronomers before Copernicus (1473-1543) wrongly assumed that the earth was stationary at the center of the universe because it feels stationary and appears to be at the center of the universe. It was intuitively obvious that the stars were all part of a giant spinning sphere, with us at its center. To suggest the Earth was spinning like a top, the surface moving at nearly a thousand miles an hour, and that the entire Earth was hurtling through space—not to mention that stars are trillions of miles away—would have marked you as a lunatic. But that turned out to be the correct framework. Simple to understand, but intuitively incorrect.

Before Darwin (1809-1882), it seemed obvious that species are fixed in their forms. Crocodiles don’t mate with hummingbirds; they are distinct and irreconcilable. The idea that species evolve went against not only religious teachings but also common sense. Evolution implies that you have a common ancestor with every living thing on this planet, including worms and the flowering plant in your kitchen. We now know this to be true, but intuition says otherwise.

I mention these famous examples because I believe that the quest for intelligent machines has also been burdened by an intuitive assumption that’s hampering our progress. When you ask yourself, What does an intelligent system do?, it is intuitively obvious to think in terms of behavior. We demonstrate human intelligence through our speech, writing, and actions, right? Yes, but only to a point. Intelligence is something that is happening in your head. Behavior is an optional ingredient. This is not intuitively obvious, but it’s not hard to understand either.

In the spring of 1986, as I sat at my desk day after day reading scientific articles, building my history of intelligence, and watching the evolving worlds of AI and neural networks, I found myself drowning in details. There was an unending supply of things to study and read about, but I was not gaining any clear understanding of how the whole brain actually worked or even what it did. This was because the field of neuroscience itself was awash in details. It still is. Thousands of research reports are published every year, but they tend to add to the heap rather than organize it. There’s still no overall theory, no framework, explaining what your brain does and how it does it.

I started imagining what the solution to this problem would be like. Is it going to be extremely complex because the brain is so complex? Would it take one hundred pages of dense mathematics to describe how the brain works? Would we need to map out hundreds or thousands of separate circuits before anything useful could be understood? I didn’t think so. History shows that the best solutions to scientific problems are simple and elegant. While the details may be forbidding and the road to a final theory may be arduous, the ultimate conceptual framework is generally simple.

Without a core explanation to guide inquiry, neuroscientists don’t have much to go on as they try to assemble all the details they’ve collected into a coherent picture. The brain is incredibly complex, a vast and daunting tangle of cells. At first glance it looks like a stadium full of cooked spaghetti. It’s also been described as an electrician’s nightmare. But with close and careful inspection we see that the brain isn’t a random heap. It has lots of organization and structure—but much too much of it for us to hope we’ll be able to just intuit the workings of the whole, the way we’re able to see how the shards of a broken vase fit back together. The failing isn’t one of not having enough data or even the right pieces of data; what we need is a shift in perspective. With the proper framework, the details will become meaningful and manageable. Consider the following fanciful analogy to get the flavor of what I mean.

Imagine that millennia from now humans have gone extinct, and explorers from an advanced alien civilization land on Earth. They want to figure out how we lived. They are especially puzzled by our network of roadways. What were these bizarre elaborate structures for? They begin by cataloging everything, both via satellites and from the ground. They are meticulous archaeologists. They record the location of every stray fragment of asphalt, every signpost that has fallen over and been carried downhill by erosion, every detail they can find. They note that some road networks are different from others; in some places they are windy and narrow and almost random-looking, in some places they form a nice regular grid, and over some stretches they become thick and run for hundreds of miles through the desert. They collect a mountain of details, but these details don’t mean anything to them. They continue to collect more detail in hopes of finding some new data that explain it all. They remain stumped for a long time.

That is, until one of them says, “Eureka! I think I see … these creatures couldn’t teleport themselves like we can. They had to travel around from place to place, perhaps on mobile platforms of a cunning design.” From this basic insight, many details begin to fall into place. The small twisty street networks are from early times when the means of conveyance were slow. The thick long roadways were for traveling long distances at high speeds, suggesting at last an explanation for why the signs on those roads had different numbers painted on them. The scientists start to deduce residential versus industrial zoning, the way the needs of commerce and transportation infrastructure might have interacted, and so on. Many of the details they had cataloged turn out to be not very relevant, just accidents of history or the requirements of local geography. The same amount of raw data exists, but it no longer is puzzling.

We can be confident that the same type of breakthrough will let us understand what all the brain’s details are about.

Unfortunately, not everyone believes we can understand how the brain works. A surprising number of people, including a few neuroscientists, believe that somehow the brain and intelligence are beyond explanation. And some believe that even if we could understand them, it would be impossible to build machines that work the same way, that intelligence requires a human body, neurons, and perhaps some new and unfathomable laws of physics. Whenever I hear these arguments, I imagine the intellectuals of the past who argued against studying the heavens or fought against dissecting cadavers to see how our bodies worked. “Don’t bother studying that, it will lead to no good, and even if you could understand how it works there is nothing we can do with that knowledge.” Arguments like this one lead us to a branch of philosophy called functionalism, our last stop in this brief history of our thinking about thinking.

According to functionalism, being intelligent or having a mind is purely a property of organization and has nothing inherently to do with what you’re organized out of. A mind exists in any system whose constituent parts have the right causal relationship with each other, but those parts can just as validly be neurons, silicon chips, or something else. Clearly, this view is standard issue to any would-be builder of intelligent machines.

Consider: Would a game of chess be any less real if it was played with a salt shaker standing in for a lost knight piece? Clearly not. The salt shaker is functionally equivalent to a “real” knight by virtue of how it moves on the board and interacts with the other pieces, so your game is truly a game of chess and not just a simulation of one. Or consider, wouldn’t this sentence be the same if I were to go through it with my cursor deleting each character, then retyping it? Or to take an example closer to home, consider the fact that every few years your body replaces most of the atoms that comprise you. In spite of this, you remain yourself in all the ways that matter to you. One atom is as good as any other if it’s playing the same functional role in your molecular makeup. The same story should hold for the brain: if a mad scientist were to replace each of your neurons with a functionally equivalent micromachine replica, you should come out of the procedure feeling no less your own true self than you had at the outset.

By this principle, an artificial system that used the same functional architecture as an intelligent, living brain should be likewise intelligent—and not just contrivedly so, but actually, truly intelligent.

AI proponents, connectionists, and I are all functionalists, insofar as we all believe there’s nothing inherently special or magical about the brain that allows it to be intelligent. We all believe we’ll be able to build intelligent machines, somehow, someday. But there are different interpretations of functionalism. While I’ve already stated what I consider the central failing of the AI and the connectionist paradigms—the input-output fallacy—there’s a bit more worth saying about why we haven’t yet been able to design intelligent machines. While the AI proponents take what I consider a self-defeating hard line, the connectionists, in my view, have mainly been just too timid.

AI researchers ask, “Why should we engineers be bound by the solutions evolution happened to stumble upon?” In principle, they have a point. Biological systems, like the brain and the genome, are viewed as notoriously inelegant. A common metaphor is that of the Rube Goldberg machine, named after the Depression-era cartoonist who drew comically overcomplicated contraptions to accomplish trivial tasks. Software designers have a related term, kludge, to refer to programs that are written without foresight and wind up full of burdensome, useless complexity, often to the point of becoming incomprehensible even to the programmers who wrote them. AI researchers fear the brain is similarly a mess, a several-hundred-million-year-old kludge, chock-full of inefficiencies and evolutionary “legacy code.” If so, they wonder, why not just throw out the whole sorry clutter and start afresh?

Many philosophers and cognitive psychologists are sympathetic to this view. They love the metaphor of the mind being like software that’s run by the brain, the organic analog of computer hardware. In computers, the hardware level and the software level are distinct from each other. The same software program can be made to run on any Universal Turing Machine. You can run WordPerfect on a PC, a Macintosh, or a Cray supercomputer, for example, even though all three systems have different hardware configurations. And the hardware has nothing of importance to teach you if you’re trying to learn WordPerfect. By analogy, the thinking goes, the brain has nothing to teach us about the mind.

AI defenders also like to point out historical instances in which the engineering solution differs radically from nature’s version. For example, how did we succeed in building flying machines? By imitating the flapping action of winged animals? No. We did it with fixed wings and propellers, and later with jet engines. It may not be how nature did it, but it works—and does so far better than flapping wings.

Similarly, we made a land vehicle that could outrun cheetahs not by making four-legged, cheetah-like running machines, but by inventing wheels. Wheels are a great way to move over flat terrain, and just because evolution never stumbled across that particular strategy doesn’t mean it’s not an excellent way for us to get around. Some philosophers of mind have taken a shine to the metaphor of the “cognitive wheel,” that is, an AI solution to some problem that although entirely different from how the brain does it is just as good. In other words, a program that produces outputs resembling (or surpassing) human performance on a task in some narrow but useful way really is just as good as the way our brains do it.

I believe this kind of ends-justify-the-means interpretation of functionalism leads AI researchers astray. As Searle showed with the Chinese Room, behavioral equivalence is not enough. Since intelligence is an internal property of a brain, we have to look inside the brain to understand what intelligence is. In our investigations of the brain, and especially the neocortex, we will need to be careful in figuring out which details are just superfluous “frozen accidents” of our evolutionary past; undoubtedly, many Rube Goldberg-style processes are mixed in with the important features. But as we’ll soon see, there is an underlying elegance of great power, one that surpasses our best computers, waiting to be extracted from these neural circuits.

Connectionists intuitively felt the brain wasn’t a computer and that its secrets lie in how neurons behave when connected together. That was a good start, but the field barely moved on from its early successes. Although thousands of people worked on three-layer networks, and many still do, research on cortically realistic networks was, and remains, rare.

For half a century we’ve been bringing the full force of our species’ considerable cleverness to trying to program intelligence into computers. In the process we’ve come up with word processors, databases, video games, the Internet, mobile phones, and convincing computer-animated dinosaurs. But intelligent machines still aren’t anywhere in the picture. To succeed, we will need to crib heavily from nature’s engine of intelligence, the neocortex. We have to extract intelligence from within the brain. No other road will get us there.