It’s hard to predict the ultimate uses of a new technology. As we’ve seen throughout this book, brains make predictions by analogy to the past. So our natural inclination is to imagine that a new technology will be used to do the same kinds of things as a previous technology. We imagine using a new tool to do something familiar, only faster, more efficiently, or more cheaply.

Examples are abundant. People called the railroad the “iron horse” and the automobile the “horseless carriage.” For decades the telephone was viewed in the context of the telegraph, something that should be used only to communicate important news or emergencies; it wasn’t until the 1920s that people started using it casually. Photography was at first used as a new form of portraiture. And motion pictures were conceptualized as a variation on stage plays, which is why movie theaters had retracting curtains over the screens for much of the twentieth century.

Yet the ultimate uses of a new technology are often unexpected and more far-reaching than our imaginations can at first grasp. The telephone has evolved into a wireless voice and data communications network permitting any two people on the planet to communicate with each other, no matter where they are, via voice, text, and images. The transistor was invented by Bell Labs in 1947. It was instantly clear to people that the device was a breakthrough, but the initial applications were just improvements on old applications: transistors replaced vacuum tubes. This led to smaller and more reliable radios and computers, which was important and exciting in its day, but the main differences were the size and reliability of the machines. The transistor’s most revolutionary applications weren’t discovered until later. A period of gradual innovation was necessary before anyone could conceive of the integrated circuit, the microprocessor, the digital signal processor, or the memory chip. The microprocessor, likewise, was first developed, in 1970, with desktop calculators in mind. Again, the first applications were just replacements of existing technologies. The electronic calculator was a replacement for the mechanical desktop calculator. Microprocessors were also clear candidates to replace the solenoids that were then used in certain kinds of industrial control, such as switching traffic lights. However, it was years before the true power of the microprocessor began to be manifest. No one at the time could foresee the modern personal computer, the cell phone, the Internet, the Global Positioning System, or any other piece of today’s bread-and-butter information technology.

By the same token, we would be foolish to think we can predict the revolutionary applications of brainlike memory systems. I fully expect these intelligent machines to improve life in all sorts of ways. We can be sure of it. But predicting the future of technology more than a few years out is impossible. To appreciate this you need only read some of the absurd prognostications futurists have confidently made over the years. In the 1950s, it was predicted that by the year 2000 we’d all have atomic reactors in our basements and take our vacations on the moon. But as long as we keep these cautionary tales in mind, there’s a lot to be gained by speculating about what intelligent machines will be like. At a minimum, there are certain broad and useful conclusions we can draw about the future.

The questions are intriguing ones. Can we build intelligent machines, and, if so, what will they look like? Will they be closer to the humanlike robots seen in popular fiction, the black or beige box of a personal computer, or something else? How will they be used? Is this a dangerous technology that can harm us or threaten our personal liberties? What are the obvious applications for intelligent machines, and is there any way we can know what the fantastic applications will be? What will the ultimate impact of intelligent machines be on our lives?

Yes, we can build intelligent machines, but they may not be what you expect. Although it may seem like the obvious thing to do, I don’t believe we will build intelligent machines that act like humans, or even interact with us in humanlike ways.

One popular notion of intelligent machines comes to us from movies and books—they are the lovable, evil, or occasionally bumbling humanoid robots that converse with us about feelings, ideas, and events, and play a role in endless science-fiction plots. A century of science fiction has trained people to view robots and androids as an inevitable and desirable part of our future. Generations have grown up with images of Robbie the Robot from Forbidden Planet, R2D2 and C3PO from Star Wars, and Lieutenant Commander Data from Star Trek. Even HAL in the movie 2001: A Space Odyssey, although not possessing a body, was very humanlike, designed to be as much a companion as a programmed copilot for the humans on their long space journey. Limited-application robots—things like smart cars, autonomous minisubmarines to explore the deep ocean, and self-guided vacuum cleaners or lawn mowers—are feasible and may well grow more common someday. But androids and robots like Commander Data and C3PO are going to remain fictional for a very long time. There are a couple of reasons for this.

First, the human mind is created not only by the neocortex but also by the emotional systems of the old brain and by the complexity of the human body. To be human you need all of your biological machinery, not just a cortex. To converse like a human on all matters (to pass the Turing Test) would require an intelligent machine to have most of the experiences and emotions of a real human, and to live a humanlike life. Intelligent machines will have the equivalent of a cortex and a set of senses, but the rest is optional. It might be entertaining to watch an intelligent machine shuffle around in a humanlike body, but it will not have a mind that is remotely humanlike unless we imbue it with humanlike emotional systems and humanlike experiences. That would be extremely difficult and, it seems to me, quite pointless.

Second, given the cost and effort that would be necessary to build and maintain humanoid robots, it is difficult to see how they could be practical. A robot butler would be more expensive and less helpful than a human assistant. While the robot might be “intelligent,” it would not have the kind of rapport and easy understanding a human assistant would have by virtue of being a fellow human being.

Both the steam engine and the digital computer evoked robotic visions, which never came to fruition. Similarly, when we think of building intelligent machines, many people find it natural to imagine humanlike robots once again, but it is unlikely to happen. Robots are a concept born of the industrial revolution and refined by fiction. We should not look to them for inspiration in developing genuinely intelligent machines.

So what will intelligent machines look like if not walking talking robots? Evolution discovered that if it attached a hierarchical memory system to our senses, the memory would model the world and predict the future. Borrowing from nature, we should build intelligent machines along the same lines. Here, then, is the recipe for building intelligent machines. Start with a set of senses to extract patterns from the world. Our intelligent machine may have a set of senses that differ from a human’s, and may even “exist” in a world unlike our own (more on this later). So don’t assume that it has to have a set of eyeballs and a pair of ears. Next, attach to these senses a hierarchical memory system that works on the same principles as the cortex. We will then have to train the memory system much as we teach children. Over repetitive training sessions, our intelligent machine will build a model of its world as seen through its senses. There will be no need or opportunity for anyone to program in the rules of the world, databases, facts, or any of the high-level concepts that are the bane of artificial intelligence. The intelligent machine must learn via observation of its world, including input from an instructor when necessary. Once our intelligent machine has created a model of its world, it can then see analogies to past experiences, make predictions of future events, propose solutions to new problems, and make this knowledge available to us.

Physically, our intelligent machine might be built into planes or cars, or sit stoically on a rack in a computer room. Unlike humans, whose brains must accompany their bodies, the memory system of an intelligent machine might be located remotely from its sensors (and “body,” if it had one). For example, an intelligent security system might have sensors located throughout a factory or a town, but the hierarchical memory system attached to those sensors could be locked in a basement of one building. Therefore, the physical embodiment of an intelligent machine could take many forms.

There is no reason that an intelligent machine should look, act, or sense like a human. What makes it intelligent is that it can understand and interact with its world via a hierarchical memory model and can think about its world in a way analogous to how you and I think about our world. As we will see, its thoughts and actions might be completely different from anything a human does, yet it will still be intelligent. Intelligence is measured by the predictive ability of a hierarchical memory, not by humanlike behavior.

Let’s turn our attention to the largest technical challenge we will face when building intelligent machines, creating the memory. To build intelligent machines, we will need to construct large memory systems that are hierarchically organized and that work like the cortex. We will confront challenges with capacity and connectivity.

Capacity is the first issue. Let’s say the cortex has 32 trillion synapses. If we represented each synapse using only two bits (giving us four possible values per synapse) and each byte has eight bits (so one byte could represent four synapses), then we would need roughly 8 trillion bytes of memory. A hard drive on a personal computer today has 100 billion bytes, so we would need about eighty of today’s hard drives to have the same amount of memory as a human cortex. (Don’t worry about the exact numbers because they are all rough guesses.) The point is, this amount of memory is definitely buildable in the lab. We aren’t off by a factor of a thousand, but it is also not the kind of machine you could put in your pocket or build into your toaster. What is important is that the amount of memory required is not out of the question, whereas only ten years ago it would have been. Helping us is the fact that we don’t have to recreate an entire human cortex. Much less may suffice for many applications.

Our intelligent machines will need lots of memory. We will probably start building them using hard drives or optical disks, but eventually we will want to build them out of silicon as well.

Silicon chips are small, low power, and rugged. And it is only a matter of a time before silicon memory chips could be made with enough capacity to build intelligent machines. In fact, there is an advantage intelligent memory has over conventional computer memory. The economics of the semiconductor industry is based on the percentage of chips that have errors. For many chips even a single error will make the chip useless. The percentage of good chips is called the yield. It determines whether a particular chip design can be manufactured and sold at a profit. Because the chance of an error increases as the size of the chip does, most chips today are no bigger than a small postage stamp. The industry has boosted the amount of memory on a single chip not by making the chip larger but, mostly, by making the individual features on the chip smaller.

But intelligent memory chips will be inherently tolerant of faults. Remember, no single component of your brain holds any indispensable item of data. Your brain loses thousands of neurons each day, yet your mental capacity decays at only a slow pace throughout your adult life. Intelligent memory chips will work on the same principles as cortex, so even if a percentage of the memory elements come out defective, the chip will still be useful and commercially viable. Most likely, the inherent tolerance to errors of brainlike memory will allow designers to build chips that are significantly larger and denser than today’s computer memory chips. The result is that we may be able to put a brain in silicon sooner than current trends might indicate.

The second problem we have to overcome is connectivity. Real brains have large amounts of subcortical white matter. As we noted earlier, the white matter is made up of the millions of axons streaming this way and that just beneath the thin cortical sheet, connecting the different regions of the cortical hierarchy with each other. An individual cell in the cortex may connect to five or ten thousand other cells. This kind of massively parallel wiring is difficult or impossible to implement using traditional silicon manufacturing techniques. Silicon chips are made by depositing a few layers of metal, each separated by a layer of insulation. (This process has nothing to do with the layers in the cortex.) The layers of metal contain the “wires” of the chip, and because wires can’t cross within a layer, the total number of wired connections is limited. This is not going to work for brainlike memory systems, where millions of connections are necessary. Silicon chips and white matter are not very compatible.

A lot of engineering and experimentation will be necessary to solve this problem, but we know the basics of how it will be solved. Electrical wires send signals much more quickly than the axons of neurons. A single wire on a chip can be shared, and therefore used for many different connections, whereas in the brain each axon belongs to just one neuron.

A real-world example is the telephone system. If we ran a line from every telephone to every other telephone, the surface of the globe would be buried under a jungle of copper wire. What we do instead is have all the telephones share a relatively small number of high-capacity lines. This method works as long as the capacity of each line is far greater than the capacity required to transmit a single conversation. The telephone system meets this requirement: a single fiber optic cable can carry a million conversations at once.

Real brains have dedicated axons between all cells that talk to each other, but we can build intelligent machines to be more like the telephone system, sharing connections. Believe it or not, some scientists have been thinking about how to solve the brain chip connectivity problem for many years. Even though the operation of the cortex remained a mystery, researchers knew that we would someday unravel the puzzle, and then we would have to face the issue of connectivity. We don’t need to review the different approaches here. Suffice it to say that connectivity might be the biggest technical obstacle we face in building intelligent machines but we should be able to handle it.

Once the technological challenges are met, there are no fundamental problems that prevent us from building genuinely intelligent systems. Yes, there are lots of issues that will need to be addressed to make these systems small, low cost, and low power, but nothing is standing in our way. It took fifty years to go from room-size computers to ones that fit in your pocket. But because we are starting from an advanced technological position, the same transition for intelligent machines should go much faster.

Throughout the twenty-first century, intelligent machines will emerge from the realm of science fiction into fact. Before we get there, we should think about ethical issues, whether the possible dangers of intelligent machines outweigh the likely benefits.

The prospect of machines that can think and act on their own has worried people for a long time. This is understandable. New areas of knowledge and new technologies always scare people when they first come along. Human creativity lets us imagine the terrible ways a new technology may take over our bodies, outmode our usefulness, or cancel out the very value of human life. But history shows that these dark imaginings almost never play out the way we expect. When the industrial revolution came along, we feared electricity (remember Frankenstein?) and steam engines. Machinery that had its own energy, that could move itself in complex ways, seemed miraculous and at the same time potentially sinister. But electricity and internal combustion engines are no longer strange and sinister. They are as much a part of our environment as air and water.

When the information revolution began, we quickly came to fear computers. There were countless science-fiction stories about powerful computers or computer networks that spontaneously became self-aware and then turned on their organic masters. But now that computers have become integrated into daily life, this fear seems absurd. The computer in your home, or the Internet, has as much chance of spontaneously turning sentient as does a cash register.

Any technology can be applied to good or evil ends, of course, but some are more inherently prone to misuse or catastrophe than others. Atomic energy is dangerous whether it’s in the form of nuclear warheads or power plants because a single accident or a single misuse could harm or kill millions of people. And although nuclear energy is valuable, alternatives are available. Vehicular technology can take the form of tanks and fighter jets, or it can take the form of cars and passenger airplanes, and a mishap or misuse can cause harm to many people. But vehicles are arguably both more essential to modern life and less dangerous than nuclear power. The damage caused by a single misuse of an airplane is much less than that of a nuclear bomb. There are many technologies that are almost wholly beneficial. Telephones are an example. Overwhelmingly, their tendency to bring and keep people together exceeds any negative effects. The same goes for electricity and public health science. In my opinion, intelligent machines are going to be one of the least dangerous, most beneficial technologies we have ever developed.

Still, some thinkers, like Sun Microsystems cofounder Bill Joy, fear that we may develop intelligent robots that could escape our control, swarm the Earth, and remake it according to their own agenda. The image puts me in mind of those magically animated broomsticks from the Sorcerer’s Apprentice, regenerating themselves from their splinters and working tirelessly to bring about disaster. Along similar lines, some AI optimists offer extended-life prophecies that are unsettling. For instance, Ray Kurzweil talks about the day when nanorobots will crawl within your brain, recording every synapse and every connection, and then report all the information to a supercomputer, which will reconfigure itself into you! You’ll become a “software” version of yourself that will be practically immortal. These two predictions about machine intelligence, the intelligent machines–-run-amok scenario and the upload-your-brain-into-a-computer scenario, seem to surface over and over.

Building intelligent machines is not the same as building selfreplicating machines. There is no logical connection whatsoever between them. Neither brains nor computers can directly self-replicate, and brainlike memory systems will be no different. While one of the strengths of intelligent machines will be our ability to mass-produce them, that’s a world apart from selfreplication in the manner of bacteria and viruses. Self-replication does not require intelligence, and intelligence does not require self-replication.

Further, I seriously doubt we will ever be able to copy our minds into machines. There are at present, as far as I know, no actual or imagined methods capable of recording the trillions of details that make “you.” We would need to record and re-create all of your nervous system and your body, not just your neocortex. And we would need to understand how all of it works. One day, certainly, we might be to able do this, but the challenges extend far beyond understanding how the cortex works. Figuring out the neocortical algorithm and building it into machines from scratch is one thing, but scanning in the zillions of operational details of a living brain and replicating them in a machine is something completely different.

Beyond self-replication and the copying of minds, people have another concern with intelligent machines. Might intelligent machines somehow threaten large portions of the population, as nuclear bombs do? Might their presence lead to the superempowerment of small groups or malevolent individuals? Or might the machines become evil and work against us, like the implacable villains in The Terminator or the Matrix movies?

The answer to these questions is no. As information devices, brainlike memory systems are going to be among the most useful technologies we have yet developed. But like cars and computers, they will only be tools. Just because they are going to be intelligent does not mean they will have special abilities to destroy property or manipulate people. And just as we wouldn’t put the control of the world’s nuclear arsenal under the authority of one person or one computer, we will have to be careful not to rely too much on intelligent machines, for they will fail as all technology does.

This gets us to the malevolence question. Some people assume that being intelligent is basically the same as having human mentality. They fear that intelligent machines will resent being “enslaved” because humans hate being enslaved. They fear that intelligent machines will try to take over the world because intelligent people throughout history have tried to take over the world. But these fears rest on a false analogy. They are based on a conflation of intelligence—the neocortical algorithm—with the emotional drives of the old brain—things like fear, paranoia, and desire. But intelligent machines will not have these faculties. They will not have personal ambition. They will not desire wealth, social recognition, or sensual gratification. They will not have appetites, addictions, or mood disorders. Intelligent machines will not have anything resembling human emotion unless we painstakingly design them to. The strongest applications of intelligent machines will be where the human intellect has difficulty, areas in which our senses are inadequate, or in activities we find boring. In general, these activities have little emotional content.

Intelligent machines will range from simple, single-application systems to very powerful superhuman intelligent systems, but unless we go out of our way to make them humanlike, they won’t be. Maybe someday we will have to place restrictions on what people can do with intelligent machines, but that day is a long way off, and when it comes, the ethical issues are likely to be relatively easy compared with such present-day moral questions as those surrounding genetics and nuclear technology.

Now on to the question What will intelligent machines do?

I’m often asked to give talks about the future of mobile computing. A conference organizer will ask me to describe what handheld computers or cell phones will look like in five or in twenty years. They want to hear my vision for the future. I can’t do this. I try to avoid what I call the V word altogether. To make a point, I once walked onstage with a wizard’s hat and a crystal ball. I explained that no one can see the future in detail. Crystal balls are fiction, and anyone who pretends to know exactly what will happen in the coming years is sure to fail. Instead, the best we can do is to understand broad trends. If you understand a broad idea, you can successfully pursue it wherever it goes as the details unfold.

The most famous example of a technology trend is Moore’s Law. Gordon Moore correctly predicted that the number of circuit elements that could be placed on a silicon chip would double every year and a half. Moore didn’t say whether the chips would be memory chips, central processing units, or something else. He didn’t say what kinds of products the chips would be used in. He didn’t predict whether the chips would be housed in plastic or ceramic or glued to circuit boards. He didn’t say anything about the various processes used to manufacture chips. He stuck to the broadest trend he could, and he got it right.

We in the present day cannot predict the ultimate uses of intelligent machines. There is simply no way we are going to get the details right. If I, or anyone else, predict in detail what these machines will do, we will inevitably be proven wrong. Nonetheless, we can do better than just shrugging our shoulders. There are two lines of thought that may be helpful. One is to envision the very near-term uses for brainlike memory systems—the obvious, but less interesting, things to try first. The second approach is to think about the long-term trends, like Moore’s Law, that can help us imagine the applications that could possibly be part of our future.

Let’s begin with some near-term applications. These are the things that seem obvious, like replacing tubes in a radio with transistors or building calculators with a microprocessor. And we can start by looking at some areas that AI tried to tackle but couldn’t solve—speech recognition, vision, and smart cars.

If you have ever tried to use speech recognition software to enter text on a personal computer, you know how dumb it can be. Like Searle’s Chinese Room, the computer has no understanding of what is being said. The few times I tried these products, I grew frustrated. If there was any noise in the room, from a dropped pencil to someone speaking to me, extra words would appear on my screen. The recognition error rates were high. Often the words the software thought I said made no sense. “Remember to fell Mary that the bog is ready to be piqued up.” A child would know this is wrong, but not the computer. Similarly, so-called natural language interfaces have been a goal for computer scientists for years. The idea is for you to be able to tell a computer or other appliance what it is you want, in plain language, and let the machine do the work. On a personal digital assistant, or PDA, you might say, “Move my daughter’s basketball game on Sunday to ten in the morning.” This kind of thing has been impossible to do well with traditional AI. Even if the computer could recognize each word, for it to complete the task it might need to know where your daughter goes to school, that you probably mean the coming Sunday, and maybe what a basketball game is because the appointment might only say “Menlo vs. St. Joe.” Or perhaps you want a computer to listen to the audio stream from a radio broadcast for any mention of a particular product, but the radio announcer describes the product without using its name. You and I would know what she is talking about, but not a computer.

These and many other applications require that a machine be able to listen to spoken language. But computers can’t perform these tasks, because they do not understand what is being said. They match auditory patterns to word templates by rote, without knowing what the words mean. Imagine if you learned to recognize the sounds of individual words in a foreign language, but not the meaning of the words, and I asked you to transcribe a conversation in that language. As the conversation flows, you have no idea what it is about, but you try to pick out the words in isolation. However, the words overlap and interfere, and pieces of sound drop out because of noise. You would find it extremely difficult to separate words and recognize them. These obstacles are what speech recognition software struggles with today. Engineers have discovered that by using probabilities of word transitions, they can improve the software’s accuracy somewhat. For example, they use rules of grammar to decide between homonyms. This is a very simple form of prediction, but the systems are still dumb. Today’s speech recognition software succeeds only in highly constrained situations in which the number of words you might say at any given moment is limited. Yet humans perform many language-related tasks easily, because our cortex understands not only words but sentences and the context within which they are spoken. We anticipate ideas, phrases, and individual words. Our cortical model of the world does this automatically.

So we can expect that cortexlike memory systems will transform fallible speech recognition into robust speech understanding. Instead of programming in probabilities for single word transitions, a hierarchical memory will track accents, words, phrases, and ideas and use them to interpret what is being said. Like a person, such an intelligent machine could distinguish between various speech events—for example, a discussion between you and a friend in the room, a phone conversation, and editing commands for a book. It won’t be easy to build these machines. To fully understand human language, a machine will have to experience and learn what humans do. So even though it may be many years before we can build an intelligent machine that understands language as well as you and I do, we will be able, in the near term, to improve on the performance of existing speech recognition systems by using cortical-like memories.

Vision offers another set of applications that AI has been unable to accomplish but that real intelligent systems should be able to handle. Today there is no machine that can look at a natural scene—the world in front of your eyes, or a picture using a camera—and describe what it sees. There are a few successful applications of machine vision that work in very restricted domains, such as visually aligning chips on a circuit board or matching facial features to a database, but it’s currently impossible for a computer to identify varieties of objects or analyze a scene more generally. You have no difficulty looking around a room to find a place to sit, but don’t ask a computer to do it. Imagine looking at a video screen from a security camera. Can you tell the difference between someone knocking on a door holding a gift and someone knocking down the door with a crow bar? Of course you can, but the distinction is well beyond the capability of today’s software. Consequently we hire people to keep an eye on the screens of security cameras round-the-clock, looking for something suspicious. It is difficult for human watchers to stay alert, whereas an intelligent machine could perform the task tirelessly.

Finally, let’s look at transportation. Cars are getting pretty sophisticated. They have Global Positioning Systems to map your route from A to B, sensors to turn on the lights when it gets dark, accelerometers to deploy airbags, and proximity sensors to tell you if you are about to back over something. There are even cars that can drive autonomously on special highways or in ideal conditions, although they are not commercially available. But to drive a car safely and effectively on all types of roads and in all kinds of traffic conditions requires more than a few sensors and feedback control circuits. To be a good driver, you must understand traffic, other drivers, the way cars work, signal lights, and a slew of other things. You need to be able to understand hazard signs or notice when another car is driving dangerously. You need to see a directional signal on another car and anticipate that it is likely to change lanes, or, if the signal has been on for several minutes, realize that the driver probably doesn’t know that the signal is on and therefore probably won’t change lanes. You need to recognize that a puff of smoke far ahead might mean that an accident has occurred and therefore you should slow down. A driver seeing a ball roll across the street and thinking, automatically, that a child may run out to grab it intuitively comes to a stop.

Let’s say we wanted to build a truly smart car. The first thing we would do is to select a set of sensors that would allow our smart car to sense its world. We might start with a camera for seeing, perhaps multiple cameras in front and back, and microphones for hearing, but we might also want to provide radar or ultrasound sensors that can accurately determine the range and speed of other objects in lit or dark conditions. The point is, we don’t have to rely on or restrict ourselves to the senses humans use. The cortical algorithm is flexible, and, as long as we design our hierarchical memory system appropriately, it should work no matter what types of sensors we install. Our car could, in theory, be better at sensing the world of traffic than we are because its set of senses can be chosen to match the task. The sensors would then be attached to a sufficiently large hierarchical memory system. The car designers would train the smart car’s memory by exposing it to real-world conditions so that it learns to build a model of its world in the same way humans do—only in a more limited realm. (For example, the car needs to know about roads but not about elevators and airplanes.) The car’s memory would learn the hierarchical structure of traffic and roads so that it could understand and anticipate what is likely to happen in its world of moving automobiles, road signs, obstacles, and intersections. The car’s engineers could design the memory system so that it actually drives the car or just monitors what happens when you drive. It could give advice, or take over in extreme situations, like a backseat driver that you don’t resent. Once the memory is fully trained and the car can understand and handle anything that might happen, the engineers would have the option of permanently setting the memory so that all cars coming off the assembly line behave the same way, or they could design the memory to continue learning after the car is sold. And as with a computer but not a human, the memory could be reprogrammed with an updated version should new conditions warrant doing so.

I am not saying we will definitely build smart cars or machines that understand language and vision. But these are good examples of the kinds of devices we might research and develop, and that seem possible to build.

Personally, I am less interested in the obvious applications of intelligent machines. To me, the true benefit and excitement of a new technology is in finding uses for it that were inconceivable before. In what ways will intelligent machines surprise us, and what fantastic capabilities will emerge over time? I am certain that hierarchical memories, like the transistor and the microprocessor, will transform our lives for the better in incredible ways, but how? One way we can glimpse the future of intelligent machines is to think of aspects of the technology that will scale well. That is, which attributes of intelligent machines will grow cheaper and cheaper, faster and faster, or smaller and smaller. Things that grow at exponential rates rapidly outpace our imagination and are most likely to play a key role in the most radical evolutions in future technology.

Examples of technologies that have improved exponentially for many years include the silicon memory chip, the hard disk, DNA sequencing techniques, and fiber optics. These rapidly scaling technologies have been the basis for many new products and businesses. In a different fashion, software also scales well. A desirable program, once it is written, can be copied endlessly at virtually no cost.

In contrast, some technologies, such as batteries, motors, and traditional robotics, scale poorly. Despite lots of effort and steady improvements, a robotic arm built today is not dramatically better than one built a few years ago. The developments in robotics are gradual and modest, nothing like the exponential growth curves of chip design or software proliferation. A robot arm built in 1985 for a million dollars cannot be built today a thousand times as strong for just ten dollars. Likewise, batteries today are not vastly better than they were ten years ago. You might say they are two or three times better, but not a thousand or ten thousand times better, and progress moves forward only bit by bit. If the capacity of batteries increased at the same rate as the capacity of hard disks, then cell phones and other electronics would never have to be charged, and lightweight electric cars that traveled a thousand miles per charge would be common.

So it behooves us to think about which aspects of brainlike memory systems will scale dramatically beyond our biological brains. These attributes will suggest where the technology will ultimately land. I see four attributes that will exceed our own abilities: speed, capacity, replicability, and sensory systems.

While neurons work on the order of milliseconds, silicon operates on the order of nanoseconds (and is still getting faster). That’s a million-fold difference, or six orders of magnitude.

The speed difference between organic and silicon-based minds will be of great consequence. Intelligent machines will be able to think as much as a million times faster than the human brain. Such a mind could read whole libraries of books or study huge, complicated bodies of data—tasks that would take you or me years to complete—in mere minutes, while getting exactly the same understanding out of it. There is nothing magic about this. Biological brains evolved with two time-related constraints. One is the speed at which cells can do things and the other is the speed at which the world changes. It might not be too useful for a biological brain to think a million times faster if the world around it is inherently slow. But there is nothing about the cortical algorithm that says it must always operate slowly. If an intelligent machine conversed or interacted with a human, it would have to slow down to work at human speed. If it read a book by flipping pages, there would be a limit to how fast it could read. But when it is interfacing with the electronic world, it could function much more quickly. Two intelligent machines could hold a conversation a million times faster than two humans. Imagine the progress of an intelligent machine that solved mathematical or scientific problems a million times faster than a human. In ten seconds it could give as much thought to a problem as you could in a month. Never-tiring, never-bored minds of such lightning speed are sure to be useful in ways we can’t yet imagine.

Despite the impressive memory capacity of a human cortex, intelligent machines can be built to surpass it by large margins. The size of our brains has been constrained by several biological factors, including the ratio of infant skull size to maternal pelvis diameter, the high metabolic cost of running a brain (your brain is about 2 percent of your body weight but uses around 20 percent of the oxygen you breathe), and the slow speed of neurons. But we can build intelligent memory systems of any size, and, unlike the blind, meandering process of evolution, we can bring foresight and specific intention to the details of the design. The capacity of the human neocortex may turn out to be relatively modest some decades from now.

As we build intelligent machines, we could increase their memory capacity in several ways. Adding depth to the hierarchy will lead to deeper understanding—the ability to see higherorder patterns. Enlarging the capacity within regions will allow the machine to remember more details, or perceive with greater acuity, in the same way a blind person has a more refined sense of touch or hearing. And adding new senses and sensory hierarchies permits the device to construct better models of the world, as I will discuss shortly.

It will be interesting to see if there is an upper limit to how big an intelligent memory system can get and in what dimensions. Conceivably, a device might become too cluttered to be useful, or it might fail as it approaches some theoretical limit. Perhaps the human brain is already near the maximum theoretical size, but I find that unlikely. Human brains became large very recently in evolutionary time, and there is nothing to suggest that we are at some stable maximum size. Whatever the peak capacity for an intelligent memory system turns out to be, the human brain is almost certainly not there. It’s probably not even close.

One way to see what these systems might do is to look at the limits of known human performance. Einstein was undoubtedly extremely smart, but his brain was still a brain. We can assume his extraordinary intelligence was largely the product of physical differences between his brain and the typical human brain. What made Einstein so rare was that the human genome doesn’t often produce brains like his. However, when we are designing brains in silicon, we can build them any way we want. They could all be capable of Einstein’s high level of thought, or even smarter. At another extreme, savants can give us insight into other possible dimensions of intelligence. Savants are mentally retarded individuals who exhibit remarkable abilities such as nearphotographic memories or the capacity to perform difficult mathematical calculations at lightning speeds. Their brains, although not typical, are still brains, working with the cortical algorithm. If an atypical brain can have amazing memory abilities, then, in theory, we could add those capabilities to our artificial brains. Not only do these extremes of human mental ability indicate what should be possible to re-create; they represent the directions in which we can probably exceed the best human performance.

Each new organic brain must be grown and trained de novo, a process that takes decades in human beings. Each human must discover for himself or herself the basics of coordinating the body’s limbs and muscle groups, of balancing and moving, and learn the general properties of multitudinous objects, animals, and other people; the names of things and the structure of the language; and the rules of family and society. Once those basics are mastered, years and years of formal schooling commence. Every individual must trudge up the same set of learning curves in life, even though they have been trudged countless times by others, all to build a model of the world in the cortex.

Intelligent machines need not undergo this long learning curve, since chips and other storage can be replicated endlessly and the contents transferred easily. In this sense, intelligent machines could replicate like software. Once a prototype system has been satisfactorily honed and trained, it could be copied as many times as we please. It may take years of chip design, hardware configuration, training, and trial and error to perfect the memory system for a smart car, but once the final product is achieved, it can be mass-produced. As I mentioned earlier, we could choose to allow the copies to continue learning or not. For some applications, we will want to limit our intelligent machines to operate in a tested and known way. Once a smart car knows everything we need it to know, we wouldn’t want it to develop bad habits or come to believe in some false analogy it thinks it sees. If nothing else, we would expect all cars of a similar make to behave similarly. But for other applications, we’ll want our brainlike memory systems to be fully capable of continuous learning. For instance, an intelligent machine designed to discover mathematical proofs will need the ability to learn from experience, to apply old insights to new problems, and to be generally flexible and open-ended.

It should be possible to share components of learning the way we share components of software. An intelligent machine of a particular design could be reprogrammed with a new set of connections to lead to different behavior, as if I could download a new set of connections into your brain and instantly change you from an English speaker to a French speaker, or from a political science professor to a musicologist. People could swap and build on the work of others. Let’s say I have developed and trained a machine with a superior visual system, and another person has developed and trained a machine with a superior sense of hearing. With the proper design, we could combine the best of both systems without retraining from the bottom up. Sharing expertise in this way is simply impossible for humans. The business of building intelligent machines could evolve along the same lines as the computer industry, with communities of people training intelligent machines to have specialized knowledge and abilities, and selling and swapping the resultant memory configurations. Reprogramming an intelligent machine will not be too different from running a new video game or installing a piece of software.

Humans have a handful of senses. These senses are deeply ingrained in our genes, in our bodies, and in the subcortical wiring of our brains. We can’t change them. Sometimes we use technology to augment our senses, such as with night-vision goggles, radar, or the Hubble space telescope. These high-tech instruments are clever tricks of data translation, not new modes of perception. They convert information we can’t sense into visual or auditory displays that we can interpret. Still, it is a tribute to our brain’s amazing flexibility that we can look at a radar screen and understand what it represents. Many animal species possess truly different senses, such as the echolocation of bats and dolphins, the ability of bees to see polarized and ultraviolet light, and the electric field sense of some fish.

Our intelligent machines could perceive the world through any sense found in nature as well as new senses of purely human design. Sonar, radar, and infrared vision are obvious examples of the kinds of nonhuman senses that we may want our intelligent machines to possess. But they are only the beginning.

Far more interesting is the way intelligent machines could experience genuinely exotic, alien worlds of sensation. As we have seen, the neocortical algorithm is fundamentally concerned with finding patterns in the world. It has no preference for the physical origins of those patterns. So long as the inputs to the cortex are nonrandom and have a certain amount of richness or statistical structure, an intelligent system will form invariant memories and predictions about them. There is no reason for these input patterns to be analogous to animal senses, or even to derive from the real world at all. It is in the realm of exotic senses that, I suspect, the revolutionary uses of intelligent machines lie.

For example, we could design a sensory system that spans the globe. Imagine weather sensors spaced every fifty or so miles across a continent. These sensors would be analogous to the cells in a retina. At any point in time, two adjacent weather sensors will have a high correlation in their activity, just like two adjacent cells on a retina. There are large weather objects, such as storms and fronts that move and change over time, just as there are visual objects that move and change over time. By attaching this sensory array to a large cortical-like memory, we would enable the system to learn to predict the weather in the way that you and I learn to recognize visualize objects and predict how they move over time. The system would see local weather patterns, large weather patterns, and patterns that exist over decades, years, or hours. By placing sensors close together in some regions, we could create the equivalent of a fovea, allowing our intelligent weather brain to understand and predict microclimates. Our weather brain would think about and understand global weather systems as you and I think about and understand objects and people. Meteorologists aim to do something similar today. They collect recordings from diverse locations and use supercomputers to simulate the climate and forecast the future. But this approach, which is fundamentally different from the way an intelligent machine would work, is akin to how a computer plays chess—dumb and without understanding—whereas our intelligent weather machine is akin to how a human plays chess—thoughtfully and with understanding. The intelligent weather machine would discover patterns that humans have not. It was only in the 1960s that the weather phenomenon known as El Niño was discovered. Our weather brain could find more patterns like El Niño, or learn how to predict tornadoes or monsoons far better than humans. Putting large amounts of weather data into a form that humans can readily understand is difficult; our weather brain, in contrast, would sense and think about weather directly.

Other large distributed sensory systems could allow us to build intelligent machines that understand and predict animal migrations, changes in demographics, and the spread of disease. Imagine having sensors distributed over a country’s electrical power grid. An intelligent machine attached to these sensors would observe the ebb and flow of electricity consumption in the same way you and I see the ebb and flow of traffic on a road, or the movement of people at an airport. Through repeated exposure, humans learn to predict these patterns—just ask an employee who commutes by car, or an airport security guard. Similarly, our intelligent electrical grid monitor would be able to predict demands for power, or dangerous situations likely to lead to a power outage, better than a human. We might combine sensors for weather and for human demographics, in order to anticipate political unrest, famines, or disease outbreaks. Like a supersmart diplomat, intelligent machines may play a role in reducing conflict and human suffering. You might think intelligent machines would need emotions to foresee patterns involving human behavior, but I don’t think so. We are not born with a set culture, a set of values, and a set religion; we learn them. And just as I can learn to understand the motivations of people with values different from mine, intelligent machines can comprehend human motivations and emotions, even if the machine doesn’t have those emotions itself.

We could make senses that sample minute entities. It is theoretically possible to have sensors that could represent patterns in cells or large molecules. For example, an important challenge today is to understand how the shape of a protein molecule can be predicted from the sequence of amino acids that comprise the protein. Being able to predict how proteins fold and interact would accelerate the development of medicines and the cures for many diseases. Engineers and scientists have created three-dimensional visual models of proteins, in an effort to predict how these complex molecules behave. But try as we might, the task has proven too difficult. A superintelligent machine, on the other hand, with a set of senses specifically tuned to this question might be able to answer it. If this sounds far-fetched, remember that we wouldn’t be surprised if humans could solve the problem. Our inability to tackle the issue may be related, primarily, to a mismatch between the human senses and the physical phenomena we want to understand. Intelligent machines can have custom senses and largerthan-human memory, enabling them to solve problems we can’t.

With the proper senses and a slight restructuring of the cortical memory, our intelligent machines might live and think in virtual worlds used in mathematics and physics. For example, many endeavors in math and science require understanding how objects behave in worlds that have more than three dimensions. String theorists, who study the nature of space itself, think about the universe as having ten or more dimensions. Humans have great difficulty thinking about mathematical problems in four or more dimensions. Perhaps an intelligent machine of the correct design could understand high-dimensional spaces in the same way that you and I understand threedimensional spaces, and therefore be adept at predicting how they behave.

Finally, we might unite a bunch of intelligent systems in a grand hierarchy, just as our cortex unites hearing, touch, and vision higher up the cortical hierarchy. Such a system would automatically learn to model and predict the patterns of thinking in populations of intelligent machines. With distributed communications mediums such as the Internet, the individual intelligent machines could be distributed around the globe. Larger hierarchies learn deeper patterns and see more complex analogies.

The point of these musings is to illustrate that there are many ways brainlike machines could surpass our own abilities, and in dramatic ways. They might think and learn a million times faster than we can, remember vast quantities of detailed information, or see incredibly abstract patterns. They can have senses more sensitive than our own, or senses that are distributed, or senses for very small phenomena. They might think in three, four, or more dimensions. None of these interesting possibilities depend on intelligent machines mimicking or acting like humans, and they don’t involve complex robotics.

Now we can see fully how the Turing Test, by equating intelligence with human behavior, limited our vision of what is possible. By first understanding what intelligence is, we can build intelligent machines that are far more valuable than merely replicating human behavior. Our intelligent machines will be amazing tools and will dramatically expand our knowledge of the universe.

How long will it be before any of this comes about? Will we be building intelligent machines in fifty years, twenty years, or five years? There is a saying in the high-tech world that change takes longer than you expect in the short term but occurs faster than you expect in the long term. I have seen this many times. Someone will get up at a conference, announce a new technology, and claim that it will be in every home in four years. The speaker turns out to be wrong. Four years become eight years, and people start thinking it will never happen. Just around that time, when it looks like the whole idea was a dead end, it starts taking off and becomes a big sensation. Something similar will likely happen in the intelligent machine business. Progress will seem slow at first, and then take off rapidly.

At neuroscience conferences, I like to go around the room and ask everyone to state his or her opinion on how long it will be before we have a working theory of cortex. A few people—fewer than 5 percent—say “never” or “we already have one” (surprising answers given what they do for a living). Another 5 percent say five to ten years. Half of the rest say ten to fifty years, or “within my lifetime.” The remaining people say fifty to two hundred years, or “not within my lifetime.” I side with the optimists. We have been living in the “slow” period for decades, so to many people it seems that progress in theoretical neuroscience and intelligent machines has stalled completely. Judging by the progress in the past thirty years it is natural to assume we are nowhere near an answer. But I believe we are at the turning point and the field is about to take off.

It is possible to accelerate the future, to move the turning point closer to the present. One of the goals of this book is to convince you that, with the correct theoretical framework, we can make rapid progress in understanding the cortex—that with the memory-prediction framework as our guide, we can decipher the details of how the brain works and how we think. This is the knowledge we need to build intelligent machines. If this is the right model, progress can proceed rapidly.

So although I am hesitant to predict when the age of intelligent machines will be a reality, I think that if enough people apply themselves to solving this problem today, we may be able to create useful prototypes and cortical simulations within just a few years. Within ten years, I hope, intelligent machines will be one of the hottest areas of technology and science. I am reluctant to be more specific than this, because I know how easy it is to underestimate the time it takes to make something important happen. So why am I so optimistic about the speed of progress in understanding the brain and building intelligent machines? My confidence stems mostly from the long time I have already spent working on the problem of intelligence. When I first fell in love with brains, in 1979, I felt solving the puzzle of intelligence was something that could be achieved in my lifetime. Over the years, I have carefully observed the decline of AI, the rise and fall of neural networks, and the Decade of the Brain in the 1990s. I have seen how attitudes toward theoretical biology, and theoretical neuroscience in particular, have evolved. I have seen how the ideas of prediction, hierarchical representation, and time have crept into the language of neuroscience. I have seen the progress in my own understanding and that of my colleagues. I became excited about the role of prediction eighteen years ago and have in some ways been testing it ever since. Because I have been immersed in the neuroscience and computer fields for over two decades, perhaps my brain has built a high-level model of how technological and scientific change occurs, and that model predicts rapid progress. Now is the turning point.