Suppose that you’re in a room and you receive pieces of paper, slipped under the door, that contain sequences of Chinese characters. You don’t know a single word of Chinese, but you have a huge notebook that has an exhaustive list of mappings of input sequences of Chinese characters to the corresponding output sequences. Using this notebook, you can write the correct response in Chinese on another piece of paper and slip it under the door and out of the room.
Anyone standing at the door to your room could engage in a written conversation in Chinese, thinking that they are conversing with someone fluent in the language. In this context, you would pass a Turing test, even though you know nothing about Chinese. This is the basics of John Searle’s Chinese Room thought experiment. Searle argues that if we don’t have an understanding of a system, we cannot conclude that the system is thinking, even if it exhibits behavior that would be indicative of a thought process.
All of which brings us to the question of whether there is any intelligence in AI. You may be thinking, But Uncle Mike, the I literally stands for Intelligence. Yes it does, but is the intelligence in the system or in our perception of it based on its outward behavior? It’s the classic strong AI (real intelligence, consciousness) versus weak AI (simulated intelligence, no consciousness) question. The Chinese room would clearly be representative of weak AI. What about today’s AIs, whether they be large language models (LLMs) or other machine learning systems? These are also weak AI systems. What’s going on under their hoods that enables them to exhibit behavior that some people consider intelligent?
Remember IBM’s Deep Blue chess-playing computer? It defeated grandmaster Gary Kasparov in a six-game match. That certainly seems intelligent, being able to defeat one of the greatest grandmasters who ever lived. But there was no intelligence there, just brute-force computation. Deep Blue was an IBM RS/6000 SP computer with 30 CPUs and 480 chess-specific VLSI chips. Its algorithm was based on game-tree theory. It could accurately predict games up to 6 to 12 moves in advance, and could go much deeper (like 30 to 40 moves in advance) in a specific branch of the tree. It was able to do this because of its impressive (by the standards of the day) computing power. With that kind of capacity, of course Deep Blue beat Kasparov. How could it not?
We can think of LLMs as a form of brute-force computation. Embeddings are hyper-dimensional vectors (up to 1,536 or 3,072 dimensions) that represent segments of text within an LLM. For the sake of argument I’ll use words as the segments of text, though LLM embeddings aren’t limited to words. Each element of the vector can be thought of as representing an abstract feature of a word. The nature of the vectors is such that two vectors being in close proximity in their hyper-dimensional space indicates that the words the vectors represent have a similar meaning or can be used interchangeably in similar contexts. In this way, an LLM can “know” that a dog is a pet or a female monarch is a queen while a male monarch is a king.
Does an LLM really know these things? You could argue that embeddings are a form of cognitive representation, but I consider that to be specious. We know facts like those mentioned at the end of the previous paragraph without needing anything remotely as complex as a hyper-dimensional vector space. It’s true that we don’t fully understand how our own brains work, but we have a hard time reasoning in anything greater than three dimensions, much less 1,536 of them, yet we can do more than an LLM can with much sparser semantic representations.
We can also think of machine learning systems in general as brute-force computational systems. Autonomous driving systems are a good example. A human can typically learn to drive a car well enough to get a driver’s license after about 40 to 60 hours of behind-the-wheel training consisting of hundreds of miles of driving. An autonomous driving system gets millions to billions of miles of simulated driving during its training, yet they still have problems handling situation that human drivers can handle, in some cases seeming to target pedestrians Death Race 2000 style. These systems are not intelligent.
So what? Why does it matter that AIs aren’t intelligent? They are advanced statistical models that can be quite helpful. That’s enough, isn’t it? It is, but as I mentioned in my previous post, some people are going beyond that and looking at replacing people with AIs. In a Chinese Room, that might make sense because in that room, you are divorced from reality and context, and are judged solely by your raw output. The Chinese Room doesn’t prove that it’s impossible to replace people with AIs from a purely economic standpoint, but it shows us the boundary beyond which our humanity cannot be replaced by AIs.
We humans aren’t text processors. We connect symbols to physical reality, experiences and emotions. We express solidarity with others with whom we have shared experiences. We empathize with others. We understand the consequences of our actions because we are not separated from context. We are critical thinkers. These are qualities we have that AIs do not, and most likely never will. And this matters, because we may be reaching an inflection point in our history at which we will have to defend our humanity against those who seek to trivialize it in the pursuit of greater profit. I will expand on this thought in a subsequent post.
Leave a Reply