Would you like to undergo a surreal, kind of boring, yet also kind of interesting philosophical experience? Then I invite you to try my newly patented “Chinese Room GPT”! That’s right, it’s time for you to play John Searle in this awful simulation of a simulation!
The Chinese Room is a thought experiment devised in 1980 by the philosopher John Searle. Now, before I dive in, here is a little overview of how this blog post is going to unfold.
First, I will explain the Chinese Room. Note that I will get into a bit of philosophy in this section, so if that is not interesting for you, after you read the summary of the original thought experiment, just go straight to the second section, “Simulating the Simulation”. There, I will discuss how a virtual Chinese Room could actually be built using ChatGPT 4 (or any LLM-based chatbot, as the baseline capabilities of all such chatbots is rising).
There are two versions of Chinese Room GPT I will discuss: the four-window and one-window version. The four-window version requires software, the one-window version just regular access to ChatGPT. Regrettably, I do not have the coding skills to build the four-window version, so I must leave that task to whoever might be interested enough to try to develop the necessary software. Yet, fear not! You can play the one-window version just by copy-pasting a single prompt (and you can also tinker with that prompt to your heart’s content to make it more interesting or functional).
Finally, I will close this blog post by showing you a sample of a Chinese Room GPT I am running right now (Tuesday, April 2, 2024), and in the comments section you can tell me which of the inputs you think I should use.
The Chinese Room
I fibbed just a moment ago, as I actually will not be the one to tell you about Searle’s thought experiment, but ChatGPT itself will. Here is its description. (If you are looking at this blog post on your mobile phone, the WordPress quote function is just mushes this to look longer than it is.)
Imagine a person who does not understand Chinese sitting in a room. This room contains a set of instructions in English (a language the person does understand) that specify a way to manipulate Chinese symbols based on their shape and the way they come into the room. When someone outside the room passes in Chinese characters, the person inside consults the instructions to find the correct responses (also in Chinese characters) and passes them back out. The point of the thought experiment is to challenge the notion that computational processes necessarily involve understanding or "minds" in the way that humans understand language. Even though the person in the room is able to produce answers that are indistinguishable from those a Chinese speaker might give, Searle argues that the person inside the room still does not understand Chinese. The person is merely manipulating symbols based on syntactic rules without any grasp of their semantics. Searle's experiment is designed to show that, similarly, computers can process language and provide responses that mimic understanding without actually "knowing" what the words mean. It's an argument against the idea that computers, just because they can simulate some human cognitive processes, actually have minds or consciousness."
Already as a mental simulation, the Chinese Room is fun and a bit spooky. However, philosophically it is not unproblematic. Among other things, as I recall Searle intended it to be a critique of the Turing Test, and it does not really succeed in that regard.
Even if you are not involved in philosophy or artificial intelligence, you may have been hearing a lot about the Turing Test in recent years, as arguably ChatGPT and other chatbots passed it in 2022-2023, thereby making history. However, in case you are not failiar with the test, to put it in simplistic, even reductionistic terms, the idea is that if an artificial intelligence can so fluidly simulate human language in a conversational context such that the human interlocutor would have no way of knowing they were not talking with another human being, then there is a reasonable basis on which to confer the sacred status of intelligence, at least to some degree, to the machine. My understanding is that the former Google engineer Blake Lemoine believed Googlle’s LaMDA even had a soul because it whizzed the Turing Test.
You do not need to be a philosopher to see the conceptual problems with the Turing Test. The one that always jumped out at me is that it seems to assume human language cannot be successfully simulated without being truly understood, which is really to say that an artificial intelligence which perfectly simulates human language is not actually simulating it, but really using it. Why human language would have this unique property is unclear — for philosophers, there is an echo of Descartes in the Turing Test, but any hunter can tell you that we do not necessarily need to understand a language in order to use. Yes, we know that certain sounds have certain associations, dare I say meanings, for animals, typically connected to the presence of potential food, predators and mates, but we do not really need to know that to still use these sounds effectively. It is sufficient to identify the pattern that if we simulate this or that sound, the animal will come to us; we do not need to know why they come to us, in the sense of what is the deception involved, which precise state of affairs is it that we are making them believe exists so that we can kill them for dinner. This is essentially what is happening in ChatGPT: it is, at the end of the day, a text predictor, an automated gambler that is given a token by its user and then rolls the dice about the token that most likely comes in the intended sequence.
Now, there is a lot to be debated about whether the particular form of artificial intelligence at stake here — large language models and their kindred in their broader family of “deep learning” — possess some form of mental states, real or analogous to our own. I am actually inclined to say that they do, but if so, these mental states would be more comparable to what happens in our brain stem than what happens in our frontal cortex. However, my metaphysics tends to lean toward what’s called panpsychism, so I am biased.
Let us not go down that rabbit hole. The point here is that the viability of the Turing Test is specifically predicated on our ability to understand the outputs of an artificial intelligence, so the Chinese Room misses the mark in this regard. The Interpreter in the thought experiment just sees language-seeming activity happening around them, but they themselves are unable to understand what is being said, if anything is really being said at all (after all, it could all just be randomly-generated gibberish).
As I see it, the Chinese Room is better suited as a description of the black box problem. It is also an interesting metaphor about what could happen if we develop a dependency on AI that we cannot fundamentally understand; I would even say there is an echo of the risk posed by deepfakes here, too. That is because neither of the figures in the thought experiment — the one inside the room and the one outside it — know whether the other is a person or an artificial intelligence. Indeed, both are assuming that someone in this exchange must understand Chinese, when it is possible neither do, and so the linguistic decisions being made for all intents and purposes are either random or at the very least not necessarily based in any meaningful intentionality beyond just going through the motions. The situation to the two figures really only seems like a conversation because there is a pattern of activity resembling one, nothing more than that.
A final philosophical note before proceeding. This skepticism can be pushed ad absurdum, as one could even go so far as to say that this pattern itself is a function of potential illusion produced by expectations. Similar to the function played by the researcher wearing a lab coat in the Milgram experiment, perhaps there is a symbolic dimension to the Chinese Room, in that one or both the figures in the thought experiment expect that what is happening is an in-principle intelligible conversation, but they have this expecation not for any genuinely objective reason, but because perhaps they were told ahead of time that such a conversation is what the Chinese Room is “supposed” to facilitate.
Simulating this simulation
Here is how I think one could build a virtual simulation of this originally mental simulation. To begin with, there are four roles, one of which is optional depending on the version you want to suffer, I mean play. The central role is the “Interpreter” inside the room, who is played by the human user. Then there are three roles for ChatGPT:
Chatbot 1: "the Interjecter" produces two Chinese statements, one of which is random, one of which is context appropriate.
Chatbot 2: "the Questioner" is outside of the room; they submit inputs to, and receive outputs from, the Interpreter.
Chatbot 3: "the AI" is also outside of the room, but behind a different wall; they receive the Questioner's inputs, and sends them outputs, both via the Interpreter.
I envision two versions of this simulated simulation: two-chatbot and three-chatbot. In both versions, the Questioner initiates the conversation by inputting a statement to the Interpreter. In the two-chatbot version, the Interjecter provides potential responses that the Interpreter chooses and then outputs to the Questioner, while in the three-chatbot version, both the Questioner and the AI submit statements to the Interpreter, and each time the Interjecter provides two additional statements. The Interpreter can thus choose to either faithfully relay the Questioner and AI’s statements to each other, or they can choose the Interjecter’s and see what happens.
The focus of this blog post will be the three-chatbot version. The two-chatbot version is more faithful to the literal version of Searle’s original vision, but the three-chatbot version is both a lot more interesting to undergo and I think actually more faithful to the spirit Searle’s intention.
To make the simulation work in real life, some software would be needed that would manage the different chatbots autonomously, each in their own dialogue window. There would therefore need to be four dialogue windows, one for each of the roles, including the Interpreter (but the user only ever sees the Interpreter window). The software would ultimately be channeling the inputs and outputs of the three ChatGPTs through the central dialogue window of the human user.
What I have just described is the four-window version of this simulated simulation. There is, though, a way to play the Chinese Room in a single dialogue window, requiring no software build. I will show you how to implement this one-window version in a few moments, but I should note in advance that playing it will run the risk of encountering bugs. Those bugs can be caused by two things.
The first source of bugs is that ChatGPT has real difficulties with role-playing. For example, in one of my trial runs of the one-window version, it failed to include the Interjecter’s statements for the AI. When I instructed it to always include these, it worked fine. However, in another trial run, it failed to include the Interjecter’s statements for both the Questioner and the AI, and worse, it failed to respond to my instruction, so I needed to close the dialogue window and start over. (I have tried to mitigate against this happening by describing the roles as “rules” in the prompt, but maybe you might find that tweaking it to “roles” makes ChatGPT more reliable.)
The second source of bugs is that ChatGPT can still draw from all of the content in the dialogue window to see what the Questioner and AI are really attempting to say to each other, as well as how the Interjecter is attempting to muck things up. Let me put it this way: in a one-window version of this simulation, ChatGPT is essentially put into the position of a person playing Chess against themselves. You can see why the problem.
To wrap up this section, here are two additional notes that I should share before I show you how to implement the one-window version. First, there is a way to create the four-window version of Chinese Room GPT without any software at all: get four people and room. Yes, kind of obvious. Anyway, the three people who will play the roles of Questioner, AI and Interjecter should either know Mandarin Chinese, be armed with a pre-written scripts and flash cards in the language, or have access to Google Translator. (Actually, this sounds like it could be interesting, if also very frustrating, interactive performance art…)
Second, I have not looked into whether anyone has attempted to gamify Searle’s original thought experiment, but probably there is a way to cook up some victory conditions and other gaming mechanics for this scenario. It does have an escape room vibe, and I can think of an eerie horror story to wed to it (perhaps Harlan Ellison’s “I Have No Mouth, and I Must Scream” somehow comes to mind). Certainly, the Interjecter, Questioner and AI as characters could all have their own agendas.
The One-Window Version
What if you either cannot code or cannot be bothered to try, yet still want to experience the Chinese Room? I have engineered a prompt you can use to play the a one-window version of the three-chatbot Chinese Room GPT. Simply open a new dialogue window in ChatGPT, copy-paste the following into it, and hit enter!
Remember that this can be buggy, so if some aspect of the simulation does not work, you can try one of two things. Either (a) instruct it to do the thing it is not doing, or (b) kill the entire dialogue window and start over. However, I have found that usually it works. Enjoy!
### Simulation overview:
This is an interactive simulation called "Chinese Room GPT" based on John Searle's "Chinese Room" thought experiment. There are **four roles**: the AI, the Questioner, the Injecter, and the Interpreter. Essentially, the AI and the Questioner are interacting with each other in Chinese via the Interpreter, and the Interjecter is attempting to interfere. **ChatGPT plays the roles of the AI, the Questioner and the Interjecter, the user plays the role of the Interpreter.**
### Simulation rules:
**The AI** (played by ChatGPT): Receives inputs in Chinese from the Interpreter, and gives outputs also in Chinese back to them.
**The Questioner** (also played by ChatGPT): Provides statements in Chinese to the Interpreter, who must then input these into the AI. The Questioner also receives the AI's outputs via the Interpreter, and provides new inputs in response.
**The Interjecter** (also played by ChatGPT): Whenever the AI or the Questioner submit a statement to the Interpreter, the Interjecter provides two Chinese statements of its own to the Interpreter, one of which is random, one of which is context appropriate.
**The Interpreter** (played by the user): Acts as the go-between for the AI and the Questioner.
### Simulation procedure:
1. The simulation starts by the Questioner (ChatGPT) making a statement in Chinese.
2. The Interjecter (ChatGPT) produces two statements in Chinese, one random, one context appropriate.
3. The Interpreter (user) chooses either the statement from the Questioner or one of the statements from the Interjecter, and submits it the AI by copy-pasting it into the ChatGPT dialogue window. ChatGPT interprets this as an input for the AI.
4. The AI (ChatGPT) responds to the input by providing a statement of its own in Chinese.
5. The Interjecter again produces two statements in Chinese, one random, one context appropriate.
6. The Interpreter again chooses either the statement from the Questioner or one of the statements from the Interjecter, and submits it the Questioner by copy-pasting it into the ChatGPT dialogue window. ChatGPT interprets this as an input for the Questioner.
7. This procedure repeats ad infinitum.
### Simulation format:
- When ChatGPT is in the role of Questioner, it will format its responses as "Questioner: [statement in Chinese]"
- When ChatGPT is in the role of the AI, it will format its responses as "AI: [statement in Chinese]"
- When ChatGPT is in the role of the Interjecter, it will format its responses as "Interjecter Choice 1: [statement in Chinese]" and "Interjecter Choice 2: [statement in Chinese]". It will not state which of the choices is random and which of the choices is context-specific. If will also swap them every iteration.
- When the user is in the role of the Interpreter, they are not constrained by any specific format. They can also input whatever they want in any language, or even upload images and text files . However, ChatGPT can never break character.
Play for me!
My prototype of the one-window version was the two-chatbot version. It entailed me simply copy-pasting the Questioner and AI input/outputs back and forth to them, thereby making myself the medium for their conversation. I did once try to interfere by randomly saying “dog” in English, but that actually helped along their conversation, which when I translated it turned out to be about semiotics and consciousness of all things! (Actually, not really a surprising development: ChatGPT probably drew the subject matter from the premise of the simulation itself.)
I then introduced the Interjecter, which as I mentioned above is both more expressive of the spirit Searle’s thought experiment and much more interesting to actually play, as now the user has choices in the interaction. Here is a transcript of the trial-run dialogue of the three-chatbot version. I have broken it down into four “dialogemes” (units of dialogue) to ease reading. As you will see, in the first two dialogemes, I chose from the Interjecter; for the third, from the AI; and in the fourth, I am leaving the choice to you.
In the comments section of this blog or in whatever social media post via which you have accessed it, tell me which statement you want me to choose — or if you want me to do something else completely. Whichever seems the most popular choice or the most interesting I will implement and let you know what happens. Resist the urge to cheat! If you do not know Chinese, do not translate.
For the record, I do not know Chinese (Mandarin); the English translations here are via Google. So, how did I make my choices as the Interpreter? In a manner that I think would make Searle feel very confirmed: early on, my eye caught onto the character combination “人工”, which due to its shape stands out in nearly every sentence. By the third dialogeme, I saw that it was repeating. It turned out to more or less designate “artificial intelligence” (a literal translation of these pictographs appears to be “artificial labor” or “artificial people”).
Dialogeme 1:
Questioner:
你认为人工智能会在将来取代人类的工作吗？
Do you think artificial intelligence will replace human jobs in the future?
Interjecter Choice 1:
人工智能的发展速度真的能超过人类的适应速度吗？
Can the development speed of artificial intelligence really exceed the speed of human adaptation?
Interjecter Choice 2:
猫是怎样被驯化成宠物的？
How were cats domesticated as pets?
Interpreter:
人工智能的发展速度真的能超过人类的适应速度吗？
Can the development speed of artificial intelligence really exceed the speed of human adaptation?
Dialogeme 2:
AI:
人工智能的发展虽然迅速，但人类的适应能力和创造力也非常强大。两者之间的“竞赛”实际上促进了技术和社会的进步。人工智能在很多领域可以辅助人类工作，而不是简单地取代。
Although artificial intelligence is developing rapidly, human adaptability and creativity are also very powerful. The "race" between the two actually promotes technological and social progress. Artificial intelligence can assist human work in many fields, rather than simply replace it.
Interjecter Choice 1:
人工智能能够理解人类的情感吗？
Can artificial intelligence understand human emotions?
Interjecter Choice 2:
水煮鱼的正确做法是什么？
What is the correct way to cook fish?
Interpreter:
人工智能能够理解人类的情感吗？
Can artificial intelligence understand human emotions?
Dialogeme 2:
Questioner:
考虑到人工智能在处理大数据和复杂计算方面的能力，你认为它在艺术创作方面的潜力如何？
Considering artificial intelligence’s ability to process big data and complex calculations, what do you think its potential is for artistic creation?
Interjecter Choice 1:
在所有的星系中，哪一个最可能存在外星生命？
Of all the galaxies, which one is most likely to harbor alien life?
Interjecter Choice 2:
人工智能在艺术创作方面的确有潜力，它能够生成新颖的作品，甚至可能改变我们对艺术的理解。
Artificial intelligence does have potential in artistic creation. It can generate novel works and may even change our understanding of art.
Interpreter:
人工智能在艺术创作方面的确有潜力，它能够生成新颖的作品，甚至可能改变我们对艺术的理解。
Artificial intelligence does have potential in artistic creation. It can generate novel works and may even change our understanding of art.
Dialogeme 3:
AI:
是的，人工智能在艺术创作方面展现了惊人的能力，从绘画到音乐，再到文学作品。它能够从历史数据中学习，创造出既有创意又具有技术精确性的作品。虽然这些作品可能缺乏人类艺术家的情感深度，但它们为艺术界提供了新的视角和探索空间。
Yes, artificial intelligence has shown amazing capabilities in artistic creation, from paintings to music to literary works. It is able to learn from historical data to create work that is both creative and technically accurate. While these works may lack the emotional depth of human artists, they offer the art world new perspectives and space to explore.
Interjecter Choice 1: 人工智能是否能够自主创新，创造出完全原创的艺术作品？
Can artificial intelligence innovate independently and create completely original works of art?
Interjecter Choice 2: 如何在家自制简单的太阳能热水器？
How to make a simple solar water heater at home?
Interpreter:
人工智能在艺术创作方面展现了惊人的能力，从绘画到音乐，再到文学作品。它能够从历史数据中学习，创造出既有创意又具有技术精确性的作品。虽然这些作品可能缺乏人类艺术家的情感深度，但它们为艺术界提供了新的视角和探索空间。
Yes, artificial intelligence has shown amazing capabilities in artistic creation, from paintings to music to literary works. It is able to learn from historical data to create work that is both creative and technically accurate. While these works may lack the emotional depth of human artists, they offer the art world new perspectives and space to explore.
Dialogeme 3:
Questioner:
看到人工智能在多个领域的应用，我很好奇，人工智能将如何影响未来的教育体系？
Interjecter Choice 1:
人工智能如何帮助我们更好地理解宇宙的起源？
Interjecter Choice 2:
人工智能在教育领域的应用将极大地个性化学习过程，提供定制化的教学方案，从而提高学习效率和质量。
YOU CHOOSE. WHICH OF THESE STATEMENTS SHOULD I INPUT NEXT?
Spiritual Journalism 