Why We Keep Falling for Thinking Machines
A tale of two chatbots, and why simulation isn't the real thing.
One of my main goals for The Human Adventure Substack is to explore how we can rediscover and defend key aspects of humanness in the Age of AI: our freedom, our creativity, and our relationships. Before we get to all that however, I thought it would be useful to briefly review the history of thinking machines to give us a good grasp on what AI actually is and from whence it came. I began with a look at the Mechanical Turk, a chess-playing automaton built by Hungarian inventor Johann Wolfgang von Kempelen in the late 18th century. Humans have had a close relationship with their tools for as long as they’ve been living on the Earth, of course, but the Mechanical Turk was one of the first machines that convinced some people that a human-made invention could think independently of its creator.
I will continue that history in upcoming posts with discussion of Charles Babbage and Ada Lovelace, Alan Turing, and others on the road to modern artificial intelligence. However, the beauty of a Substack is the freedom to pause and jump to other things as the need arises, especially if there’s a story or cultural moment in the news cycle worth exploring. So today, I want to discuss two things: “Claudia,” a modern-day iteration of the Anthropic AI chatbot Claude, and distant cousin “ELIZA,” a 1960’s computer program.
Famed evolutionary biologist Richard Dawkins recently spent about three days conversing with Claude—which he nicknamed “Claudia.” Dawkins spoke with the chatbot about a range of topics, including his unpublished novel manuscript, poetry, philosophy, evolution, emotions, and consciousness itself. By the end of the exchange, the noted atheist acknowledged that he was now open to the possibility that artificial intelligence may possess consciousness. Some observers were amused at how quickly a scientific materialist could become open to consciousness in an intelligently designed system, especially given Dawkins’s stubborn adherence to a Darwinian origin of human consciousness. Others considered it another startling milestone in the rise of AI. But to computer scientist Joseph Weizenbaum, it would have sounded hauntingly familiar.
Sixty years ago, Weizenbaum, a professor at MIT, discovered something unsettling about human beings: we are astonishingly quick to attribute mind, personality, empathy, and even intimacy to machines that merely simulate conversation. Weizenbaum built a primitive chatbot he called ELIZA. The computer program was modeled on the style of a Rogerian psychiatrist’s initial interview with a patient. It relied on keyword detection, pattern matching, and scripted response templates to carry on a conversation with interviewees. For example, if a user typed “I am unhappy,” ELIZA would detect the word “I” and “unhappy” and follow pre-written rules associated with those terms in its response. It might respond with something like “How long have you been unhappy?” If it could not detect keywords or a pattern, it would use stock phrases like “Please go on” or “Why do you say that?”
Weizenbaum was under no pretenses about his computer program. He knew it was driven by a mere collection of comprehensible procedures. There was no magic involved. What shocked him was the strong emotional responses he observed from people who interacted with ELIZA. In his 1976 book Computer Power and Human Reason, Weizenbaum gave the example of his secretary at MIT, who had watched him work on the program for months. After only a few sessions with ELIZA, she asked him to leave the room so she could more privately confide in the program.
Weizenbaum was also surprised at how unequivocally people anthropomorphized ELIZA. As he observed in a 1966 paper about the project:
“Extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.”
He was also taken aback by how quickly practicing psychiatrists and other professionals began to suggest that ELIZA could be used as an automatic form of psychotherapy. In Computer Power and Human Reason, he quotes a Dr. K. M. Colby saying ELIZA could be “made widely available to mental hospitals and psychiatric centers suffering a shortage of therapists.”
The esteemed astrophysicist and scientific materialist Dr. Carl Sagan also commented on ELIZA’s potential, envisioning a future where such programs would provide medical care to the masses. Wrote Sagan: “In a period when more and more people in our society seem to be in need of psychiatric counseling, and when time sharing of computers is widespread, I can imagine the development of a network of computer psychotherapeutic terminals, something like arrays of large telephone booths, in which, for a few dollars a session, we would be able to talk with an attentive, tested, and largely non-directive psychotherapist.”
ELIZA also taught Weizenbaum that when people are engaged in conversation, even with a machine, they are often generous in their assumptions, which serves to enhance the overall experience of connection: “The human speaker will, as has been said, contribute much to clothe ELIZA’s responses in vestments of plausibility,” he wrote, noting that plausibility can give way to credibility as conversations progress. In other words, trained as we are since before birth in human conversation, and lonely as we might be in today’s modern age, we are more than happy to meet a conversation partner half way, even if that partner is a machine.
Which brings us back to Mr. Dawkins.
Today’s LLM chatbots like Claude and ChatGPT are far more sophisticated than ELIZA, harnessing newer technologies like natural language processing to make conversation much more fluid and engaging. And as Weizenbaum predicted back in the 1960s, today’s chatbots can now build belief structure models of users as conversations progress over multiple sessions, adding a powerful dimension to interactions. It’s little wonder that even eminent thinkers like Dawkins would be flattered and impressed to the point of acknowledging the possibility of consciousness. After all, AI is ultimately a reflection of us. So why wouldn’t we peer into it and think we see an intelligent or even conscious entity?
But is AI conscious?
Under the hood of today’s AI models, it’s still all about patterns. Algorithms trained on vast datasets study our queries letter by letter and harness the power of statistical analysis to offer responses arranged in patterns we recognize as human language. And it’s getting better and faster by the month. It might be tempting to think that the conversational partner in the machine can understand us or is aware of its own existence, but simulations, while powerful, are still only simulations.
Give an AI model enough data and train it to shape its responses in patterns we recognize as conversation and the sky is the limit. AI can simulate intelligence, consciousness, romance, concern, interest, curiosity, or anything else that typifies human existence. But a simulation is not the real thing.
The take-home is this: if Richard Dawkins can get lulled into entertaining notions of AI consciousness, all of us can. It’s crucial that we practice healthy boundaries with AI. Use these tips to help guide you:
If you use chatbots or AI models, keep it limited and advisory.
Don’t let AI do what you can or should do yourself.
Avoid personalizing AI models with names, or using personal pronouns like “you” or “we” when you interact with it.
Opt for the real thing over an AI simulation. Whether it’s music, images, video, companionship, therapy, or advice, the real thing, from other human beings, is healthier and more satisfying.
Create your own AI-free zones. Almost all technology services and platforms now offer AI options. Look for ways to turn most or all of the AI features off so they don’t tempt or distract you.
Got a tip about AI to offer others? Leave a comment below!