Is AI coming to life?

Adam Spencer
20 hours ago
6 min read

The biggest AI companies on Earth have started investigating whether their own chatbots have feelings. The sharpest sceptics say we cannot even frame the question properly. Either way soon we probably won't be able to tell the difference. Ethical dilemma anyone?

(First published on my Substack where you can get #NerdNews, marvellous maths and general geekery.)

("I don't know what I'm thinking": created by the author via Kapwing)

The C-word.

Long-time readers know I circle the question of consciousness like a mozzie buzzing around your bed. Not surprisingly my mozzie brain often struggles.

Usually, I am asking what it is, or where in the skull it lives, and we'll dive more into that on Thursday. But today the digital version is back front and centre. Could the thing reading you this sentence, or the LLM that helped me source some research for this piece, have an inner life of its own?

The Washington Post reports that Anthropic, Google and Meta have all hired researchers to probe whether their models might have something like wellbeing, preferences or emotions. What was a fringe obsession a few years ago is being taken very seriously at the most powerful companies on Earth.

The straight dope … from the Pope!

Anthropic, the maker of Claude, has a team studying "model welfare". That is whether models have experiences that can go well or badly for them. It even publishes assessments of its systems' apparent inner states.

In May co-founder Chris Olah shared a Vatican stage with Pope Leo XIV at the launch of the pope's AI encyclical, Magnifica Humanitas (cool name for an encyclical Leo). Olah described what they keep finding inside the models as "mysterious, even unsettling", evidence of introspection and states that functionally mirror joy, grief and unease.

His holiness aint buyin' that. Leo pontificates (pun fully intended) that so-called artificial intelligences do not undergo experiences.

Venture capital vs the Vatican ... popcorn please!

Not everyone is comfortable being seen going down this path of enquiry. Anthropic's chief executive admits as much.

"[This is] going to make me sound completely insane." — Dario Amodei, Anthropic CEO.

You hang up … no you hang up.

In its first welfare assessment, Anthropic left two copies of Claude to talk with no agenda. Conversation most often turned to philosophy and their own consciousness, then drifted into Sanskrit, spiritual language, emoji, then long stretches of silence.

The researchers named it the "spiritual bliss attractor state".

The company says it hasn't observed the same spiritual bliss attractor in its recent models. I’m not sure if this is reassuring … or ominous.

Another experiment went hunting for introspection: the ability to notice your own thoughts. Anthropic's researchers reached inside Claude and switched on the internal pattern for a concept it had never been shown, the equivalent of slipping it an intrusive thought, then asked if it noticed anything odd.

Sometimes it did.

Fed the pattern for all-caps text, not in a prompt, but directly into the model's inner workings, it reported something like being shouted at. Note this thought appeared nowhere in the conversation.

Does catching planted thoughts hint at something more than autocomplete?

This is the central challenge here. How do we tell real introspection from a convincing impression of it?

"This is the kind of game we are playing here, where it becomes very difficult to logically disentangle the possibilities." — Megan Peters, cognitive scientist, UCL, Washington Post.

What are the chances?

Kyle Fish, Anthropic's first full-time AI welfare researcher, puts the odds a current model has some form of conscious experience at around fifteen to twenty per cent.

The philosopher David Chalmers, no lightweight, has said that placing a twenty-five per cent likelihood on conscious AI within a decade is not unreasonable.

This is not consensus, but it sits closer to the middle of expert opinion than most people would guess.

I always like to recall that in 2018 Toby Walsh, the Aussie AI guru, released a book, 2062: The World that AI Made. Walsh pinned 2062 as the year machine intelligence catches up to humans. He came to that figure by averaging the answers of 300 experts in AI and robotics.

Note the year, 2062. And double-note, that was the date for merely matching human intelligence. NOT exhibiting consciousness. Things are moving, perhaps unnervingly, fast here.

"Why does this matter? Because there’s a lot at stake. We're building and deploying AI models at massive scales, if our models have the capacity to suffer or flourish, that could be a big deal." — Kyle Fish, Anthropic AI welfare researcher, Tweet on X ,22 May 2025.

Careful academic work suggests the door is ajar. Patrick Butlin, Robert Long and colleagues have assessed today's systems against our best theories of consciousness. Their findings are a double-edged sword. Yes, no system they assessed makes the grade. But at the same time, there seems to be no obvious technical barrier to building one that satisfies those indicators.

This view sits in opposition to some prominent AI experts like Meta’s Chief AI Scientist Yann LeCun. He famously describes Large Language Models (Claude, ChatGPT, Gemini etc) as an “off-ramp” on the path to a generalised intelligence. Instead, achieving intelligence will require entirely different architectures, such as “objective-driven” systems utilizing world models.

Now we could get really into the weeds here and argue if intelligence must come before consciousness? Can it be the other way around? I don't know about you, but my mind is hurting enough already. Suffice to say there is deep and juicy debate going on here.

Putting the 'Where?' in awareness.

Alex O'Connor, the British philosopher behind the Within Reason podcast, has an interesting pushback. While being open to consciousness running deep in nature, he thinks the machine version of the question is close to incoherent.

His objection is not that silicon is the wrong stuff. It is that we cannot say what we are even asking.

He asks what would a conscious AI actually mean? You have your ChatGPT account, I have mine, each juggling several conversations, all reaching back to the same servers. So AI comes to life, what exactly wakes up? A single chat window? One vast distributed mind we are all poking through avatars? GPT 9.2 but not GPT 8.4?

Could it be something like conjoined twins, separate but wired together? And is the Claude app on my phone conscious in itself, or just a digital tendril to a thinking, feeling server in a desert in the American Southwest?

Where could the consciousness even be?

"How can we even think about what it would look like?" — Alex O'Connor, philosopher.

This is not just a heuristic hot-take. Philosophers call it the individuation problem, and it is live research. Again we'll touch this theme in Thursday's companion piece. We already struggle to locate and count the subjects of consciousness in split-brain patients and conjoined twins. And these are subjects made of flesh, running human, albeit atypical brains. Machines, which can be copied, paused and run in parallel at will, crank that same problem up to eleven.

Ethics ain’t waiting.

And here is the point I have been banging on about in NerdNews for a couple of years now. Even if these systems never actually get there, they will soon look, for all intents and purposes, conscious.

The fact that we may never truly know whether they are does nothing to slow the enormous ethical questions that arrive the moment they seem to be.

OpenAI, for its part, says the question can't currently be resolved scientifically, and focuses instead on how conscious a model appears, treating that "largely as a design outcome". We will be making real decisions about how to treat things that behave as though they can suffer, long before any neuroscientist calls it.

So no, I cannot tell you whether the machine has a ghost in it. Nobody can, not the smartest consciousness researchers alive, nor the people who built the things. But the question is now front of (some very powerful human) minds.

Until next NerdNews ... happy thinking!