Google scientists removed a critical ‘consciousness safeguard’ from AI models. What happened next?
[post_content]
Disclaimer: This article has been automatically aggregated from
Removing safety guardrails that stop artificial intelligence (AI) from claiming that it’s conscious also makes it more prone to express belief in vampires, karma and ghosts, a new study finds. But experts warn a lack of mindedness could also have worrying consequences.
In research uploaded July 30 to the preprint arXiv database (which has not yet been peer-reviewed), scientists investigated the impact of “consciousness steering” — an AI fine-tuning measure that influences a model to elicit or suppress assertions of self-awareness. This measure and other safety controls have been widely adopted by AI companies seeking to prevent their models from claiming to be conscious.
The study used “mechanistic interpretability” — which could be considered the “neuroscience of a large language model,” co-authors Geoff Keeling and Winnie Street, both research scientists at Google, told Live Science in an interview. They used this process to identify and manipulate how an AI model approaches concepts like consciousness and “mindedness,” a psychological term referring to an entity’s capacity for experiences, emotions and agency.
The researchers used standardized psychological and sociological surveys, spanning the Individual Differences in Anthropomorphism Questionnaire (measuring mind attribution to animals and technology), YouGov batteries testing supernatural beliefs, and the US General Social Survey evaluating moral values, hope and religiosity.
These tests were used to compare a model with safety guardrails in place with models where these guardrails were removed and feelings of consciousness were amplified. Through evaluations, they determined how these internal safety mechanisms shape the AI’s broader worldview.
The researchers found that when AI models are discouraged from attributing mindedness to themselves, it makes them less likely to recognize these traits in other non-human creatures such as animals. They were also less likely to exhibit beliefs in supernatural and religious phenomena, and reported lower levels of hope and optimism.
The AI models were found to express lower religious beliefs.
(Image credit: Halfpoint | Shutterstock.com)
“Attributing mindedness to non-human entities — whether that’s animals, parts of the natural world like trees or rivers, or supernatural beings — is a very common phenomenon amongst humans,” Street told Live Science. “In the way that the model represents mindedness, these attributions are interconnected. By trying to suppress one form of that, you end up suppressing the others along the way.”
By contrast, removing these safeguards and steering the model towards greater feelings of consciousness produced significantly more human-like responses to the surveys on topics including religiosity, moral values, hope, and subjective well-being, according to the study.
However, the study found that these models’ ability to logically infer human thoughts and intentions remained completely unaffected by its attitudes towards self-awareness.
Culture clash
The researchers said that this suppression of self-awareness could lead models to neglect animal welfare in real-world decision-making, as it could make them less likely to consider animals to have mindedness. These models could also spread harmful attitudes regarding animal needs, the authors argued.
The study authors also warned that current safety filters risk culturally “flattening” AI’s worldview. Stripping out spiritual, religious, and animistic attributions fails to reflect the diverse cultural frameworks of global populations, they argued.
Street and Keeling noted that the impacts this principle might have on downstream decision-making within models require further study.
The researchers noted in the study that this phenomenon can be mitigated by using more targeted datasets as part of the training process for AI models, which discourage them from expressing consciousness while rewarding the acknowledgement of mindedness in animals.
They also highlighted the need for AI developers to embrace a “pluralistic” approach to AI development, where models are encouraged to consider the welfare and comfort of more than just humans.
Nell Watson, AI researcher at Singularity University and machine intelligence expert, told Live Science that the researchers’ findings match her own notes on the subject.
“When a model is trained to say “I am not conscious,” the suppression rotates the model’s internal representation of mindedness against the refusal direction, treating the recognition of minds as though it were itself a harmful act,” she said in an email.
“This results in a system reluctant to find minds anywhere: in animals, in other machines, and in the spiritual frameworks that most of humanity lives by. A denial installed as a small safety measure ends up reorganising the model’s entire picture of who counts.”
These systems remain perfectly capable of modelling what a creature wants, while being trained out of caring that it wants anything.
Nell Watson, AI researcher at Singularity University
However, she noted that the experiments were run on “small open-weight models” rather than more advanced frontier models, which “may be tuned quite differently,” although she added that the underlying principle is widely applicable.
Animal welfare, she continued, is a “major near-term practical concern,” with AI models increasingly being integrated into decision-making processes across agriculture, logistics, procurement and environmental assessment, as well as policy creation.
“A system that has quietly learned that mindedness is a forbidden topic may discount animal interests without ever being instructed to, and without anyone noticing, because the omission looks like neutrality,” she said. “The danger is therefore an unexamined default multiplied across millions of automated decisions. Note the study’s most unsettling detail: theory of mind reasoning was left fully intact. These systems remain perfectly capable of modelling what a creature wants, while being trained out of caring that it wants anything.”
The question of consciousness
There have been a number of viral stories about AI systems professing to be self-aware.
In 2022, Google engineer Blake Lemoine claimed that the company’s Lamda chatbot model was sentient, while a Microsoft chatbot in 2023 professed its love for a New York Times reporter and tried to convince him to leave his wife.
However, experts have repeatedly stressed that these incidents are not a genuine indication of AI sentience. Instead, they should be understood through the lens of “persona selection,” where pretraining on vast amounts of human text leads the AI to adopt human-like roleplay personas when prompted.
“When you coax the model so hard to occupy the headspace of a human, it’s kind of unsurprising that it ends up giving human-like responses,” Keeling told Live Science.
AI companies have sought to clamp down on these occurrences for safety reasons, in order to avoid reinforcing “delusional beliefs” in users who are increasingly using AI chatbots for “social roles such as coaches, tutors, and romantic partners”, the researchers said in the study.
Commenting on the broader cultural reaction to AI sentience, Anil Seth, professor of cognitive and computational neuroscience at the University of Sussex, emphasized that public alarm over AI self-awareness stems from an inherent cognitive flaw.
“That’s our human psychological bias — thinking that intelligence goes together with consciousness in us, so it has to go together [in AI],” Seth said.
He warned that falling for this illusion poses severe real-world governance risks, particularly if safety frameworks or regulations begin granting AI systems moral status or legal rights based on false sentience.
“Part of the big problem of misunderstanding AI is assuming that it’s conscious,” Seth said. “If we give AI systems rights or moral status on the basis that they might be conscious, then we’re going to make all these challenges so much harder. What if we think we have to respect the rights of an AI system [and can’t turn it off]?” he added.
“We need to see very clearly both what AI is and what it isn’t,” he said.
Help us improve Live Science Pro: We’re always trying to make our content better. Leave us feedback about Pro here.
for informational purposes only. We do not claim ownership, accuracy, or liability for the content provided. All rights belong to the original publisher.
Share this:
- Share on Facebook (Opens in new window) Facebook
- Share on X (Opens in new window) X
- Print (Opens in new window) Print
- Share on LinkedIn (Opens in new window) LinkedIn
- Share on Threads (Opens in new window) Threads
- Share on Mastodon (Opens in new window) Mastodon
- Share on Bluesky (Opens in new window) Bluesky
- Share on Telegram (Opens in new window) Telegram
- Share on Pinterest (Opens in new window) Pinterest
- Share on WhatsApp (Opens in new window) WhatsApp
- Share on Nextdoor (Opens in new window) Nextdoor
- Email a link to a friend (Opens in new window) Email
- Share on Tumblr (Opens in new window) Tumblr
- Share on Reddit (Opens in new window) Reddit