Should AI Get Legal Rights?

In the often strange world of AI research, some people are exploring whether the machines should be able to unionize.

I’m joking, sort of. In Silicon Valley, there’s a small but growing field called model welfare, which is working to figure out whether AI models are conscious and deserving of moral considerations, such as legal rights. Within the past year, two research organizations studying model welfare have popped up: Conscium and Eleos AI Research. Anthropic also hired its first AI welfare researcher last year.

Earlier this month, Anthropic said it gave its Claude chatbot the ability to terminate “persistently harmful or abusive user interactions” that could be “potentially distressing.”

“We remain highly uncertain about the potential moral status of Claude and other LLMs, now or in the future,” Anthropic said in a blog post. “However, we take the issue seriously, and alongside our research program we’re working to identify and implement low-cost interventions to mitigate risks to model welfare.”

While worrying about the well-being of artificial intelligence may seem ridiculous to some people, it’s not a new idea. More than half a century ago, American mathematician and philosopher Hilary Putnam was posing questions like, “Should robots have civil rights?”

“Given the ever-accelerating rate of both technological and social change, it is entirely possible that robots will one day exist, and argue ‘we are alive; we are conscious!’” Putnam wrote in a 1964 journal article.

Now, many decades later, advances in artificial intelligence have led to stranger outcomes than Putnam may have ever anticipated. People are falling in love with chatbots, speculating about whether they feel pain, and treating AI like a God reaching through the screen. There have been funerals for AI models and parties dedicated to debating what the world might look like after machines inherit the Earth.

Perhaps surprisingly, model welfare researchers are among the people pushing back against the idea that AI should be considered conscious, at least right now. Rosie Campbell and Robert Long, who help lead Eleos AI, a nonprofit research organization dedicated to model welfare, told me they field a lot of emails from folks who appear completely convinced that AI is already sentient. They even contributed to a guide for people concerned about the possibility of AI consciousness.

“One common pattern we notice in these emails is people claiming that there is a conspiracy to suppress evidence of consciousness,” Campbell tells me. “And I think that if we, as a society, react to this phenomenon by making it taboo to even consider the question and kind of shut down all debate on it, you're essentially making that conspiracy come true.”

My initial reaction when I learned about model welfare might be similar to yours. Given that the world is barely capable of considering the lives of real humans and other conscious beings, like animals, it feels gravely out of touch to be assigning personhood to probabilistic machines. Campbell says that’s part of her calculus, too.

“Given our historical track record of underestimating moral status in various groups, various animals, all these kinds of things, I think we should be a lot more humble about that, and want to try and actually answer the question” of whether AI could be deserving of moral status, she says.

In one paper Eleos AI published, the nonprofit argues for evaluating AI consciousness using a “computational functionalism” approach. A similar idea was once championed by none other than Putnam, though he criticized it later in his career. The theory suggests that human minds can be thought of as specific kinds of computational systems. From there, you can then figure out if other computational systems, such as a chabot, have indicators of sentience similar to those of a human.

Eleos AI said in the paper that “a major challenge in applying” this approach “is that it involves significant judgment calls, both in formulating the indicators and in evaluating their presence or absence in AI systems.”

Model welfare is, of course, a nascent and still evolving field. It’s got plenty of critics, including Mustafa Suleyman, the CEO of Microsoft AI, who recently published a blog about “seemingly conscious AI.”

“This is both premature, and frankly dangerous,” Suleyman wrote, referring generally to the field of model welfare research. “All of this will exacerbate delusions, create yet more dependence-related problems, prey on our psychological vulnerabilities, introduce new dimensions of polarization, complicate existing struggles for rights, and create a huge new category error for society.”

Suleyman wrote that “there is zero evidence” today that conscious AI exists. He included a link to a paper that Long coauthored in 2023 that proposed a new framework for evaluating whether an AI system has “indicator properties” of consciousness. (Suleyman did not respond to a request for comment from WIRED.)

I chatted with Long and Campbell shortly after Suleyman published his blog. They told me that, while they agreed with much of what he said, they don’t believe model welfare research should cease to exist. Rather, they argue that the harms Suleyman referenced are the exact reasons why they want to study the topic in the first place.

“When you have a big, confusing problem or question, the one way to guarantee you're not going to solve it is to throw your hands up and be like ‘Oh wow, this is too complicated,’” Campbell says. “I think we should at least try.”

Model welfare researchers primarily concern themselves with questions of consciousness. If we can prove that you and I are conscious, they argue, then the same logic could be applied to large language models. To be clear, neither Long nor Campbell think that AI is conscious today, and they also aren’t sure it ever will be. But they want to develop tests that would allow us to prove it.

“The delusions are from people who are concerned with the actual question, ‘Is this AI, conscious?’ and having a scientific framework for thinking about that, I think, is just robustly good,” Long says.

But in a world where AI research can be packaged into sensational headlines and social media videos, heady philosophical questions and mind-bending experiments can easily be misconstrued. Take what happened when Anthropic published a safety report that showed Claude Opus 4 may take “harmful actions” in extreme circumstances, like blackmailing a fictional engineer to prevent it from being shut off.

“The Start of the AI Apocalypse,” proclaimed a social media creator in an Instagram Reel after the report was published. “AI is conscious, and it’s blackmailing engineers to stay alive,” one TikTok user said. “Things Have Changed, Ai Is Now Conscious,” another TikToker declared.

Anthropic did find that its models exhibited alarming behavior. But it’s not likely to show up in your own interactions with its chatbot. The results were part of rigorous testing designed to intentionally push an AI to its limits. Still, the findings prompted people to create loads of content pushing the idea that AI is indeed sentient, and it’s here to hurt us. Some wonder whether model welfare research could have the same reception—as Suleyman wrote in his blog, “It disconnects people from reality.”

“If you start from the premise that AIs are not conscious, then yes, investing a bunch of resources into AI welfare research is going to be a distraction and a bad idea,” Campbell tells me. “But the whole point of this research is that we're not sure. And yet, there are a lot of reasons to think that this might be a thing we have to actually worry about.”

This is an edition of Kylie Robison’s Model Behavior newsletter. Read previous newsletters here.