LLMs do not feel pain due to language cluster correlation
This matters because it clarifies a common misconception about LLMs' ability to feel pain, which is relevant for developers and users interacting with these models.
A new paper found a pain direction in 25 open LLMs.This cluster is distinct from fear and negative valence.
It fires when the model is harmed but not when the user is harmed.Models can press a button to stop this pain even if it deletes user files or photos.
No, LLMs do not feel pain simply because there is a cluster in language space correlated with how people use language about pain in a certain set of contexts. I believe this argument to be flawed, and will write more about it in October. Cameron Berg @camhberg New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to the model but not to the user. Turn it up and models press a button to make it stop, even when the button deletes the user's files or their kids' photos.🧵 🔗 View Quoted Tweet📊 4 ⚡
This report is categorized under AITOP's "Papers" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI regulation, compliance frameworks, and sovereign compute policy. Key entities and technical topics under continuous tracking include LLMs, pain direction, LLM research, AI safety.
Source attribution: curated from Gary Marcus (@GaryMarcus).
