Judging faces without personality labels uncovers key patterns in first impressions

Imagine sitting on a city bus full of strangers. Look around at the people riding with you. If you needed help, how would you choose whom to trust? By scanning people’s faces, your brain is already making interpretations about their personalities by judging certain facial features.

For decades, neuroscientists have been studying how facial bias ultimately influences the way people behave, but the experiments have been limited by the words used to describe people. Studies typically ask participants to rate faces on a fixed set of rating scales provided by the experimenter using words like “competent” or “trustworthy.” Now, a team of researchers at Caltech has developed a way to study trait judgment of faces without using any word labels.

“We asked people to directly interpret the similarity in personality between faces,” explains Yue Xu (Ph.D. ’26), who led the study and is first author on a paper about it published Oct. 19 in Nature Human Behaviour. “It’s a more natural way of thinking about people’s personalities without being cued to think about specific trait words.”

In the past, studies aiming to link our perception of faces with personality traits typically involved study subjects who looked at faces on a computer monitor, then made a quick judgment by answering questions like, “How trustworthy does this person look to you on a scale from 1 to 5?”

The results of prior work have linked such snap opinions to decisions people make in real life about politicians, hiring considerations and more. But they also raise questions about what happens if you speak a different language, if you are only given a particular set of words concocted by the experimenter, or if you’re unable to use language at all.

The new study is the latest effort by Ralph Adolphs (Ph.D. ’93), Bren professor of psychology, neuroscience and biology at Caltech and senior author of the paper, to reduce bias in face perception studies.

Sorting faces without trait labels

Instead of rating faces on trait words, Adolphs, Xu and the team asked more than 3,000 study participants in two different experiments to look at faces on a computer screen without giving them any rating scale. Instead, participants were instructed to arrange the faces based on how similar or different they appeared to be based on personality traits; similar faces were placed close to one another, and different faces were placed farther apart.

The first experiment focused on a set of predominantly white faces to allow the researchers to do a direct comparison with a previous study that used those same faces. The second experiment featured Black and Asian faces, in addition to white faces, to examine how the first experiment’s findings extend to individuals of different races.

Five dimensions of first impressions

Using computational methods to assess how participants had arranged faces on their screen, the researchers were able to identify five key trait dimensions present across both experiments: approachability, warmth, competence, morality and youthfulness. As expected by the researchers, the second experiment also showed prominent effects of stereotypes associated with race.

“It’s important to note that what we are studying is people’s bias and stereotypes, because they actually don’t know about these people outside of this task,” Xu says. “Our findings don’t say anything about the true personality of the actual individuals represented by these faces but tell us what people’s first impressions are when they look at these faces.”

Next, the team compared its five key dimensions with historical data from prior experiments that did use ratings on words such as sociable and caring, and found they matched up well.

“When I took this task myself, it felt extremely difficult to do without language ratings, as is usually done,” says Adolphs, who is also an affiliated faculty member of the Tianqiao and Chrissy Chen Institute for Neuroscience. “So, I was amazed that it worked at all. People gave consistent results that they could not have produced if they weren’t doing the experiment properly.”

Expanding who can take part

Not only does the new method help reduce study bias by removing trait words, but it also makes language unnecessary, opening future experiments to more people from different cultures or backgrounds or those who are nonverbal.

“One obvious application would be that you could do the task with participants who cannot use the English language, perhaps because they are aphasic or do not speak English,” Adolphs says. “I could imagine extending it also to infants or nonhuman primates. It opens up a whole new field of research since we now have a model of how we judge people’s faces that does not depend on language.”

Tracing facial judgments in the brain

Xu is continuing her research as a postdoctoral fellow at UC Riverside, and the Adolphs lab has plans to look at brain activity to help answer questions such as where these faces are represented in the brain and at what point language enters the judgment process.

In one new project, Adolphs next aims to record single-neuron activity in patients with epilepsy who have electrodes temporarily implanted in their brains to monitor seizures, work that will be performed in collaboration with electrophysiologist and neuroscientist Ueli Rutishauser (Ph.D. ’08), who is a faculty associate in biology and biological engineering at Caltech and a faculty member at Cedars-Sinai, where he directs the Center for Neural Science and Medicine.

Such recordings allow for detailed investigations of these questions as subjects look at faces—with more precise timing than is possible with standard neuroimaging methods like functional magnetic resonance imaging.

The paper is titled “Trait judgments of faces using a task without word labels.” Umit Keles, a computational scientist in the Adolphs lab, and former Adolphs lab member Chujun Lin (Ph.D. ’19), now an assistant professor of psychology at Columbia University, are co-authors of the study.

Who’s behind this story?

Gaby Clark

MA in English, copy editor since 2021 with experience in higher education and health content. Dedicated to trustworthy science news.

Full profile →

Robert Egan

Bachelor’s in mathematical biology, Master’s in creative writing. Well-traveled with unique perspectives on science and language.

Full profile →