Phrenology is Back, Baby! AI is VERY Good at Making Predictions From Face Scans

Join Malcolm and Simone as they delve into the controversial and data-heavy topic of predicting personal traits from facial features. From the discredited study of phrenology to modern AI research, they discuss the potential and ethical implications of determining criminality, political views, sexuality, aggression, and even socioeconomic status just by looking at someone's face. With references to various scientific studies and a touch of humor, this episode offers a thought-provoking look at the intersection of technology, genetics, and human behavior. [00:00:00]

Malcolm Collins: Hello Simone. Today we are gonna be doing a very spicy episode on and very, very data, data heavy episode. On phonology for people who dunno was the study of people's skull shape and whether it affected outcomes or intelligence or anything like that. Get out your calipers.

Simone Collins: Ladies and gentlemen. This is gonna be fun.

Speaker 2: She's the sloping brow and cranial bumpy to the career criminal. Ah, sir. Ality was dismissed as quackery 160 years ago. Of course, you'd say that you are the brain pan of a stagecoach tilter.

Simone Collins: They

Malcolm Collins: used to put these big, like clamps on your head to like, measure and like certain, like rich guys, you'd go to their house and they'd like do this to you, to like, oh, it was

Simone Collins: like a party trick. Yeah. Like, you'd have everyone over and someone would get out the calipers and they'd be like, yeah.

And they had like little models of, of, of heads that would show like the different, you know.

Malcolm Collins: Here's what indicates this. One of our very wealthy friends is, is into this these days as well. Getting back into phenology. Yeah,

Simone Collins: a little.

Malcolm Collins: But apparently he is not wrong. Apparent. Well, so phenology does not appear to [00:01:00] work to my knowledge.

I, I can look more into it, but that is not what this episode is on. This episode is on just being able to tell a person's politics, sexuality, criminality. Any number of things from facial features. Can you actually like, even, even worse than phenology, 'cause phenology you need are the calipers. You need all that.

Speaker 3: Yeah.

Malcolm Collins: Invasive. What can you judge about a person just from looking at their face and how accurately can you judge?

Speaker 3: Hmm.

Malcolm Collins: And if you are like, well, okay, he's gonna point to some minor statistical differences, not that big. Let's just go right into criminality, right? Oh, because this may one of the, okay.

The spiciest. Okay. So fine. There was a study in 2016 called Automated Inference on criminality using face images. It used a supervised machine learning for classification CNNs on [00:02:00] 1,856 Chinese male ID photos, seven 30 convicted criminals and 1,126 non-criminals. It claimed an 89.51% accuracy in determining criminals.

Simone Collins: Oh no,

Malcolm Collins: that's around a nine. That's, that's 0.5. Less than 0.5 away from a 90% accuracy rate at determining who's a criminal. Oh.

Simone Collins: Just from looking at their face.

Malcolm Collins: That is probably more accurate than any other way you could determine a criminal. In fact, I bet court cases aren't even that accurate in determining criminals.

Yes. Seriously, meaningly that the mistake could be in the court case and not the machine. So if you're wondering how, how can you tell a criminal it identified structural face shield features like narrow eye, corner distance, 4%, norm distance, higher lip, cur curvature, 23.4. Wait, so the more

Simone Collins: curvy the lips, the more criminal?

Malcolm Collins: Yes. And smaller [00:03:00] nose, mouth angle, 19.6% as predictive. Which I find wild. So, and, and, and, and note this isn't the, the only paper to do this, right? So we also have predicting criminality from facial images. This was a 2020. Paper that was covered in Wired that had to be redacted and was never published because everybody freaked the F out.

But it built software that based on facial recognition stuff that was a single image could, was 80% accuracy depict the, the criminality of an individual. And you can be like, okay, well how good is an average person at this not using ai. 'cause obviously AI is gonna be like super humanly good at this.

The a a paper titled, the accuracy of inferences about criminality based on facial appearance in 2010 showed only a 53% probability. Mm-hmm. So humans are litter better than half at this. Butis are incredibly good at this. And [00:04:00] of course, a bunch of people freaked the F out after this happened. And I think we're gonna see this like as.

We DOIs on more of this stuff and we'll, we'll go into some other areas where humans are very good at determining things about people based on their face. But I wanted to start with a criminality thing. 'cause like the phenologist dream has come true in 2024. Five. Feels very much like the dream of the nineties came true in Portland, right?

Like the dream of the Phenologist came true in 2025. With AI able to immediately determine just from looking at someone if they're a criminal, but anyway, so obviously progressives freaked the F out when this happened. Right? So, there was a a, a paper, the. Criminal, the criminality from face illusion.

2020 trends in cognitive sciences. Reviewed claims like the 2016 study and found no valid evidence for predicting criminality from faces. Arguing that algorithms detected artifacts like smiling bias and dataset rather [00:05:00] than innate traits criticized as pseudo silence reviving ology. Here's the problem.

Okay. First they claimed that it was the non-criminals who were smiling. Yet we actually know that the uplifted lips was the 23% tied to criminality and the original dataset, right, which implied it was the criminalities who were doing this. The, the, the worst is that I imagine what they're saying here.

They're saying that from a facial expression was a 90% AC accuracy. You could determine who's a criminal. That's just as bad, first of all, but also stupid and obviously wrong, right? I can't determine, and I don't think an AI could only boiling it down to facial expressions, who's a criminal just from the facial expressions they're wearing.

That is significantly less plausible than me being able to determine it from facial features, which are correlated with genetics, which we know [00:06:00] from polygenics cores correlate to criminality. I.

Malcolm Collins: But most of this paper, if you go over it, is just whining about phenology and racism and blah, blah, blah, blah, blah, blah, blah. So obviously they are at a, a freakout about this.

Any thoughts before we go further? Simone?

Simone Collins: I feel like Palantir must already be doing this as much as other people are freaking out. I can't imagine a world in which I, this is not being used.

Malcolm Collins: Yeah. I, I, by the way, love the progressive, like, pushback on this. It's like, ugh. It, 90% accuracy. It must have been looking at facial in the other paper.

80% accuracy. I oh, we don't want these in courts. Imagine the freakout, right? You just, you just put on our minority report in courts. You walk into the court and it's like, this is criminal, non-criminal, criminal, [00:07:00] non-criminal.

Speaker 4: When the Precogs declare a victim and a killer, their name is embedde