The good-government fantasists among us would like to believe that we make political choices on rational grounds, through careful weighing of the issues and sophisticated analysis of the candidates’ qualifications (this seemed to be Jeb Bush’s concept). But no one watching an American election—particularly this year's carnival of a contest—believes that this is actually what takes place. And so the question stands: How do voters come to make their political choices? 

Nobel Prize-winning psychologist Daniel Kahneman is the de facto dean of this field. Five years ago, he published a book called Thinking Fast and Slow, which encapsulated his decades of research on human cognition into a simple insight, embodied by the title of the book. There are two systems, he posited, one conscious and slow and involved, the other devoted to jumping to conclusions, mostly subconsciously. Superficial impressions and simplifications inevitably fill in gaps where we may lack actual evidence.

This effect is rife in the voting booth. The social scientists Alexander Todorov, of Princeton, and Christopher Olivola, of Carnegie Mellon’s Tepper School of Business, have focused on the role of appearance in how we come to our electoral choices. Based on their years of research, they contributed the analysis of faces of 2016 presidential contenders that accompanies this article. The findings codify the subconscious but powerful preconceptions that people might bring to their political decision-making.

Todorov and Olivola have been studying these effects for more than a decade. “The reality is that people strive to be coherent and rational in their decisions,” Olivola said, “but the amount of information they have to deal with is overwhelming, even tiring. It’s not like there are three pieces of information to know and star ratings—plus there’s contradictory information—so they fall back on heuristics, first impressions.”

In other words, the human brain reclines onto things it is good at—like processing faces. “Within one-tenth of a second,” Olivola told me, “you look at a person’s face and—generally speaking—recognize gender and ethnicity, and you get a rough idea of their age.”

Not only that. “Showing a face for less than 100 milliseconds is sufficient for people to make all kinds of judgments,” Todorov said, “like whether the person is trustworthy, competent, aggressive, and so on.” Certain sorts of biases and judgments seem to be programmed into our brains. 

Which has ramifications in our politics. “If you ask voters, ‘What do you care most about in a politician or a leader?’” Olivola said, “competence is the one that people say they care most about.”

To tease out these effects and begin to measure them, Todorov conducted a study in which he showed pairs of photographs to around 1,000 people, and asked them to rate who looked more competent.

Subjects were shown two pictures side by side, pictures of candidates for office—but the subjects didn't know that.
Subjects were shown two pictures side by side, pictures of candidates for office—but the subjects didn't know that.
Christopher Olivola/Carnegie Mellon

The photos Todorov showed were portraits of rival candidates for Senate races, gubernatorial races, and House of Representatives races from 1996-2006—not that the test subjects knew the portraits were of politicians. (Whenever a subject did recognize the pictures, that particular rating wasn’t included in the experiment.) “You have these participants, these human judges, seeing a pair of photos of anonymous individuals,” Olivola said. But their responses to the question “Which person looks more competent?” reliably predicted the outcome of each election, anywhere from 66 to 73 percent of the time.

In 2010, Olivola and Todorov published a study in the Journal of Nonverbal Behavior that showed that “rapid judgments about the personality traits of political candidates, based solely on their appearance, can predict their electoral success.”

And very rapid judgments were particularly reliable.

“What you see,” Olivola said, “is that the strongest predictor is the extent to which a candidate looks more competent than his or her rival. It stands even after you control for how likable, how attractive, how babyfaced they look—or gender, political party, incumbency status, age, and ethnicity. Of all these facial traits, the consistently strongest of all these stereotypes is competence. It consistently comes out, in Western elections, as the strongest predictor of success.” (South Korean elections, he said, don’t seem to be as well predicted by facial competence stereotypes.)

Todorov noted that studies conducted subsequently by Gabriel Lenz and Chapell Lawson, political scientists, then at M.I.T., showed that watching television only magnifies the effect of first impressions.

Using a hefty set of empirical data, Todorov created a quantitative model that roughly summarizes people’s stereotypes about faces. He and Olivola used a computer program to generate hundreds of emotionally neutral faces that participants then ranked on dimensions including competence and dominance, which allowed them to build a facial model that they could then manipulate on a scale of competence (among other factors).

Competence is the predominant factor by which people make snap judgments when assessing candidates’ faces.
Competence is the predominant factor by which people make snap judgments when assessing candidates’ faces.
Christopher Olivola/Carnegie Mellon

They also can exaggerate any face “along the competence dimension.” 

Researchers were able to manipulate the competence of their facial models, and tease out some of the factors that contributed to subjects’ perceptions of competence.
Researchers were able to manipulate the competence of their facial models, and tease out some of the factors that contributed to subjects’ perceptions of competence.
Christopher Olivola/Carnegie Mellon

Because the traits were reverse-engineered based on what people said they perceived, it can be hard to exactly describe the set of attributes that make a face look competent (or not). “This represents an aggregation of people’s stereotypes,” Olivola told me. In their study, he and Todorov wrote:

Exaggerating faces in the positive direction of the competence dimension resulted in more mature-looking and attractive faces. Faces became less round, the distance between the eyebrows and the eyes decreased, the cheekbones were higher, and the jaws became more angular. Consistent with these changes, participants judged more competent faces to be more attractive and less babyfaced.

Olivola said that, for men, facial competence was associated with looking more masculine and less feminine, whereas, in his words, “women who from their faces look too ‘feminine’ or too ‘masculine’ are judged as less competent-looking than women who are at neither extreme.”

With scientific inquisition in mind, we asked Olivola and Todorov to analyze the faces of the 2016 presidential candidates with their model, on software called FaceGen. We included the field of 14 major candidates that stood as the Iowa caucuses began. Here are the results.

Carly Fiorina looks most competent

In the measure of competence—that key factor in Todorov’s and Olivola’s study of political impressions—Carly Fiorina, the former CEO, scored the very highest. Of those still in the race, Marco Rubio does best.

Andre Tartar/Bloomberg

Donald Trump looks most extroverted

Donald Trump’s face, according to their analysis, came off as most extroverted—and by a good deal.

Andre Tartar/Bloomberg

Hillary Clinton looks most trustworthy

And in terms of who is most trustworthy-looking, Hillary Clinton wins—this despite the fact that polls show many Americans regard her as “dishonest” and “untrustworthy.” (Olivola explained that “there is a bias to view feminine faces as more trustworthy-looking” than masculine faces.)

Andre Tartar/Bloomberg

Chris Christie’s face doesn’t score well

Clinton scored best of the 14 candidates when it came to being the least threatening, least frightening, and least mean. In contrast, Chris Christie, who is no longer in the race, scored as the most threatening, frightening, and mean, and the least trustworthy.

But he also scored as most dominant. Olivola said that appearing dominant could “help or hurt a candidate, depending on the context. Studies have shown that people prefer dominant-looking candidates during times of war.”

Andre Tartar/Bloomberg

The two front-runners, overall, score best

In a composite of desirable traits (likable, extroverted, dominant, trustworthy, competent, attractive), the front-runners scored high: Trump led, with Clinton just behind. Fiorina and Martin O’Malley came next—candidates from opposing parties who have since exited the race. Below O’Malley came Ted Cruz, Rubio, and Bernie Sanders. The top five candidates in the race, that is, fill five of the top seven slots. Factoring in the undesirable facial traits as well, Trump and Clinton still come out on top.

Andre Tartar/Bloomberg

I asked if it was common that all the candidates included have faces that rank positively in terms of competence. Olivola said it was to be expected.

“These candidates have been vetted, in a way, throughout their lifetime. These are people who have been selected and in some ways groomed to be candidates,” he said. “I wouldn’t be surprised if explicitly—or at least implicitly—parties or others selected candidates in part based on their looks.”

He noted that these candidates may have more “winning” faces than the average person: faces that are “quite dominant and attractive” compared to the average face, and that look—scientifically speaking—“very likable, extroverted, trustworthy, and competent.”

Of the candidates remaining, in these facial measures, Rubio looks the most competent and dominant, Trump the most likable, and Carson the most attractive.

How to make sense of this?

Olivola offered an analogy. “There is a study that found that business CEOs have on average faces that are higher in how competent they look than another person who is comparable in terms of age, etcetera—who perhaps works in the business sector, but is not a CEO.”

“One way you could interpret this is that they’re actually more competent and it shows,” Olivola said, then added, “but I don’t believe that. On top of the intellectual and other advantages,” he said, “there is a kind of bias to select leaders based on how competent they look. We know this now quite well with business CEOs and I suspect what you’re seeing is the same with politicians.”

Of course, appearances aren't everything. And these judgments are especially fluid. “Recently, we have shown that first impressions are quite fragile and dependent on the image of the person,” Todorov told me. “Different images of the same person result in different impressions.”  

“People are extremely good at learning evaluative charged facts about other people,” he continued, and that “this knowledge becomes part of the face representation of the person.”

Still it seems a very sensible tradition that only words, and not pictures, show up on ballots.