Can Facial-Recognition Software Help Stores Manage Angry Customers?

I got a glimpse of the future of shopping at this week’s National Retail Federation conference, and it revolves around some very impressive and unsettling technology. There was technology that collects digital currency (otherwise known as coupons) if users agree to watch commercials instead of skipping them. New facial-recognition software claimed to determine gender with better than 90 percent accuracy and age range with about 70 percent accuracy. And, most interesting to me, there was a demonstration of facial-expression recognition software that can read some of our emotions. It was developed by a company called Emotient; Intel Capital, the venture capital division of Intel, is an investor.

I stood in front of a screen with a camera and made faces for a couple of minutes as the chief executive officer of Emotient, Ken Denman, watched. The software is supposed to be able to identify seven of our greatest—and mostly negative—emotions: sadness, anger, fear, disgust, and contempt, as well as surprise and joy. It’s based on the work of Paul Ekman, a psychologist who studied a remote tribe in Papua New Guinea in the 1960s and found that at least some facial expressions of emotion are universal. Ekman, who’s an adviser to Emotient, also discovered “microexpressions” that could be used to reliably detect concealed emotions. From that, he developed the Facial Action Coding System. “It’s an alphabet of facial expressions,” says Denman. The software can learn, too, so it will become more precise and sophisticated the more often it’s used.