Concerns about using facial analysis at events: part three

An illustration showing computer experts debating whether their facial analysis software of the Mona Lisa is showing a smiling, neutral, or sad expression. Illustration by Peter Arkle; Associated Press (Mona Lisa)In early 2024, I wrote two long, detailed posts (1, 2) that explained why using “facial analysis” technology at events is ethically and legally dubious. Now I’ve learned of strong evidence that the core claim of such technology — that it can reliably measure attendee emotions at events — is seriously flawed.

Dr. Lisa Feldman Barrett is a neuroscientist, psychologist, and the Northeastern University Distinguished Professor of Psychology. In her May 16, 2024 article in the Wall Street Journal “Think AI Can Perceive Emotion? Think Again. Training algorithms on stereotypical facial expressions is bound to mislead.” she writes [emphasis added]:

The best available scientific evidence indicates that there are no universal expressions of emotion.

In 2019, the journal Psychological Science in the Public Interest engaged five senior scientists, including me, to examine the scientific evidence for the idea that people express anger, sadness, fear, happiness, disgust and surprise in universal ways. We came from different fields—psychology, neuroscience, engineering and computer science—and began with opposing views. Yet, after reviewing more than a thousand papers during almost a hundred videoconferences, we reached a consensus: In the real world, an emotion like anger or sadness is a broad category full of variety. People express different emotions with the same facial movements and the same emotion with different facial movements. The variation is meaningfully tied to a person’s situation.

In real life, angry people don’t commonly scowl. Studies show that in Western cultures, they scowl about 35% of the time, which is more than chance but not enough to be a universal expression of anger. The other 65% of the time, they move their faces in other meaningful ways. They might pout or frown. They might cry. They might laugh. They might sit quietly and plot their enemy’s demise. Even when Westerners do scowl, half the time it isn’t in anger. They scowl when they concentrate, when they enjoy a bad pun or when they have gas.

Similar findings hold true for every so-called universal facial expression of emotion. Frowning in sadness, smiling in happiness, widening your eyes in fear, wrinkling your nose in disgust and yes, scowling in anger, are stereotypes—common but oversimplified notions about emotional expressions.

Where did these stereotypes come from? You may be surprised to learn that they were not discovered by observing how people move their faces during episodes of emotion in real life. They originated in a book by Charles Darwin, “The Expression of the Emotions in Man and Animals,” which proposed that humans evolved certain facial movements from ancient animals. But Darwin didn’t conduct careful observations for these ideas as he had for his masterwork, “On the Origin of Species.” Instead, he came up with them by studying photographs of people whose faces were stimulated with electricity, then asked his colleagues if they agreed.”

“…In short, we can’t train AI on stereotypes and expect the results to work in real life, no matter how big the data set or sophisticated the algorithm. Shortly after the paper was published, Microsoft retired the emotion AI features of their facial recognition software.”
Dr. Lisa Feldman Barrett, “Think AI Can Perceive Emotion? Think Again.”, Wall Street Journal, May 16, 2024

Facial analysis is a poor indicator of emotional states

Here is the detailed 2019 research article Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements by Dr. Barrett et al.

barrett-et-al-2019-emotional-expressions-reconsidered-challenges-to-inferring-emotion-from-human-facial-movements

Dr. Barrett concludes:

“In real life, when you perceive someone else as emotional, your brain combines signals from your eyes, ears, nose, mouth, skin, and the internal systems of your body and draws on a lifetime of experience. An AI model would need much more of this information to make reasonable guesses about a person’s emotional state.”

One of the research paper’s general recommendations is to “Direct healthy skepticism to tests, measures, and interventions that rely on assumptions about “reading facial expressions of emotion” that seem to ignore published evidence and/or ignore integration of contextual information along with facial cues.”

Based on the presented research, that sounds like good advice to anyone considering acquiring facial analysis technology.

Postscript

My sharing of the above information upset at least one technology vendor that claims to provide useful facial analysis at events. He characterized it as a publicity stunt, and asked two attorneys to “please keep this for our file and action, as needed.”

I stand by my opinions and assert my right to share other’s research on this ethically dubious and scientifically suspect technology.

Image attribution: Illustration by Peter Arkle; Associated Press (Mona Lisa)

The importance of music in our lives

Photograph of Adrian Segar's iPod, showing a smart playlist of music he loves
Adrian’s iPod

My ancient iPod now has only one job: storing my music library of 765 tracks. Some of these performances bring me to tears when I listen to them. Many are bound to experiences in my life, and hearing them connects me to those powerful memories in a way that no other sense — save perhaps smell — can equal.

You probably have this kind of relationship with music. Your taste may vary dramatically from mine, the intensity of your connection may be different, but there’s no argument that music is an important ingredient in most human lives.

Long ago, my father played drums in a dance band, Billy Merrin and His Commanders, on the weekends. A few years before he died, I tracked down a collection of old recordings of his band. I vividly remember his delight and animation when he began listening once again to the music he had helped to create sixty years earlier.

“People haven’t always been there for me but music always has.”
—Taylor Swift

If/when I am old and feeble, unable to do much, I want to have my music at hand. (On shuffle, please.) I hope I will still be able to listen and recall and remember. I want to sing along when the spirit moves me and feel the intense wondrous emotions that music has the power to grant.

How to inspire transformational learning

transformational learningWhy do some learning experiences stay burning in our brains and others fade into oblivion? Transformational learning is the key!

I still vividly remember the first Event Camp held on February 6, 2010, in New York City. I learned so much at this one-day event, meeting many progressive meeting industry professionals for the first time and making what have turned out to be lasting connections and friendships.

And I believe that most participants experienced something similar.

Why did we have this shared learning and connecting experience? Was there a critical factor that made this meeting such a transformational experience?

Emotional connection

While reading J. Scott Wagner‘s wonderful book The Liberal’s Guide to Conservatives — a must-read for liberals and conservatives who want to communicate better with each other — I came across a passage that answers these questions:

“It’s easy to forget that inspiration is the only voluntary catalyst for transformation.

There’s only one way I’ve found that our adult unconscious mind can consistently be inspired to shed…heuristics and biases and learn something challenging from someone else. It’s actually miraculously easy, often: we experience a positive emotional connection together.”
—J. Scott Wagner, The Liberal’s Guide to Conservatives

Scott is not talking here about our routine day-to-day right-brain learning. Rather, he is describing transformational learning, the kind where real change can occur in how we view the world and our experience of it. He says, and I agree, that a positive emotional experience of connection inspires transformational learning.

transformational learning: photograph of Adrian Segar at the first Event Camp in New York City in 2010That’s what happened at Event Camp 2010. We came together for the first time and discovered kindred souls who were thirsting to learn and share about how to make meetings better. And in one day, our positive emotional connection changed our preconceptions of what meetings could be.

The original participants at the first edACCESS conference, which I and others convened in 1992, felt the same way. The experience of this early peer conference led to an annual conference that’s still thriving 33 years later. Over time, it has become clear that the driving force behind the event’s success has been how its design fosters participants’ positive emotional experiences, creating and supporting opportunities for transformational change in how the professional attendees view and do their work.

Fostering learning experiences

Traditional meetings don’t treat sessions as times to foster positive emotional learning experiences but as times to learn from lectures. So, at such meetings, positive emotional experiences are restricted to non-session socials and non-session-entertainment. The official learning opportunities are segregated from exactly the kind of environments that can make them inspirational and transformative.

Paradoxically, we design special events to create positive emotional experiences—but special events don’t focus on learning! Rather, to inspire transformational learning, you need to create conferences and conference sessions designed around appropriate positive emotional experiences that relate to the participants’ real learning wants and needs. Do this, and you’ll discover how powerful, transformational, and unforgettable meetings can be!