In marketing and advertising research, “zapping” is defined as the action when a viewer skips a commercial advertisement. Researchers analyze audience’s behavior in order to prevent zapping, which helps advertisers to design effective commercial advertisements. Since emotions can be used to engage consumers, in this paper, we leverage automated facial expression analysis to understand consumers’ zapping behavior. To this end, we collect 612 sequences of spontaneous facial expression videos by asking 51 participants to watch 12 advertisements from three different categories, namely Car, Fast Food, and Running Shoe. In addition, the participants also provide self-reported reasons of zapping.
We adopt a data-driven approach to formulate a zapping/non-zapping binary classification problem. With an in-depth analysis of expression response, specifically smile, we show a strong correlation between zapping behavior and smile response. We also show that the classification performance of different ad categories correlates with the ad’s intention for amusement. The video dataset and self-reports are available upon request for the research community to study and analyze the viewers’ behavior from their facial expressions.