For human interaction, it is important to understand what emotional state others are in. Especially the observation of faces aids us in putting behaviours into context and gives insight into emotions and mental states of others. Detecting whether someone is nervous, a form of state anxiety, is such an example as it reveals a person’s familiarity and contentment with the circumstances. With recent developments in computer vision we developed behavioural nervousness models to show which time-varying facial cues reveal whether someone (...) is nervous in an interview setting. The facial changes, reflecting a state of anxiety, led to more visual exposure and less chemosensory (taste and olfaction) exposure. However, experienced observers had difficulty picking up these changes and failed to detect nervousness levels accurately therewith. This study highlights humans’ limited capacity in determining complex emotional states but at the same time provides an automated model that can assist us in achieving fair assessments of so far unexplored emotional states. (shrink)

Faces provide information about a person's identity, as well as their sex, age, and ethnicity. People also infer social and personality traits from the face — judgments that can have important societal and personal consequences. In recent years, deep convolutional neural networks (DCNNs) have proven adept at representing the identity of a face from images that vary widely in viewpoint, illumination, expression, and appearance. These algorithms are modeled on the primate visual cortex and consist of multiple processing layers of simulated (...) neurons. Here, we examined whether a DCNN trained for face identification also retains a representation of the information in faces that supports social‐trait inferences. Participants rated male and female faces on a diverse set of 18 personality traits. Linear classifiers were trained with cross validation to predict human‐assigned trait ratings from the 512 dimensional representations of faces that emerged at the top‐layer of a DCNN trained for face identification. The network was trained with 494,414 images of 10,575 identities and consisted of seven layers and 19.8 million parameters. The top‐level DCNN features produced by the network predicted the human‐assigned social trait profiles with good accuracy. Human‐assigned ratings for the individual traits were also predicted accurately. We conclude that the face representations that emerge from DCNNs retain facial information that goes beyond the strict limits of their training. (shrink)

Social-rank cues communicate social status or social power within and between groups. Information about social-rank is fluently processed in both visual and auditory modalities. So far, the investigation on the processing of social-rank cues has been limited to studies in which information from a single modality was assessed or manipulated. Yet, in everyday communication, multiple information channels are used to express and understand social-rank. We sought to examine the voluntary nature of processing of facial and vocal signals of social-rank using (...) a cross-modal Stroop task. In two experiments, participants were presented with face-voice pairs that were either congruent or incongruent in social-rank. Participants’ task was to label face social dominance while ignoring the voice, or label voice social dominance while ignoring the face. In both experiments, we found that face-voice incongruent stimuli were processed more slowly and less accurately than were the congruent stimuli in the face-attend and the voice-attend tasks, exhibiting classical Stroop-like effects. These findings are consistent with the functioning of a social-rank bio-behavioural system which consistently and automatically monitors one’s social standing in relation to others and uses that information to guide behaviour. (shrink)

Own-age bias is a well-known bias reflecting the effects of age, and its role has been demonstrated, particularly, in face recognition. However, it remains unclear whether an own-age bias exists in facial impression formation. In the present study, we used three datasets from two published and one unpublished functional magnetic resonance imaging study that employed the same pleasantness rating task with fMRI scanning and preferential choice task after the fMRI to investigate whether healthy young and older participants showed own-age effects (...) in face preference. Specifically, we employed a drift-diffusion model to elaborate the existence of own-age bias in the processes of preferential choice. The behavioral results showed higher rating scores and higher drift rate for young faces than for older faces, regardless of the ages of participants. We identified a young-age effect, but not an own-age effect. Neuroimaging results from aggregation analysis of the three datasets suggest a possibility that the ventromedial prefrontal cortex was associated with evidence accumulation of own-age faces; however, no clear evidence was provided. Importantly, we found no age-related decline in the responsiveness of the vmPFC to subjective pleasantness of faces, and both young and older participants showed a contribution of the vmPFC to the parametric representation of the subjective value of face and functional coupling between the vmPFC and ventral visual area, which reflects face preference. These results suggest that the preferential choice of face is less susceptible to the own-age bias across the lifespan of individuals. (shrink)

Research in face recognition has tended to focus on discriminating between individuals, or “telling people apart.” It has recently become clear that it is also necessary to understand how images of the same person can vary, or “telling people together.” Learning a new face, and tracking its representation as it changes from unfamiliar to familiar, involves an abstraction of the variability in different images of that person's face. Here, we present an application of principal components analysis computed across different photos (...) of the same person. We demonstrate that people vary in systematic ways, and that this variability is idiosyncratic—the dimensions of variability in one face do not generalize well to another. Learning a new face therefore entails learning how that face varies. We present evidence for this proposal and suggest that it provides an explanation for various effects in face recognition. We conclude by making a number of testable predictions derived from this framework. (shrink)