Speech production and facial expression (including body movement) are linked with each other by a synchrony phenomenon [23][20][69][29][128]. Synchrony implies that changes occurring in speech as well as in body movements should appear at the same time. For example, when a word begins to be articulated, eye blink, hand movement, head turning, and brow raising can occur and finish at the end of the word. The effective presence of a movement and its intensity is related to the emotional state of the person. However, the time of appearance is linked to the synchrony phenomenon.
A person expresses his or her thoughts with words and expressions. For example, smiling, raising eyebrows, nose wrinkling co-occur within a verbal content. Some facial expressions accompany the flow of speech and are synchronized at the verbal level, punctuating accented phonemic segments and/or pauses. Other facial expressions emphasize what is being said, substituting for a word.
There are studies to support the notion that verbal statements can modify the interpretation of emotion in a facial expression. When congruent context statements are heard at the same time with a facial expression, people will tend to agree on what the facial expression's emotion represents. On the other hand, people will differ on implied emotional content in a facial expression when incongruent context statements are heard [97]. This is an extremely important observation for any facial animation that attempts to mix speech and facial expressions.