The charter of this group was to investigate
manipulation and control frameworks for facial animation. These
frameworks should encompass a rich set of techniques applicable to a
variety of face geometries such as those proposed by the modeling group.
Ultimately, the goal would be to provide a complete and succinct set
of guidelines for controlling facial animation.
Facial animation control is the mechanism whereby a model can be articulated. Selecting a particular facial control mechanism depends on the purpose of the animation; for example, a lip-readable facial model requires a high level of precise three-dimensional manipulation of the lips, mouth, teeth and tongue. However, facial image-coding requires only a relatively simple model with a focus on pixel manipulation. Obviously, it would be inappropriate to suggest a single strategy for both situations; there are, however, many overlapping techniques that apply to both applications, and this section explores such commonalities by example.
To date, facial animation control has focused principally on three sub-disciplines: (1) facial expression, (2) facial conformation, and (3) lip motion for speech synchronization. Facial expression modeling involves deforming a model of the face into a recognizable configuration, such as the six canonical expressions of happiness, anger, fear, surprise, disgust, and sadness. Conformation control specifies individual faces from the universe of possible face prototypes that can also be associated with morphological or long-term facial changes, such as growth or aging. Lip-synchronization concerns the coordination of the jaw, lip, and mouth parts with real or synthetic speech samples. While some control strategies have been developed within each sub-discipline, a coherent framework encompassing all three has not yet evolved.
The three sub-disciplines within facial animation typically use one of four distinctive control techniques: (1) 3D shape interpolation (2) ad hoc surface shape parameterization, (3) muscle-based, or abstract muscle models, and (3) physically based models. No one technique provides an optimum control strategy since each possesses particular advantages and disadvantages. Each technique is therefore briefly described in this section.
Facial animation has also focused principally on synthesis. However, a great deal can be gained from processing and analyzing temporal sequences of real faces in motion. For example, it should be possible to control an animation from live camera input, or to derive facial control parameters from image sequences to validate a facial model's behavior. Despite a small body of research in this area, there is agreement that facial analysis will undoubtedly play an important role in facial animation control strategies.
Validation is an important part of any facial animation control strategy. Typically, the validity of a facial animation sequence is done by inspection; for example does it look right? This criteria is sufficient for semi-realistic, cartoon-based faces; as the realism of the synthetic facial models improves, we would like them to mimic reality as closely as possible. A more rigorous approach utilizes coding schemes, such as the Facial Action Coding System (FACS), to calibrate actions. The muscle-based abstraction provided by FACS maps well into a facial animation control strategy and has been used in a number of facial animation systems.
While FACS has provided a valuable foundation for facial animation, two shortcomings limit its functionality in facial animation: (1) muscular contractions timing is not available, and (2) a lack of detail around the mouth for speech synchronization. Ultimately, it should be possible to validate a muscle-based control strategy by quantifying facial motion from temporal images of real people's faces, parameterizing facial articulators, and validating the resulting model. In this form it would be possible to provide a closed-loop solution.
Finally, validating the accuracy of facial articulation plays an important role in facial animation. Without a quantitative measure of faces in motion, facial animation will remain subjective. A body of work within speech analysis sheds some interesting light on the problems associated with validating facial motion.
We organize this section of the report by first describing the basic techniques that investigators have used to date for facial animation. Next, we describe frameworks for animation control that investigators have used. Then we discuss temporal control issues, followed by a discussion of extensions to FACS. Finally, we discuss validation issues.