Background

  
Figure 1: Sensor Placement and Support Polygon

Ideally, a virtual environment interface should be able to measure and recreate a participant's posture exactly. Rather than the traditional ``disembodied hand'' approach, we would like to generate a complete, realistically postured human image. However, the equipment needed to accurately track every body segment (or joint angle) of a human is costly and cumbersome. We face several questions: how closely must the virtual human's posture match the operator's, and how much information do we need to achieve this degree of realism?

As in other areas of VR, the degree of realism necessary depends greatly on the tasks we would like to perform [11]. We have concentrated on creating an interface that will allow a human participant to perform basic tasks, using a minimal number of sensors to derive feasible, reasonably accurate postures. Three pieces of information are essential for our posture reconstruction algorithm: the participant's view vector, center of mass, and the location of the end-effectors the participant will use to interact with the virtual environment.

It is fairly obvious that the view vector needs to be accurate. If the synthesized view does not correspond to the participant's head motions, disorientation is inevitable [4][8]. Similarly, end-effectors must be placed accurately in the environment. User interaction will be difficult if the virtual representations of end-effectors do not match reality closely. The body posture can then be computed, based on these inputs, the participant's center of mass, and our human body model. While this posture may not match that of the participant's at every joint, the vital aspects will be preserved. For many applications, this degree of accuracy is sufficient.

In our simulation, we are considering the hands to be the only end-effectors available to the participant. Therefore, we need four sensors to get the information required to drive the simulation. These are placed on each palm, the head, and the waist (Fig. 1).

This posture information can then be used in a variety of ways:

• A participant's motion can be used to directly drive a simulated human in a virtual environment. This can occur in real-time, and additionally allows for interaction with other virtual humans in a networked environment.

• This information can also be used for motion/task recording and analysis. A movement can be recorded and used to drive human figures of different sizes, in different environments. For example, a percentile operator's sequence of motions can be mapped onto a percentile figure.

• It can be used as an intuitive way for controlling or positioning figures in a traditional, non-immersive environment. Even if the figure being controlled is not a direct representation of the operator in a virtual environment, this technique provides a simple way of inputting a human posture for a keyframed animation.

All our work is in the context of Jack, a multifaceted system for interactively modeling, manipulating, and animating articulated figures, principally human figures [7]. Jack represents figures as collections of rigid segments connected by joints that may have arbitrary rotational or translational degrees of freedom. The model of the human figure that we use for the example here has 36 joints with a total of 88 degrees of freedom, excluding the hands and fingers. It has a torso consisting of 17 segments and 18 vertebral joints.

Jack also provides a set of general constraints (e.g. point-to-point, point-to-plane) and the methods to solve them [12]. Jack provides a set of behavior functions for human models [9]. The behavior functions manage a set of constraints and sub-models (e.g. the spine, active stepping) of the human model. We use both of these features, constraints and the behavior functions, to map sensor values into human postures.