Michael Hollick and John Granieri have been working on several projects that involve the use of Jack as a part of large VR systems. Typically a non-graphical version of Jack is configured to communicate with the rest of the system via sockets or shared memory. Information about the human figures in the simulation is sent to the Jack process, which is responsible for realistic animation of the figures. Posture information is sent back to the rest of the system with each simulation frame.
One example of this use was demonstrated at INCOMMS '94 at Ft. Benning, Georgia. Jack was used as a part of the Army's Individual Soldier Mobility Simulator, along with software developed at the Naval Postgraduate School, University of Utah Center for Engineering Design, and Sarcos, Inc. In this system, a soldier was outfitted with a body suit to sense upper body joint angles, a head-mounted display (or projector screens), and a rifle equipped with a Polhemus tracker, and was seated on a pedal-based device that allowed him to ``walk'', feel the slope of the terrain, and turn in a convincing manner. Jack was used for 3 purposes:
A similar application is currently in progress with Sandia National Laboratories. One major difference is that upper body angles will be computed by Jack's real-time inverse kinematics routines based on sensor data. Additionally, they are planning on experimenting with driving the locomotion from stationary walking. Semi-autonomous humans will also be included in the simulation. These agents will react to the environment and the human participants, and will play several roles in different scenarios, such as a victim pinned under a tree, or a sleeping security guard.
Another application of Jack in VR is the TTES system being designed and built by the Navy (NAWCTSD). This system projects a soldier into a virtual environment, where he may engage hostile forces. The soldier stands in front of a large projection screen, which is his view into the environment. He has a sensor on his head and gun. He locomotes through the environment by stepping on a resistive pad. The soldier may move his head, and the view frustrum is updated accordingly. Essentially, both the hostiles and the soldier can move around the environment and engage each other. The hostiles are controlled via a DIS stream of commands comming from a Computer-Generated Forces (CGF) simulator written by the Institute for Simulation and Training at the University of Florida (Orlando). TTES filters and translates the DIS stream into a set of posture tokens that are passed to Jack. Jack then animates the human figures transitioning from one posture to another. Jack passes the joint angles back to TTES for animating an SGI Performer run-time articulated database of human geometry. The connection is made through a double-buffered shared memory area. This allows typical updates of 6 human figures at 30 fps, with each human having 73 joints. The animation of the human is done via pre-recording posture transitions at 30fps, storing the joint angles, then playing back in real-time by selecting the appropriate frame from the recording, based on the real or wall-clock time of the simulation. TTES controls the global position of each human figure, using DIS dead-reckoning algorithms and information from the DIS stream. It also creates the necessary DIS Entity State Protocol Data Units (PDUs) to represent the real soldier, and sends them out to all other nodes on the network (i.e., to other TTES stations and the CGF system). It also performs the ballistics computation for firing the gun into the scene, and determining where the human figures get hit.