Subsea Teleprogramming

Principle researchers are Craig Sayers and Richard Paul (UPenn), Dana Yoerger, Louis Whitcomb and Josko Catipovic (WHOI).

Introduction
The master station
The slave station
Experiments
For more information
Acknowledgements

Introduction

The motivation for this research comes from a desire to perform subsea manipulative tasks under operator control via an acoustic communications link. This is a challenging problem due to the limitations of the acoustic link. We can expect bandwidths of only 10 kbits/sec and round-trip delays on the order of 10 seconds.

The teleprogramming paradigm provides a means to cope with this limited communications channel. The operator is presented with a simulation of the remote site. His or her actions within that simulation are monitored and translated into a sequence of robot program commands for transmission to, and execution by, the remote slave robot. Errors are detected at the slave site by comparing expected sensor readings (predicted by the master station and encoded within each command) with those actually measured during command execution. When such an error is detected the slave pauses and sends a signal to the operator. Is is then his/her responsibility to diagnose and correct the error, generating new commands for the remote robot. In this way the operator programs his or her way around each problem as it occurs.

The master station

An example of the operator display is shown here. The upper windows show different views of the simulated world. The lower large window shows a view from a real camera at the remote site while the remaining small window shows an image fragment. In this case the operator is about to calibrate the subsea cameras. He or she will select corresponding features between the simulation and real imagery. The system may then use those feature points to compute calibration parameters. Real images from the calibrated cameras may then be overlayed on the graphical simulation and used to update the operator station model of the remote environment.

As the operator performs the task within the simulated world the system continually observes his or her actions to generate a stream of symbolic commands for transmission to the remote site.

The slave station

This picture shows the current slave system, utilizing the JASON subsea vehicle, as it is prepared for entry into the water.
The manipulator is mounted on the front right of the vehicle while cameras (the long dark tubes) are mounted above and on the left of the vehicle. These cameras provide images for use in updating the world model and also serve an important role in diagnosing errors at the remote site. Due to the limited communication bandwidth it is infeasible to transmit every imaged pixel back to the operator station. Thus we must adopt an intelligent approach. The master station predicts which fragment of the available imagery would best aid the operator if an error were to occur. That information is encoded within the transmitted commands and used by the slave to select appropriate image fragments for transmission back to the operator station.

Experiments

Early experiments were performed in air, later experiments were performed in a test tank. The most recent trials have been in the sea. For these tests the operator station was located in Philadelphia while the remote slave robot was mounted on the JASON subsea vehicle and submerged on the sea floor around 100m off the Massachusetts coast. Communication between the operator and remote sites was via the internet with an additional time delay inserted via software to simulate the effects of an acoustic communications link. Communication between the surface and submerged vehicle was via optical fibre cable.

The chosen task was retrieval of a capsule dropped to the sea floor. The vehicle was first manually piloted to the target area and settled on the sea bed within reach of the capsule. The manipulator arm was then deployed and calibrated before control was turned over to the teleprogramming operator (located over 500 km away in Philadelphia). He or she then calibrated remote cameras, updated the world model using real visual imagery, and proceeded to perform the task.

Here we show one of the simplest examples of execution encountered during testing - the communications time delay was 15 seconds and no errors or problems occurred.

Master	Slave	Description
		0:03:00 The operator updates the master station model by moving the simulated object until its position is consistent with the overlayed real imagery.
		0:03:57 The operator has begun the task by performing a joint-space motion to achieve a desired arm configuration.
		0:04:08 The master station aids the operator in grasping the capsule while the slave robot begins to receive and execute the first commands.
		0:04:26 As the operator raises the capsule within the virtual world the real slave robot moves to grasp it.
		0:04:35 The operator completes the task while the real slave robot grasps the capsule. Note the displayed image fragment on the right of the operator screen.
		0:04:46 By observing the received visual imagery the operator confirms that the task has been successfully completed.

For more information

Link to page describing synthetic fixturing - an operator aid for virtual reality and teleoperation systems.

Craig Sayers, Angela Lai and Richard Paul. "Visual Imagery for Subsea Teleprogramming", IEEE Robotics and Automation Conference, May, 1995.

Craig Sayers and Richard Paul. "An operator interface for teleprogramming employing synthetic fixtures", Presence, Vol. 3, No. 4, 1994.

Janez Funda, Thomas Lindsay and Richard Paul, "Teleprogramming: Toward Delay-Invariant Remote Manipulation", Presence, Vol. 1, No. 1, pages 29-44, Winter 1992.

Acknowledgements

A number of other people have contributed to development of the JASON submersible and its operation during the teleprogramming experiments. They include Bob Ballard, Andy Bowen, Tom Crook, Dick Edwards, Bob Elder, Larry Flick, Skip Gleason, Matt Grund, David Hoag, Gaylord Holder, Bill Lange, Bob McCabe, Dave McDonald, Will Sellers, Tim Silva, Tad Snow and Nathan Ulrich.

This material is based upon work supported by the National Science Foundation under Grant No. BCS-89-01352. The JASON ROV System is operated by the Woods Hole Oceanographic Institution's Deep Submergence Operations Group, supported by NSF Grant OCE-9300037. The Teleprogramming In-water Tests were funded by the Naval Underwater Warfare Center, Newport, Rhode Island, under Contract N66606-4033-4790. Any opinions, findings, conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of any funding agency.

---

---

Craig Sayers (sayers@grip.cis.upenn.edu) May 1995