Among our five senses, the sense of touch is not only what we feel at the surface of our skin, such as temperature and texture; it also includes the responses as discerned by the muscles in sensing weight or hardness. All these sensations are known as somatic sensations. These sensations emerge from responses to actions, that is, the interactions of humans with their world. By reproducing somatic sensations that are more realistic, creating perceptions of the presence of "objects," a virtual world is created in which a subject actually experiences moving such objects.

Professor IWATA Hiroo, Faculty of Engineering, Information and Systems

■ Matching actions with perceptions
Recent 3D movies present quite realistic images, but there is nothing there that the audience can reach out and touch. A door opening in response to a push, or sensing the unevenness of the ground when one stamps on it--such perceptions are indispensable for us in perceiving our own actions. The essence of virtual reality consists not only in displaying a world that does not exist as though it really does, but in causing the subject to feel an appropriate response to an action taken in that world.

One promising application of this technology is in surgery simulators. A system to reproduce sensations when manipulating surgical tools or removing organs is being developed jointly with the Department of Surgery, Division of Gastroenterdogical and Hepatobiliary Surgery at the University of Tsukuba Hospital. Time is required to train surgeons, and patients cannot be used for practice, so virtual reality could be thought of as an ideal technology for efficient surgeon training.

Such research began in the latter half of the 1980s, at which time the term "virtual reality" had not yet been coined. When announcing a research result, a sense of touch cannot be conveyed in a research paper or on a poster; hence a mode of announcement relying on performance was conceived. Today it is widely used, but at the time it was a radical style of presentation.

■ Opening a world of device art

An academic conference is a closed world. A desire to release the results to the world at large culminated in the submission in 1996 of a device to a media art festival called Ars Electronica, held in Austria; the submission was a brilliant success, winning a prize and providing general visitors with an opportunity to experience the actual system. Both the scale of the event and the target audience were widely at variance from an ordinary academic conference, and the reaction of the people involved was also strikingly different.

The device of that era was called the Cross-Active System: a sensation could be experienced in which one person seemed to be playing on the palm of the hand of another participant. Various types of communication could then arise, depending on the nature of the relation between the two persons. In this way, engineering systems, including such novel ideas for communication, were evaluated as works of art.

By transcending an engineering framework, problems and needs were discovered that were not evident in discussions among researchers, and these discoveries led to further research. Up until now, more than 20 research results have been exhibited and experienced in Japan and abroad. This area, which fuses art and engineering, came within the past decade or so to be known as "device art."

■ "Walking" also goes virtual
One example of device art is Robot Tile. When walking on one of these tiles, the tiles move rearward, and another tile moves into place to receive an outstretched foot. The tiles detect the positions of the feet and move around and into place. The walking subject advances as usual, but in actuality stays in place, while moving as if crossing a series of stepping stones. The tiles can move upward and downward as well, so that the perception of, for example, climbing stairs or walking in mud can also be presented.

When the action of walking is incorporated, the range of application of virtual reality expands dramatically. One example is a device to simulate evacuation during an emergency. Images of a fire that has occurred at a certain place are projected onto a spherical display of about 1.5 meters in diameter; when viewed from the inside of the sphere, it seems as though one is at the site of the conflagration. An evacuation experience that is far more realistic than merely tracing a route on a map is possible, in which the subject walks and searches for an escape route, avoids obstacles, feels fatigue, and the like.

■ Changing one's worldview with a giant robot
The most advanced devices to create virtual worlds sometimes appear to be mechanical and unrefined. Particularly when accurately reproducing the sensation of walking, complicated combinations of machines and subtle adjustments become essential. Only three laboratories in the world are capable of creating such systems, including the one here at the University of Tsukuba. Research on virtual reality requires the passion and the skills of a craftsman.

A near-humanoid, five-meter high giant robot is currently under construction. It will be introduced into a new Empowerment Studio being planned as part of the Empowerment Informatics Program, the leader of which is Prof. Iwata; of course the robot can be operated. When the operator can act with the big robot's actions and perceptions as his own, such an experience should instantly change the operator's worldview. The aim is to realize a new means of cultivating human talent, endowed with the ability to represent ideas and concepts, and to achieve coordination that spans different fields.

Virtual reality is a type of interaction between humans and engineering systems. Through device art, which is one mode of representation of VR, the possibilities for research and education into virtual reality, as well as communication, are greatly expanded.

Article by Science Communicator at the Office of Public Relations