Gestures control true 3D display

How do you interact with a display that contains depth as well as height and width?

Researchers from the University of Toronto have put together a system that allows for direct gestural interaction with virtual objects contained in a volumetric display.

Volumetric displays occupy a volume of space, like a crystal ball. The challenge in interacting with a volumetric display is akin to pointing out objects in a crystal ball.

The researchers' method involves using fingers to gesture in the space around and on the surface of the volumetric display. The user's finger positions and postures are tracked by a set of four cameras.

Any application that displays three-dimensional graphics can make use of the volumetric display interaction technique. It could eventually be used as an interface for three-dimensional medical, architectural and graphic design applications, according to the researchers.

The interface includes two-dimensional menus projected on the surface of the display and a browser for selecting three-dimensional objects used to construct models. The browser uses a grid that contains three-dimensional images of objects like cubes, spheres and pyramids.

Using finger gestures, users can point at objects, make gestures to trigger commands, and manipulate three-dimensional models projected in the display, including moving, rotating and resizing the models or portions of the models.

It will be 10 to 20 years before the device can be used for practical applications, according to the researchers. The work was presented at the User Interface Software and Technology 2004 (UIST '04) conference held October 24 to 27 in Santa Fe, New Mexico.