13 May 2014
University of Bristol project uses DLP projectors to form interactive images in fog screens.Bristol Interaction and Graphics group at the University of Bristol in the UK takes the idea of images projected onto planar layers of mist or fog, already used for some entertainment and signage applications, and develops it as a possible approach to interactive screens for multi-user collaboration.
MisTable is built around a conventional horizontal work table, onto which images and graphics can be projected and manipulated - a familiar approach to collaborative work in design-oriented industries.
But in MisTable, users positioned around the work surface address it through vertical fog screens, on which additional visual information can be projected and interacted with.
BIG presented MisTable, along with other interactive technologies being developed by the group, at the 2014 Conference on Human Factors in Computing Systems, an annual meeting organized by the Association for Computing Machinery.
In the proof-of-concept system developed at BIG, MisTable features four fog screens positioned around the horizontal light table. DLP projectors provide the images, and a system of tracking sensors monitors the user's hands and allows the manipulation of the images on the fog screen, in a similar manner to the way the images can be reorientated and manipulated on the shared solid work surface.
The fog gives each user an additional layer of visual information to control, one that can be customized to suit individual preferences or requirements, but can also be reached through whenever interaction with the visual data on the solid worktable is needed.
According to Diego Martinez of BIG, the goal is to provide users with a range of possible customizations and novel interactions, perhaps ultimately including the presence of 2D personal contents on the screen, 3D contents above the tabletop, or the relighting of tangible objects differently for each user.
The system takes advantage of the rapid response capabilities of current digital light processing projection systems to control and manipulate the visual images projected into the fog, but also relies upon a number of individual tracking systems to provide the interactivity. For MisTable, the team took commercially available motion sensors and incorporated them into the prototype.
"A Kinect sensor on top of the tabletop tracks the user's hands, detecting interactions with the surface of the tabletop and also in the volume above it," Martinez explained. "Another sensor, a Leap Motion, is located in the top part of each personal fog screen, and detects interactions there. Finally, another Kinect is located high above the tabletop, to track the user's heads. This is necessary to display perspective-corrected 3D graphics, and for our brightness compensation algorithm."
That algorithm proved the key to combating an inherent, and potentially detrimental, change of brightness in the fog-mounted images, depending on the angle between incident light and observer. The team discovered that with its particular projector set up and a standard fog screen composed of 0.4 micron particles, the observed brightness was five times greater at a 5 degree viewing angle than at 60 degrees. Adjustments based on a calculated attenuation profile for each image dealt with the problem.
No similar algorithm was needed to control the chromaticity of the images, which seems not to vary with angle in the same way. "The fog is an emulsion containing both air and glycerol particles, and the number of particles might be enough to counterbalance the effect of differences in the refractive index," commented Martinez. "The angle of incidence of a ray onto each of the particles, after already bouncing from a few previous particles, might also be influencing its outward direction."
Zone of comfort
The most significant challenges to developing MisTable involved the registration of the projection and tracking systems, and control of the fog screens.
"All of the tracking systems must provide coordinates relative to the same system of reference, and use the same units," said Martinez. "One must also be extremely careful to determine the location and size of all the elements in the system - projectors, screens, tabletop - again all relative to a common system of reference. Failure to correctly register any of these elements totally breaks the illusion we are trying to create; the brightness compensation will not apply correctly, for example."
As for the fog screens themselves, maintaining a laminar flow of air was critical. MisTable creates a fog curtain 15 mm thick, sandwiched between two thicker 70 mm curtains of air to protect the fog from external air currents. These flows travel vertically downwards, and need to move at the same controlled speed to ensure a laminar flow and remove any turbulence that would decrease the sharpness of the image. The challenge for the team was to find a flow of air and fog that was fast enough to successfully reach the bottom of the screen, but slow enough to remain laminar.
With the MisTable concept proven, work to further improve its interactivity and immersive visual capabilities might now follow. Such issues of visual perception can pose difficult questions, as Martinez noted, but MisTable could offer solutions to some of them.
"Humans use many different depth perception cues to perceive three dimensions: occlusions, relative size, perspective, shadows, and others," he said. "The current implementation of MisTable supports most of these techniques. It even provides better support for occlusions than the use of larger projection walls can offer, and better use of the 'zone of comfort', where the conflict in focusing that forms a major limitation of planar 3D display technologies is reduced. With all these cues, the system already provides a very compelling experience."
About the Author
Tim Hayes is a contributing editor to Optics.org.