The Comm Tech Lab "Hands On Hawaii" interface was an experiment to let users see their real hands inside of the virtual worlds. Prior research had shown that people would rather see their real self than computer-generated hands.
Our experience features an interface that lets users sit down at a custom-designed whale shaped kiosk, slip their hands under a curtain, and watch their hands appear in a virtual Hawaii on a screen in front of them. Both their life sized hands and the virtual world are photorealistic. Users can "touch" video-graphic objects to explore the islands and to learn about their ecosystems. The dividing line between real and virtual world is marked by the curtain you slip your hands under to enter the virtual world. We hoped the interface would create a compelling and involving sense of presence in a virtual world. And we conducted research on user reactions.
Hands On Hawaii is a "guided discovery learning" infotainment experience based on actual for-college-credit 3 week learning experiences in the Hawaiian Islands. Users can explore the extinct and active volcanoes on Hawaii and Maui. They can explore the coastline or visit sites which show human impacts on the island ecosystem at different points in history. They can control a 70 million year animation of the birth of the islands. Grabbing the guide of their choice, users can parachute down the surface from the aerial interface to explore.
There are several components to the interface which use Real Hands in different ways. The most intuitive interface element is simply touching a graphical object to interact. In Hands on Hawaii, touching menu choices selects them. Touching the flying whale takes you back to the main menu. Touching the helicopter takes you up to the overhead view of the islands.
The Volcano Finger Drum was similar to Vivid Effect's full body interface drums, where touching a drum pad triggered a drum sound and visual effect. In our case, we wanted to try the concept using the hands interface. Three drum heads each played a different "volcano" sound -- one was a lava eruption; one was steam sizzling where hot lava touched the ocean; one was the sound of a pebble-sized cinders hitting the ground during a cindercone eruption. The drum could be played with three fingers. Parallel processing on the Amiga allowed for simultaneous triggering.
Within the beaches section, we built in a segment where a beach ball appears on a white sand beach. The ball would react to your touch, letting you launch it, catch it, and bounce it around on the sand. It was a totally intuitive way to use your real hands -- the edge detection was quite imperfect, so it did not work as well as desired, but it was quite effective and would work very well under a more robust system.
Content from two different hikes was included, one in linear fashion hiking into Holeacola canyon on Maui, and one nonlinear to demonstrate vegetation at different elevation levels climbing up Mona Loa on the Big Island. For the Holeacola hike, we used camera motion on still frames to give a feel for motion along the path the students were following. The Larry or Trista made comments about interesting facts observed along the way, with sound effects of hiking on a path. Graphical road sides appeared at the end of each camera motion, which, when touched, moved the observer in the direction the arrow was pointing to the next sequence. The flying whale appeared periodically to allow escape from the linear sequence of ten segments.
For Mona Loa we instead offered users choice of which of 4 elevations they visited, and in what order. The interface was based on dragging a miniature animated hiker onto one of four framed rectangular closeup shots of the surface located along a representation of the mountain. The interface transported you to the surface at that elevation, where camera motion on the slide accompanied a brief narration and hiking sounds. Touching the hiker returned you to the elevation menu.
Within the segment on petroglyphs carved by early Hawaiians into volcanic rock, we ended with a segment where the user could stick their finger into a bucket of white paint and draw their own petroglyph onto a rock background.
We frequently resorted to an interface method where users dragged a small graphical pointing hand onto a visual representation of a choice or action-- for example, to choose a guide, they dragged a hand onto the image of either Larry or Trista. To choose an island locale to visit, they dragged a hand onto the parachute of their choice over one of the islands. This was necessary to ensure that what got activated was intentional, to avoid accidental triggering of "hot spots" on the screen. We have no way of knowing where a user's hands are at any given moment. By making it a two step process, we added unfortunate complexity to the user's task, but helped to insure that triggered actions were intentional. Touching a graphical hand caused the graphic to stick to your real hand, moving with your hand around on the screen. The edge detection software we used has two options -- left hold or right hold, which means that the graphic sticks either to the leftmost or the rightmost edge of the user's hand. Unfortunately, this meant that users had to hold their hand in a pointing position to have good control over where they dragged the graphic. If some part of the hand other than the finger you were pointing with stuck out further than your pointing finger, the graphic object would slide down the side of your hand to the bottom of the screen. Also, fast jerky moves would release the hold, so one had to move moderately slowly to successfully drag the graphic onto a guide or parachute.
284 SIGGRAPH '93 attendees who tried the interface, then filled out a survey
Hands On Hawaii is a research prototype to test our Real Hands,
Virtual Worlds interface concept. Is the interface a mode that you
would like use for learning?
26%-DEFINITELY 66%-POSSIBLY 7%-DEFINITELY NOT
Is it an interface you think would be good for children to use for
learning?
59%-DEFINITELY 38%-POSSIBLY 4%-DEFINITELY NOT
Do you think this could be an effective museum interface?
52%-DEFINITELY 52%-POSSIBLY 5%-DEFINITELY NOT
Would you be interested in seeing and using your real hands inside
of other computer applications, such as electronic mail or graphic
design?
38%-DEFINITELY 47%-POSSIBLY 14%-DEFINITELY NOT
Given the choice, would you prefer the real hands interface or a
traditional point and click mouse interface for the type of content
you just experienced?
54%-REAL HANDS 46%-A MOUSE
Some VR experiences let you interact with objects using your whole
body, looking across the room at your "self" on a large screen. Given
the choice, would you prefer the real hands interface or a whole body
interface for the content you just experienced?
44%-WHOLE BODY 56%-REAL HANDS
How important was it to not see your physical hands?
VERY IMPORTANT 7 6 5 4 3 2 1 NOT IMPORTANT
Mean = 3.2 34% answered "1" (not important)
Between seeing your real hands and seeing a computer-generated
hand in this kind of interface, which do you prefer?
REAL HAND 1 2 3 4 5 6 7 COMPUTER HAND
Mean = 2.9 41% answered "1" (real hand)
Are you left handed or right handed?
9%-LEFT 86%-RIGHT 5%-BOTH
Which hand(s) did you use in the interface?
6%-LEFT 73%-RIGHT 22%-BOTH
How difficult was it to move the small hands where you wanted them
to go?
VERY DIFFICULT 7 6 5 4 3 2 1 VERY EASY
Mean = 5.1 19% answered "7" (very difficult); 5% answered "1" (very
easy)
Were you interested in the content about Hawaii?
VERY MUCH 7 6 5 4 3 2 1 NOT AT ALL
Mean = 4.1
Which guide did you usually choose?
21%-LARRY 46%-TRISTA 32%-BOTH
Between seeing your real hands and seeing a computer-generated
hand in this kind of interface, which do you prefer?
REAL HAND 1 2 3 4 5 6 7 COMPUTER HAND
Mean = 2.9 41% answered "1" (real hand)
Are you left handed or right handed?
9%-LEFT 86%-RIGHT 5%-BOTH
Which hand(s) did you use in the interface?
6%-LEFT 73%-RIGHT 22%-BOTH
How difficult was it to move the small hands where you wanted them
to go?
VERY DIFFICULT 7 6 5 4 3 2 1 VERY EASY
Mean = 5.1 19% answered "7" (very difficult); 5% answered "1" (very
easy)
Were you interested in the content about Hawaii?
VERY MUCH 7 6 5 4 3 2 1 NOT AT ALL
Mean = 4.1
Which guide did you usually choose?
21%-LARRY 46%-TRISTA 32%-BOTH
How would you rate your enjoyment of different aspects of the
experience:
(please rate only if you tried it)
(note -- I have reordered the items so that they appear in order of
decreasing average enjoyment)
Mona Loa Elevation Windows
ENJOYED VERY MUCH 7 6 5 4 3 2 1 NOT AT ALL
36 responses. Mean = 4.9
Parachute Down - Helicopter Up Island Access
ENJOYED VERY MUCH 7 6 5 4 3 2 1 NOT AT ALL
189 responses. Mean = 4.7
Petroglyphs
ENJOYED VERY MUCH 7 6 5 4 3 2 1 NOT AT ALL
64 responses. Mean = 4.7
Birth of an Island Timeline
ENJOYED VERY MUCH 7 6 5 4 3 2 1 NOT AT ALL
98 responses. Mean = 4.6
Holeacola Hike
ENJOYED VERY MUCH 7 6 5 4 3 2 1 NOT AT ALL
52 responses. Mean = 4.2
Pulling the Small Hands onto a Parachute or Guide
ENJOYED VERY MUCH 7 6 5 4 3 2 1 NOT AT ALL
263 responses. Mean = 3.8
Volcano Finger Drum Kit
ENJOYED VERY MUCH 7 6 5 4 3 2 1 NOT AT ALL
56 responses. Mean = 3.8
If 10 is VERY IMPORTANT and 0 is NOT IMPORTANT AT ALL, please rate the importance of the following interface elements for future designs:
(note -- I have reordered the items so that they appear in order of decreasing average importance)
attribute: average
Better User Control 9.4
Sound Effects 8.5
Motion Video 8.3
More Interactivity 8.2
Music 7.8
3-D Video 7.1
3-D Sound 6.6
Tactile Experiences 6.6
Motion Simulator 5.7
Different Expertises 5.6
Other Real People 5.4
Immersive Helmet 4.3
Some of the interactivity was just for fun, trying to keep the
experience engaging. Is "fun" stuff an important part of the learning
experience?
VERY IMPORTANT 7 6 5 4 3 2 1 NOT IMPORTANT
Mean = 6.1 54% answered "7" (very important)
Did you feel like you were part of the virtual world?
VERY MUCH 7 6 5 4 3 2 1 NOT AT ALL
Mean = 3.4 23% answered "1" (not at all)
How old are you?
Mean = 34 range = 16 to 2
Are you:
30%-FEMALE 70%-MALE
Do you consider yourself artistic or technically oriented?
18%-ARTISTIC 29%-TECHNICAL 51%-BOTH 2%-NEITHER