Self Paced LinearLinear BranchingNonlinear BranchingQuestionnairesDatabasesHypertextCreation ToolsDramaSpatial NavigationVirtual LaboratoriesLive LinkagesTaxonomy of Hypermedia
Self Paced Linear
Linear Branching
Nonlinear Branching
Questionnaires
Databases
Hypertext
Creation Tools
Drama
Spatial Navigation
Virtual Laboratories
Live Linkages
Virtual Reality
Immersion VR
Desktop VR
Second Person/Unencombered VR
Telepresence
VR Models
Hypermedia has always been hard to explain without showing examples. The word does little to send uninitiated imaginations wandering. Although a hypermedia market is beginning to emerge, it is doing so without a clear identity. We are accustomed to naming communication media after the box we have to buy to use them (for example, television or telephone). With computers we stopped buying typewriters or word processors and welcomed a multipurpose machine into our homes or offices. The term "multimedia" began to be used to describe software which required additional purchase of speakers, videodisc players, TV sets, and so on. Digital video confounds the box-based naming tradition by ceasing to require any boxes other than a personal computer to deliver software with motion video. Hypermedia has always been a function-based name, rather than a box-based name. Proponents of hypermedia find themselves at meetings waving their arms around trying to describe amazing new things that the computer already sitting on the client's desktop could do. Virtual reality (VR) also lacks a uniform, identifying box. But everyday computers obviously can't run VR, and although people are not clear about exactly what else is needed, they at least know something more must be purchased. The broad idea of VR sounds sexy and elitist, a mysterious new frontier. Just by hearing the word, people with no interest in technology immediately begin to picture applications more advanced than we will be able to design for decades. We are not faced with examples of VR, and therefore are not left wondering whether something is or is not VR. As VR applications are introduced, the true complexity of the concept will become apparent.
As function-based names, hypermedia and virtual reality are both gloriously vague. When pressed for a definition, I have always said that hypermedia is "cool things computers can do." When pressed for a definition, I have always said that virtual reality is "cool things computers can do." Instead of a Websterian one sentence abstract definition of hypermedia, I will draw upon the by now a large collection of actual hypermedia software which helped invent and define "traditional" types of hypermedia, with real world examples. Most hypermedia actually combines several elements from the typologies described here. What is striking to me about hypermedia is how diverse its implementations have become. No wonder it has been hard to describe. For virtual reality, it is only possible to describe a large collection of tools and kinds of experiences that could be created. The taxonomy of hypermedia may be useful in design discussions to clarify choices of how particular content could be hypermediated.
One of the earliest forms of hypermedia is the tour or self-paced linear presentation. Information is packaged into a collection of discrete chunks which the user can access one at a time, in order, choosing when to move on to the next chunk. Chunks might be a composed computer screen which integrates text, graphics and sound (cards in HyperCard, pages in ToolBook); an animated sequence (from Director or other animation package), a videodisc video or QuickTime/DVI digital video segment, text fields, ASCII files, and so forth. (Today the word multimedia is used to indicate that software includes sound and/or graphics and/or motion video, rather than just text. Any of the types of hypermedia described here can include text only or multimedia formats.)
Self paced linear hypermedia is frequently used in meetings and conference presentations, where the pace is controlled by a presenter. Increasingly sophisticated presentation packages (such as Aldus Persuasion or Harvard Graphics) make it easy to create linear presentations which integrate different media. Instructional hypermedia may also be self paced linear presentation, like an electronic textbook. Rather than being strictly linear, electronic links to glossaries or other information might be added.
Linear branching lets hypermedia designers enforce a fixed agenda for the user with limited interactivity. This type of hypermedia, often called a tutorial, presents a short information sequence followed by a period of interactions with the learner in the form of multiple choice questions or some other manipulation. Branching may take the form of feedback and/or remediation in response to user performance on interactive tests, after which the software resumes its singular linear path. Many of the early interactive videodiscs showed viewers five to ten minute video segments, paused to ask comprehension questions, provided positive or negative feedback, and either moved to the next video sequence or repeated the last one until the viewer got the answers right.
Electronic flashcards such as Number Munchers are another example of linear branching. The software poses a question, the child being drilled "munches" the right answer, the software gives feedback and presents the next question. Electronic flashcards may be programmed to randomize the order of presentation, but offer the user only one choice (to move to the next test item).
Interactive stories, like Amanda Stories may have a fixed beginning and ending (the cat goes out the door to begin an adventure; the cat comes back in the house after an adventure). During the story, child users can pick different places for the cat to go. Order of content presented matters and is controlled, but limited freedom of choice is provided.
The linear presentation may include more involved "interruptions" than answering questions. The learner may arrive at an opportunity to apply a concept that has just been demonstrated. For example, Physics Simulation 2.0 is a tutorial with occasional simulations where learners can program and test variables.
Nonlinear branching lets the user choose what to look at. Designers relinquish a great deal of control over how the software is used; instead the user is allowed to "explore." With nonlinear branching, users are likely NOT to be exposed to every chunk of content available in the software.
Nonlinear branching often makes use of menus offering lists of choices. The hypermedia software may contain a handful of branching options or it may have thousands. Menu structures tend to be either narrow and deep, offering great depth of information on a small number of topics (e.g., Breast Cancer Treatment Options medical information kiosk), or they may be wide and shallow (e.g., Family Doctor medical encyclopedia CD-ROM), offering many small pieces of information on a broad range of topics.
Users may get lost in the information, making it helpful for developers to provide clear directions, maps and "road signs" for navigation. Metaphors are sometimes used to define and clarify the nature of the users experience. According to Mountford (1990), metaphors have two uses in interface design: "as cognitive aids to users, and as aids to creativity for designers." Metaphor can set user expectations about content ("this is like a newspaper") or about available controls ("these controls will let you fly the hypermedia spaceship").
Nonlinear branching does not always involve menus. Shakespeare's Language Series presents the text of Shakespearean plays with an interface tool to help readers analyze the text by helping them identify and interpret familiar words used in unfamiliar ways, obsolete words, syllabic and word omissions and other techniques for understanding Shakespeare's language.
Sometimes rather than providing the user with many choices, hypermedia asks the user a series of questions, then act on the answers. The user responds, rather than explores. For example, MacDiet queries students about food intake and physical activity level and responds with four individualized reports on nutritional totals and recommended dietary allowance, dietary goals, activity and energy balance.
Databases have traditionally been accessible through search strategies such as keyword search, full text search, boolean structures of AND/OR/NOT, word frequency, word proximity and so on. The same search structure can be applied to any database, regardless of content. Hypermediated databases provide search methods which are tailored to the particular information in that database. They may add visually appealing, content specific layout. They may add a metaphor, such as a space ship control panel with a window for navigating a database about outer space. The interface may add value or meaning to the database. For example, Time Table of Science and Innovation uses a timeline as a menu interface to scientific discoveries. Physical distance between the menu entries corresponds to the time space between them, providing an intuitive sense of time relationship that is powerful and more intuitive than just listing the dates. Hypermedia database interfaces help make sense out of the content and interrelationships of the database entries.
The Montreaux Jazz Festival hypermedia database of performers consists of formatted computer screens with buttons to pop up a black and white photograph of the performer, a textual biography off the lead performer, a list of the other musicians in the group, and a CD-quality sound clip from the performance. The hypermedia front end lets users access the database by artist (selecting from alphabetical lists), by type of music, and by the festival daily performance schedule. Search capabilities are limited (for example, boolean searches are not available), but the navigation mechanisms highlight meaningful alternative organizational structures for the information.
A number of companies are developing internal video and videodisc links to Geographic Information Service (GIS) maps which activate links when users click on points on a map. Multimedia databases that use this method include railroad intersection right of way analysis, nautical navigation, and much more (Lang, 1992).
In the early 1960s, Ted Nelson coined the term hypertext to describe a system of nonsequential writing that allow authors or groups of authors to create paths through a body of information and to add their own annotations (Nelson, 1990). Nelson's project Xanadu, under development at Autodesk, is supposed to provide easy means for navigating through large collections of information. Imagine each document (text, sound byte, video, etc.) as a crossroads or intersection from which the navigator can select a next step. The system presents options to query the intersection for more information about the path choices, or to jump to a bird's eye view to examine the macro structure of paths. Navigators may select a particular "flavor" of path to display from the outset or toggle across different web options at any location within the information. Hypertext can be a dynamic collection of information and links to which new links and information can be added. Each piece of information remembers links which have been made to and from it.
Intermedia is a media-rich hypertext environment where students create their own links within a knowledge base. The software has been used in science and humanities courses.
Hypermedia often empowers users to add their own information to existing software or to create their own information packages by combining and editing content provided by the software. The ABC News Interactive videodisc series provides a tool to let users edit their own video reports by selecting and combining video sequences. The St. Louis Zoo hypermedia installation lets visitors browse animal databases and compose a "shopping list" of content segments to "print to videotape" and take home with them. Other information packages let readers add their own commentary in the margins. Chinese Hypercard for Teaching and Learning Spoken and Written Chinese lets students record their own voices and compare their pronunciation to native speakers.
Some hypermedia incorporates human or animal or inanimate actors. Videodisc content can easily employ human guides and fictional characters. But a growing number of computer-only designs are mixing live or animated beings with other hypermedia elements. These beings may express themselves through text or voice; they may be represented by still images; they may be animated; or they may be shown through digital video clips of real human actors. Dramatic hypermedia characters takes at least three forms: guides, experts and fictional characters.
Apple Computer's Guides project (Apple Computer, 1990) uses human actors to represent period-relevant human guides to a North American History database: a Native American, a settler and a miner. A contemporary guide helps users with system operation and navigation . With this narrative approach, the proposition of searching for information in space is replaced with an experience of information unfolding in time. Computer characters, called guides, provide different points of view about a common information base. Apple uses guides to engage emotions and provide a sense of companionship. The guides notice topics they are interested in and offer suggestions about next items to look at. When they have stories to tell, they get excited and try to get the users' attention. When they have nothing to say, they fall asleep. Users can also create their own guides.
The Verbum Interactive 1.1 CD-ROM includes a roundtable discussion among industry hypermedia experts. The experts appear to be seated along a speaker's table facing the audience. When the user clicks on an expert, that person appears at the top of the screen and, in living color with motion video and sound presents expert opinions about interactive media. The actual experts are captured on media, and represent themselves. Martian Lifescience Time Machine uses fictional characters as scientist/experts. Users can select historical points in time when experts were involved in heated debates about anomalies in data about Mars. The geologist, meteorologist, photographer, engineer and futurist appear in a conference room fitting the time in history. The user runs the discussion by selecting which scientist gets to speak.
The CD-ROM game Sherlock Holmes puts players into the role of Sherlock. Sherlock is primarily a "first person" dramatic experience. You alternate between watching Sherlock in video sequences and being Sherlock as you search for clues, visit suspicious locations, interview suspects and ultimately go to the judge to announce your solution to the mystery.
An "agent" is a special element in hypermedia. An agent "works for" and with the user. Some agents are dramatic -- they pop onto the screen in the form of a spinning globe or animated human and say "hi, I'm your agent..." Other agents, like the "reporters" demonstrated in Apple's Advanced Technology Group Rosebud project or the "agents" in Hewlett Packard's New Wave operating system are visible only through their actions and as icons on the desktop; their primary interface is through dialog boxes with no anthropomorphized representation.
Spatial Navigation in hypermedia allows users to navigate through imaginary or real spaces. Games like Cosmic Osmo and Spaceship Warlock let users navigate through fictional spaces. Clicking on doors opens the door and enters an adjoining room. Clicking on objects either collects them or activates some sequence of events. (Clicking on a drawer may open the drawer to reveal a book and a gun; clicking on a piano keyboard may play a song.) Apple's Virtual Museum prototype uses a computer generated museum. Visitors to the museum click on the doorway they want to enter. Footsteps are heard as the point of view "walks" down the hallway and into the room. Visitors can spin 360 degrees, and click on objects in the museum room to see interactive exhibits.
Mars Navigator lets users navigate through a real space, flying over canyons on Mars that are based on 3-D spatial data collected by NASA. A Virtual Rainforest prototype let's users navigate through videodisc recordings of the Brazilian rainforest.
Some hypermedia gives users control over a model which they can observe and manipulate. MacFrog lets users dissect a hypermedia frog, exposing muscle and skeletal layers of each body part. The software replaces laboratory experiences, making it unnecessary to kill more frogs. Chemistry Laboratory videodisc provide a virtual laboratory for testing chemical reactions and seeing the results on video. Other simulations do not precisely model the real world, instead creating fictional models like SimCity.
One of the newest (and least developed) forms of hypermedia involves linkages to live content, via phone lines, picturephones, remote cameras, television, remote sensing images direct from satellite; live microscopic images and other sensors. Zoo Atlanta is installing satellite links to zoologists in Africa, mixing prerecorded jungle sounds with live content. The Jason Project in Massachusetts connects students to a submarine called Jason, which they can control via computer (Los Angeles Times, 1992). Autoscope lets users pick intersections from a spatial map. It reads real time video images from cameras set up at that intersection and displays live data on vehicle volume, speed and que lengths (Lang, 1992).
The richness of the hypermedia typology derives from hundreds of designers who have created hypermedia content that defines and invents the medium. VR is not nearly as mature a medium; it lacks a collection of completed, accessible applications to classify.
With some exceptions, the hardware for hypermedia is uniform. The interface technology for hypermedia equals monitor + mouse + keyboard. Input/output and interface alternatives for VR are much more varied. There are at least four very different interface configurations, all of which are claimed by some to be VR. These classifications are drawn in part from the Tomorrow's Realities Gallery catalog (1991) definitions.
Immersion VR uses head mounted displays with one monitor for each eye, sound and position trackers to place the participant inside a virtual environment. The virtual world appears to respond to head movement in a familiar way (the way the natural world does: not at all) and in a way which differentiates self from world. You move and the virtual world looks like it stays still. The sense of inclusion within a virtual world which this technology creates has a powerful personal impact.
Immersion VR worlds "exist" in 3 dimensions. By sending slightly different images to each eye, participants perceive depth and dimension. What the participant sees and otherwise experiences must be recomputed (for both eyes and ears) with each movement to display the appropriate sights, sounds and other elements of the world from the new position. If the participant's ability to move were to be preordained or constrained rather than completely free, computational demands would be greatly reduced. But most VR worlds allow the participant to "go" anywhere: within structures, through walls, inside of objects, up into the sky or down into the earth (when there is a sky or earth).
At present, immersion VR stretches the limits of computational power, I/O design and understanding of human perception. The 3-D graphic VR worlds are usually made up of polygons. Some systems allow texture mapping of different patterns onto the polygon surfaces. The polygons may be shaded using different algorithms which create more or less realistic shadows and reflections. Displays are "laggy," with responses to motion being delayed, particularly for complicated worlds.
3-D sound generators simulate spatial locations of sound sources. Emerging technologies allow designers to specify spatial information about sound, and alter that sound relative to the participants current location.
Matshusta's walkthough kitchen in Japan lets consumers examine a model of a virtual kitchen from the inside, to make purchase decisions. Virtuality's Legend Quest game immerses players in an adventure inside a virtual castle.
Simulating normal human sensory inputs is only one application of VR technology. According to Robinett (1990), "any phenomenon that we can pick up with any kind of sensor, no matter where it is, no matter what size it is, no matter what the sensor is, can use a head-mounted display to make it directly perceptible."
Another form of immersion VR is caves and portals, where the immersion occurs by surrounding the body on all sides by images, rather than just the eyes. Early versions of these technologies were demonstrated at SIGGRAPH '92 in Chicago by Sun Microsystems and University of Illinois.
Some systems which call themselves VR show 2-D or 3-D displays on a computer screen rather than using a head mounted display. Although they lack the immersion quality, they consist of computer-generated environments which exist in 3 dimensions (even if they are shown on a 2-D display). Because the worlds exist in 3 dimensions, users can freely navigate in 3 dimensions around in the worlds. Examples are BattleTech in Chicago, where users climb into cockpits and drive BattleMech robot warriors around a virtual world shown on their cockpit screen, blowing each other up. Another example of desktop VR is CAD packages like Virtus Walkthrough which allow users to navigate around the 3-D worlds they have modeled. These forms of VR are less expensive and therefore more common than most of the other types. Flight simulators are another example, where participants "fly" though models of real or fantasy worlds, watching the world on a 2-D screen. FighterTown uses real Navy fighterplane cockpits with 2-D displays.
Unlike immersion VR, "unencumbered" VR systems involve real-time perceptions of and response to the actions of human unencumbered by helmets or gloves, wires or any other intrusive sensors or displays. Immersion VR simulates real world perceptions. You know you are "there" because sounds and images in the virtual world respond like the real world to your head movements. An alternative approach is what Michael Miller calls "second person VR," exemplified by some of the work of Myron Krueger (Kruger, 1991), Vivid Effects, and ENTER Corporation. In second person VR, you know you are there because you see yourself as part of the scene. On one side of the room, you stand in front of a blue background. You face a monitor and TV camera. On the monitor you see yourself, but instead of being in front of the blue background, the self you see is inside of a graphic or combined video/graphic virtual world. Edge detection software keeps track of your location and movement and allows you to interact with graphical objects on the screen. Rather than mimicking real world sensations, second person VR changes the rules, and relies strongly on a "seeing is believing" argument to induce a sense of being there.
Telepresence is a fourth major embodiment of VR. Telepresence VR uses cameras, microphones, tactile and force feedback and other devices linked to remote control capabilities to allow a participant at one site (e.g., an office building) to move their head or hands to control robots and sensors at a remote location (e.g., undersea or in space), experiencing what they would experience at that remote site. "Microteleoperation" uses a microscope and micromanipulator to give the operator a sense of presence and the ability to act in a microscopic environment (Robinett, 1992). Telepresence and other forms of VR sometimes are accomplished using "heads up displays" which superimpose the virtual display over real world sensory input.
Classifying VR on the basis of diverse interface technologies does not address the question of why those experiences should be considered virtual reality. A function-based typology of VR starts with the premise that VR lets users experience tangible models of places and things. Tangible means that the model is represented in ways that human senses can perceive directly -- not in abstractions like language or mathematics, but sight, sound, touch, smell and other senses. Experience means that the user/participant feels a sense of presence either inside of or in a shared space with the modeled world or object. In defining synthetic experience, Robinett (1992) differentiates four sources of VR models:
1.) Models can be scanned from the real world. Teleoperation uses video cameras (one for each eye) to scan the real world at a remote site. Binaural sound recordings (one microphone for each ear) scan an audio model of the real world. Remote sensing data scans the real world using different senses.
2.) Models can be computed mathematically. One of NASA's VR experiences represents air flow around the wind of a jet with visible colorized moving patterns which are generated by a mathematical formula. In some cases, a thing rather than a place is modeled, such as a chemistry molecule.
3.) Models can be constructed by artists. Polygonal CAD models are created with complete coordinate structures, allowing new views to be computed dynamically. These models can be based on actual or imaginary spaces (e.g., an exact replica of a real laboratory or an imaginary kitchen). The models are not necessarily 3-D. BattleTech, FighterTown and other virtual reality games show participants 2-D displays of the 3-D worlds they are flying or driving through.
4.) Models can be edited from a combination of scanned, computed and constructed content. ENTER-MSU second person VR combines 3-D motion video scanned from the real world with live motion video of the participant and computer-generated models of other entities to interact with (Heeter, 1992). VR worlds may add mathematical forces such as gravity, force feedback or magnetism to constructed or scanned models. Ixion combines a physical dummy and laproscope with interactive videodisc to model the gastrointestinal tract, complete with the force feedback doctors feel during laproscopy.
Thus, VR models can be based on real world places or objects, fantasy places or objects, or a combination. Time and space, if they exist in the model, can have a 1:1 relationship to the real time and real space; they can be scaled bigger or smaller, faster or slower; and they can be distorted so that some parts are bigger or faster than reality and other parts are normal size or speed (Robinett, 1992).
Returning to the original premise of this manuscript, the question was what is the difference between hypermedia and virtual reality. To the extent that hypermedia is attempting to create a tangible model for users to experience, it can be considered a form of VR. The mouse/keyboard/2-D monitor interface is not as conducive as other VR interfaces to creating a sense of presence, but that means designers have to work harder to create the illusion. Because the tools for hypermedia design are much more flexible, well developed and widely available than the tools for VR design, hypermedia is a promising development platform for experimenting with techniques and methods which may be used in VR.
Ultimately, all definitions are arbitrary. Considering some of the typical advantages and limitations of VR and hypermedia may stimulate purposive violation of those stereotypes.
Hypermedia Often Characterized by:
Virtual Reality is often characterized by:
If readers were to send BITNET Email reactions to the opinions expressed in this column, I would collect, synthesize and distribute those reactions (Heeter@MSU.edu).
Apple Computer. Guides Project Videotape. 1990.
Heeter, Carrie. Being There: The subjective experience of presence. Presence, Volume 1, Number 2, Spring, 1992.
Intellimation. Fall 1992 Higher Education Catalog.
Kruger, Myron. Artificial Reality II. Addison Wesley Publishing Co., Inc.: Reading, MA, 1991.
Lang, Laura. Example Projects Linking GIS and Multimedia. Computer Graphics World, October, 1992.
Los Angeles Times, "Interactive Field Trips," August 30, 1992, p. A3
Mountford, Joy. "Tools and techniques for creative design," in B. Laurel, Ed., The Art of Human Computer Interface Design, Addison Wesley Publishing Co., Inc.: Reading, MA, pp. 17-30, 1990.
Nelson, Ted. Literary Machines. Mindful Press: Sausalito, CA, 1990.
Robinett, Warren. Panelist on the Hip, hype and hope: Three faces of Virtual Reality panel, Panel Proceedings for SIGGRAPH, 1990.
Robinett, Warren. Synthetic Experience: A Proposed Taxonomy, Presence, Volume 1, Number 2, Spring, 1992.
Tomorrow's Reality Gallery Catalog, SIGGRAPH, 1991.
ZTeck Co. Interactive Videodiscs and CD-ROM for Education Catalog, 1992-1993.