Chapter One The interface is the application "I like John's one-sentence definition of cyberspace: `The place you are when you're on the phone.'" Mitch Kapor, Founder of Lotus, quoting John Perry Barlow, President of the Electronic Frontier Foundation and lyricist for the Grateful Dead THIS morning, at the NASA Ames Research Center in Mountain View, California, Dr. Lew Hitchner explores the surface of Mars. He's wearing virtual reality (VR) goggles with two tiny liquid crystal display (LCD) screens inside. They give him a realistic, 3-D view of the rocky Martian landscape, Utopia Planitia (see Figs. 1-2 and 1-3), which was reconstructed from satellite data sent back by a Viking spacecraft stationed on Mars. With a change of software, he'll be flying across the sprawling canyons of Valles Marineris later this afternoon. Just down the hallway in the "view lab," Dr. Stephen Ellis is remotely controlling robots, wearing VR goggles and a wired glove. The glove senses the movements of his hand and fingers, and a computer interprets and transmits them as commands for a robot arm. Ellis sees what the robot sees and the robot mimics his movements, even though he and the robot are in different rooms and can't see each other. Someday, using VR, robots will act as human eyes, arms, and ears on other planets, allowing many people to visit space as virtual astronauts. In San Diego, California, another Virtual World center has opened in a shopping mall. The first in a coming wave of location-based virtual reality entertainment companies, it's a mixture of Disney theme ride, video arcade, and sci-fi novel, with a decor that spans Victorian England and colonized Mars. Sipping cappuccino in the Explorer's Lounge, visitors await their turn to compete against each other while strapped inside space-age pods. All the pods are networked and customers compete in teams. The interior of each pod resembles a jet fighter cockpit with joystick, foot controls, and dozens of buttons. But instead of windows there are big computer screens that show you the other robot warriors, called BattleMechs, you're fighting in the interactive arcade cyberspace called BattleTech. To think of BattleTech as a giant video game doesn't do it justice. "We have to make a game, adventures, of such depth that there is almost limitless room for you to grow in your capability and experience," says Jordan Weisman, president of Virtual World Entertainment . In Seattle, Washington, Chris Esposito runs the Computer Interface Group at Boeing Corporation. The company has committed to completely developing its next generation of commercial aircraft using computers. "We've already identified three dozen applications for virtual reality, spanning the entire range of a product's life-cycle. From concept development to manufacturing and maintenance operations, even in-flight uses. And the list is getting longer." At the Center for Creative Inquiry at Carnegie Mellon University, Carl Loeffler's team has been homesteading cyberspace. They've built a virtual museum with several galleries and even the first block of a virtual city that simultaneously welcomes networked visitors from Tokyo, Vienna, and CMU. Together with Professor Lynn Holden (an Egyptologist), Loeffler is now trying to start a revolution in education. Wearing VR goggles with 3-D headphones, visitors tour a mockup of the ancient Temple of Horus. Photographs of actual carvings and re-creations of real murals cover the 60-foot-high walls. Moving through the many rooms of the temple, past huge pillars, your presence brings murals to life by triggering embedded animations. When you reach the innermost shrine, a statue becomes your guide, explaining the secrets of the chamber, while ancient Egyptian chanting fills the background. Lynn Holden sees the virtual temple as the first module in an entire ancient virtual world. He wants to blend computer game theory and role-playing with VR to experientially teach about ancient cultures. As in a video game, a student would advance to different levels (from peasant to warrior to pharaoh) by living in each character's private, professional, and religious world (a farm, village, fort, city, temple, or palace). At a grand old cathedral in San Francisco that has been converted into a theater, on Saturday night a full-house audience puts on 3-D glasses to visit Invisible Site, a George Coates Performance Works (GCPW) multimedia play that uses actors, 3-D projections, film, realtime animated computer graphics, and some sophisticated theater tricks to give people a taste of "virtual virtual reality." "It's a way of evoking the sense of awe that immersing yourself in virtual reality produces," George Coates says, and adds with a chuckle, "Someday we'll be handing out VR goggles instead of 3-D glasses." On stage, characters enter a store of the future, called Invisible Site, that sells virtual reality experiences. Once inside, as customers, they dress in goggles and gloves. Then they enter cyberspace through a computer network, where they visit new worlds and assume completely new identities. After the show, some of the audience heads towards San Francisco's nightclub dance district, where Toon Town is getting ready to open its doors. Like George Coates, Toon Town has found an audience hungry for new media concepts--using computers to visit, simulate, create, and control new worlds. In a corner of the club, dozens of people line up to pay for a chance to sit down, put on goggles, and fly around in a virtual world. "They all expect to be using one at home in a couple of years," says Vince Thomas, one of the club's owners. A second group of people in the club is gathered to use the Mandala Machine, where a Toon Town patron is dancing and playing imaginary musical instruments. Her image is captured by a video camera and fed into a computer, where it's combined with interactive video and animation, and projected onto the wall in front of her. She controls the animation through her movements as she watches herself on the wall in the scene--with synthesized notes erupting as she slaps the virtual drums (see Fig. 1-4). This is a very small sample of what's going on in the exploding realm of virtual reality. Don't worry if at first you feel a bit confused. VR is both a new technology and a set of ideas--concepts that are spreading faster than the hardware and software that produce them. There are several different approaches that fall under the label of virtual reality, though they don't all use the same effects to achieve the same results. VR is happening worldwide--inside giant corporations, universities, and small entrepreneurial start-ups; in Europe and Japan; and especially in the San Francisco Bay Area. VR has spawned a new interaction between musicians, artists, entrepreneurs, and electronic tinkerers. A rare excitement is in the air, an excitement that comes from breaking through to something new. Computers are about to take the next big step--out of the lab and into the street--and the street can't wait. Experiental computing "As long as you can see the screen, you're not in virtual reality. When the screen disappears, and you see an imaginary scene . . . then you are in virtual reality." Gabriel D. Ofeisch, Emeritus Professor of Educational Technology at Howard University Virtual reality is all about illusion. It's about computer graphics in the theater of the mind. It's about the use of high technology to convince yourself that you're in another reality, experiencing some event that doesn't physically exist in the world in front of you. Virtual reality is also a new media for getting your hands on information, getting inside information, and representing ideas in ways not previously possible. Virtual reality is where the computer disappears and you become "the ghost in the machine." There's no little screen of symbols you must manipulate nor type commands into to get the computer to do something. Instead, the computer retreats behind the scenes and becomes invisible, leaving you free to concentrate on tasks, ideas, problems, and communications. For three generations, people have experienced a type of virtual reality using the telephone. It has now been over 100 years since the introduction of the telephone, and today it's so deeply embedded in our cultural consciousness that we give it as much thought as a doorknob. As a society, we've forgotten what a shock the telephone was when it was first introduced--the strangeness of listening to a ghostly, disembodied voice. Today we rely on the telephone not only to communicate, but to give us the sense of someone else's presence, even though in reality we're listening to an electromechanical recreation of a human voice. We've learned to ignore the telephone and concentrate on the conversation; the interactivity makes the difference. Alan Kay, computer pioneer and Apple Computer fellow, has thought a long time about the way humans and computers could work together. Part of his concept of a Dynabook (a mobile, networked, multimedia, clipboard-sized computer), conceived more than 20 years ago, is coming to fruition today in small notebook computers and in machines like Apple's Newton. What's still missing is the point-of-view simulations he included in his Dynabook vision, the kind of simulations VR now makes possible. His product was people-oriented, not hardware-oriented. He was interested in the way technologies could be used by people, and the way they disappeared when their ease of use and functionality crossed a certain threshold. "I read McLuhan's Understanding Media and understood that the most important thing about any communications medium is that message receipt is really message recovery; anyone who wishes to receive a message embedded in a medium must first have internalized the medium so it can be subtracted out to leave the message behind." (Kay, 1990) Media is the plural form of medium. Webster's Ninth New Collegiate Dictionary describes medium as ". . . a means of affecting or conveying something . . . a channel or system of communication, information, or entertainment." Computers are a medium, a means or agency for doing something. What Kay suggests is that for the computer to really work, the way we use it has to become so comfortable, familiar, and effortless (like the television, telephone, and light switch) that we can unconsciously subtract it out and focus only on its function--the communication, information, and entertainment it provides. Like the telephone, computers are a communications medium. Like the telescope and microscope, computers are a tool for revealing new ways of looking at information. With virtual reality, the computer equipment disappears; it gets subtracted out. VR gives users an efficient and effortless flow of data, details, and information in the most natural format possible--vision, sound, and sensations presented as an environment, part of the natural data forms of human experience and thought. The first 21st-century tool "The primary defining characteristic of VR is inclusion, being surrounded by an environment. VR places the participant inside information. " Dr. William Bricken, Human Interface Technologies Lab New inventions are rarely appreciated for what they are when they first arise. Around the turn of the century, when the automobile first appeared on city streets, it was called a horseless carriage. Built like a buggy, it moved about as fast as a horse and was considered an expensive novelty. Nobody could imagine then that we would become so dependent on them, that they would change our way of life, the face of the land, the economy, and the quality of the air we breathe. Often inventors can't foresee what their discoveries will actually be used for, or the side effects they might cause. Alexander Graham Bell thought the telephone would be useful as a way to pipe music to people. The developers of radio (and later television) imagined that their devices would launch a world of two-way communications to replace the telephone. How then are we to understand this new method of using computers? Is virtual reality new, or just another form of television? Is it similar to or different from multimedia, the other new way of using computers to get at information? Can VR let us see things we've never seen before? If so, how? The first great wave of the computer revolution was hardware based; mainframes, mini-computers, and personal computers automated the workplace. Then computers became office rolodexes, electronic filing cabinets, typewriters, accounting ledgers, drafting tables, and the company mail. They changed the way individuals work, providing them with access to more information and new forms of communications, but people have had to struggle with the computer to get at the information--often alone. Now, as an increasing amount of media is converted into the digital language of computers, computers are beginning to adapt to people. Digitizing analog signals translates the media of sound, video, text, and graphic images into machine language, a mixture of ones and zeros (bits) that can be decoded and shared by other computers. With digitization, all media and information can be intertwined and blended together--sometimes even transformed into each other. The older media of television and radio deluge the viewer in a continuous stream of information that cannot be customized. Television and radio are passive experiences, with users as viewers or listeners. Information control is limited to an on/off switch and channel surfing. Computers give the consumer more control over the information, over their experience. Control now means mixing, matching, and blending information and media. Multimedia and virtual reality are the two best examples of this change. "Being able to mix together existing [media] forms, such as photos, images, sound, and books, and then come up with something new is what multimedia is all about," says Sandra Morris, multimedia developer and author of the book Multimedia Applications Development Using DVI Technology. "The difference between virtual reality and multimedia is that VR is about creation, while multimedia is about bringing the old media forms together into the computer. They don't change so much as they get combined in new ways. VR is about creating something completely new." Multimedia and virtual reality both benefit from digital technology developments. However, as Morris points out, a large distinction between virtual reality and multimedia is the creation of environments ("worlds"), versus the juxtaposition of existing media. Multimedia allows someone to watch a famous play on half of a computer screen while following along in the script, or with a video lecture by the playwright, on the other half. In another application, a person can study the image of an ancient stone tablet while hearing what it says in the simulated voices of ancient Egyptians. At the same time the hieroglyphics on the tablet light up in synchronization with the spoken words, accompanied by English subtitles at the bottom of the screen that reveal grammar and syntax differences. In virtual reality, the smallest details of sensory experience, the building blocks of human reality, are used to create environments. The location, tone, tempo, and pitch of each sound is a creative choice in VR. The entire visual range of experience--color, hue, brightness, saturation, the location and behavior of shapes--is under the VR developer's control. A virtual world might contain forms of old media, or multimedia, but it's used to populate the complete environments it creates. Virtual reality allows a person using a computer to assemble complex three-dimensional star fields of the galaxy and fly around in the data to understand how the universe is structured. Different types of stars can be given appropriate colors reflecting their size and age. This research is being performed by astronomers at the National Supercomputing Center in Ohio, using Cray supercomputers. As a result, the astronomers are now seeing things in the data they couldn't experience when it was just numbers on a page. One surprise was the way vast numbers of galaxies are clumped together in threads, ribbons, and clusters, leaving tremendous, empty voids between them. By speeding up the movement of all these galaxies, you could sit among the stars and see what an expanding universe looks like. The various forms of virtual reality There are a variety of different formats and systems calling themselves virtual reality. Throughout this book, we'll try to carefully categorize what we mean by virtual reality, and use more specific labels to separate the various forms. As a guideline, a virtual reality system should have the three following characteristics: response to user actions, real-time 3-D graphics, and a sense of immersion. It isn't enough to have just one or two of these properties; all three should be present. Because immersion is such a key aspect of a virtual reality system, it's important to explore it a little before venturing on. An immersive experience is one so absorbing that you cease to notice your surroundings or "how you got there." Current immersive systems stimulate your visual and aural senses in such a way that you find yourself immersed in a computer-generated experience. Though it's possible for sensory immersion to someday be-so complete that the mind is convinced it's really in another world, current implementations fall far short of this. Total sensory immersion would theoretically include temperature, tactile sensations, bodily movement, sounds, images, and even odors. Few of these senses are understood well enough to allow a computer to re-create them, despite Hollywood's many imaginings to the contrary. Keeping the three characteristics of a virtual environment in mind, let's now review existing attempts at creating immersive environments . Cab simulators Flight simulators for airplane pilots are the original virtual reality experience and the most familiar example of cab-based simulation. A user is seated inside a box that resembles the interior of a car, the cockpit of a jet, or a Star Wars' starfighter. The windows of the pod have been replaced with high-resolution computer screens. In some cases the entire pod is bolted onto a moving platform. As the user drives or flies the pod, the images on the "windows" and the tilt and angle of the cabin change in response. With the addition of surround-sound effects, a very realistic and thrilling interactive experience is created. The BattleTech game at the Virtual World center mentioned at the beginning of the chapter is an example of cab-based simulation. So is FighterTown, where you can fly an F-16 jet fighter simulator and shoot down enemy aircraft.
