fictional rollercoaster park"Multimedia and computers are to the twenty-first century what writing and paper were to the last several centuries. Together they make up the core infrastructure for the intellectual traditions of communication, composition and calculation." (R.S.H., 2003)

3D & Virtual Reality

A Multimedia Toolbench Chapter

Concepts

To work out visual problems that involve width, height and depth means thinking with and about three dimensional designs (3D). When animation is included, a fourth dimension, time or more accurately change over time is added. 3D represent the most sophisticated kind of computer spatial modeling available on the digital palette and provides more information than the two-dimensional view. 3d sceneVirtual reality refers to computer generated simulations of three dimensional views or worlds. That is, as in the pictures above and to the right, virtual reality is a scene of three dimensional panoramas and objects. There are two different forms of 3D. One form uses techniques of visual perspective to display 3D images on a two-dimensional surface such as a painting on canvas or a computer screen. The other form is sometimes called immersive VR, the three-dimensional stereoscopic view achieved by duplicating the slight separation of our two eyes with two slightly separated physical or virtual cameras. This effect is sometimes achieved with special viewing glasses or devices. The concept of 3D has roots deep into the history of the human species.

Beyond the practical application of being able to think in three dimensions for survival, for example - the construction of shelter or the creation of a tool, 3D has found also expression in our art and science. Accuracy with 3D perspective has long represented one of the peaks of drawing and painting skills. 3D is expressed in computer display in different ways for a wide range of digital simulation purposes from commercial advertising, to art for still image and cinema, to scientific visualization, and to digital gaming. With the advancement of computer and network capacity, the capacity for 3D composition and application is moving beyond highly skilled professionals and into the hands of every greater numbers of people.

Navigating, Reading and Understanding 3D

For novices to this area, experiencing computer-based 3D composition may be more important initially than reading about it. The chapter frame pages will be used to show different expressions of 3D: a two dimensional animation using 3D images; a still image and a virtual reality objects and panoramas. Clicking 3D demo will reveal the examples on the right. Many of these examples were done in Bryce, a 3D modeling application that can also do 3D animation. A brief Bryce 7 tutorial reveals the basics on doing such work.

  1. In the upper right frame is a Flash animation movie that uses images created with a 3 dimensional software application called Bryce.
  2. Scroll back up to the top of the page you are reading (bottom left frame) to see the still image drawn in 3D also using Bryce. Clicking on this 3D still image will reveal a form of virtual reality, a 360 panorama that can be spun by clicking and dragging with the mouse. Click the minus and plus symbols in the bottom left of the display to zoom in and out.
  3. The Demicron 3D application called WireFusion created these examples.  In the bottom right frame, pick picture number one in the gallery which will pop open a 3D object, then re-position this box so you can continue to read. This object can be controlled by the arrow symbols in the scene, or with different combinations of mouse buttons. Slowly and in this order, click and drag in the scene with the mouse: rotate the object using the left button; zoom in and out with the right button; and then pan by pressing both buttons at the same time before clicking and dragging within the scene. There are two species of 3D designs here, objects and scenes. Seek out and try both. Where needed, drag the edge between the frame pages to the far left to allow the full width of the scene to display.
  4. Panoramic scenes have many educational uses, from gaining greater understanding of past cultures to inventing the future, through the architecture of buildings and space stations. Explore "Virtual Worlds for Art History Teaching. Again, stop the music that may still be playing in the top left frame by clicking its play triangle as some of the these next scenes will have their own audio. Scenes will display and become useable in the right frame using the web 3D applications of Blink 3D and Atmosphere. For a better look, right click example pictures and choose to Open in a New Window, to see examples of these forms of virtual reality.  Links to even larger displays are provided toward the bottom of each web page.
  5. Such capacity led to the recent explosive development in 3D gaming which has gained further momentum through the integration of the technology of the Internet. The next step in 3D design has been to merge the 3D capacity of personal computers with the community building power of the Internet, moving 3D simulations from interaction between a person and a machine to interaction with people as 3D objects. Systems that provide excellent visual design, problems to solve and social interaction provide significant opportunity for educators.

Click for a picture of the home world of AWEDU, the educational universe being created at Activeworlds. Real understanding requires use and participation; the picture does not do justice to the audio and animated characters and full integration of all forms of web and media. As 3D rendering capacity is currently not built-in to the standard web browsers and operating systems in use today, compromises must be made during a time of transition. Separate applications must be downloaded and installed. Compromise designs are represented by the proprietary and commercial ActiveWorlds and 2nd Life and the open source and free world of OpenSim. Visit one or more of these designs and download their client software; test it out. Efforts are underway to make it possible to move avatars and objects between the worlds of different software applications using an application called Mryl.

The people working on several free 3D and immersive VR projects including Croquet/Cobalt, Project Wonderland, realXtend, Open Simulator (OpenSim), and an open source version of Second Life formed the Immersive Education Initiative.

The future of 3D will require entirely new computer operating systems, whose most visible development can be seen in the superbly engineered but still emerging Croquet Consortium project which is more recently being called the Open Cobalt project which has its own Web site. See: Using Croquet on a Smartboard - Croquet for kids; and the Quick demonstration of Open Cobalt. This Croquet application can be downloaded and run across Mac, Windows and Linux operating systems but it still a developer's build, not configured for end-users to begin productive and creative activity. (Figuring out how to start it up after downloading takes some time.) Supportive sites and resources are also beginning to appear such as CobaltWorlds.

The Edusim 3D project which is a slimmed down version of Open Cobalt for allowing multiple clasrooms to connect their interactive whiteboards for collaborative learning sessions. See the movie demonstration.

Google has chosen to create virtual worlds modeled on the real world instead of fictional words and made the creation of compositions for their emerging 3D world simple; download and explore Google Earth and the basic version of SketchUp, that is used to create 3D shapes that appear in Google Earth. They are both free. Further, the SketchUp elements of the object library built by Google Earth users can be imported into Cobalt and Edusim simulations (YouTube demo).

Another more advanced collaborative 3D virtual world project was started by the Sun Corporation, a 100% Java open source toolkit called Wonderland. When Sun was bought by the Oracle Corporation, their development of Wonderland was ended. See the YouTube demonstration prepared for the 2010 Summit conference. A group of interested programmers that picked up the development as the Open Wonderland project.

To help your eye focus back on this text again after viewing many examples, it may help to use this green screen cover to clear the display on the right.  Again, drag the edge between the frame pages to decide how wide a display feels best when reading this text. In the upper left frame, the Start Menu frame, notice the links for clearing three of the frames of detail if needed while reading this chapter.

 

History of 3D

There are many steps in culture's use of 3D design. Some of the earliest cave paintings from thousands of years ago show that the shapes of the cave wall itself were used to give more lifelike features to the images. Moving from natural shelters to the purposeful shaping of space, structure, and scale led to the art and science of designing and erecting buildings, that is, architecture. The art of sculpture shaped from wood, rock and more can be seen as a highly refined form of three dimensional reality. Renaissance artists in the 1400's, led by Brunelleschi, invented and developed the concept of perspective drawing, a topic later refined in in science and mathematics as isometric projection. The first attempts at photographic three dimensional image storage and representation harkens back to the stereoscopic images in postcards and later Hollywood movies, television and more.  Imagery requiring stereoscopic viewers that go back into the 1800's can be found at the Library of Congress. To view these, search the American Memory Project search page for stereoscopic. This work was followed with 3D film beginning in the 1920's and most recently represented by the spectacular Hollywood film Avatar. Later 3D television appeared in the 1950's and 1960's and was used for numerous television showings. In 2010, Samsung (the world's largest manufacture of televisions) along with Toshiba, Sony, Panasonic, and LG, introduced 3D TV sets that work in partnership with LCD shutter glasses. This was accompanied at the same time by the emergence of multiple groups that promised 3D TV channels in 2010 that will be exclusively in 3D, such as ESPN 3D sports, the British based Sky 3D, and Discovery Channel.

The first 3D computer graphics or modeling also appeared in the 1960's and as computer power increased led to 3D computer animation. One early format for 3D that spread rapidly through the Internet was the Quicktime format which provided object and panorama control. One of the cutting edges of research and development in 2010 is this work on 3D networked operating systems such as Croquet.

As more people began experiencing various degrees of immersion in the 3D experience while wearing specialized glasses, some potential negative effects on human health began to appear. Samsung has perhaps best summarized the concerns in its Web published 3D Warning statement (Web, pdf) which highlighted a number of concerns about: potential epileptic seizures; magnification of problems with pregnancy, lack of sleep, acohol and other drugs; motion sickness and other forms of feeling ill; and disorientation after 3D viewing that might make nearby balconies, stairwells and walking obstructions a danger.

True three dimensional design (3D) applications for desktop and larger commercial settings, such as Bryce, Maya or Cinema 4D, can create elements and scenes that might be used in not only virtual reality applications, but in movies, games, printed publications and other settings. One quick example of the perceived value of virtual reality applications would be NASA's (find the hotspots in the VR images) and Ford Motor's purchasing and installing SGI® Reality Centers, along with several hundred other buyers of such systems around the world. By using large screens, up to 30 people at a time can view what the computer is displaying. This allows different design experts to be able to collaborate at the same time.  This also allows planners and engineers to collaborate more effectively long before the availability of the mission or product begins. Numerous further examples of virtual reality devices and screen displays can be found by searching Google's image database for display and "virtual reality" and many more examples of 3D can be found by searching Google's image database for "3D".

Later, those designing for the Internet developed a display protocol called VRML (Virtual Reality Markup Language) around 1994. VRML used text characters to enable special browsers to create a fully interactive scene of animated objects with full motion of character objects called avatars. Many have created VRML development tools, but none became widely popular until Adobe brought out a beta version of the Atmosphere application in March of 2001 and then released version 1.0 of the Atmosphere application which produced many fine sites after November of 2003. For a variety of examples of Atmosphere compositions, see the link the chapter table of contents for interactive objects VR. Unfortunately, by December 2004, Adobe withdrew the popular application from the market. Some Atmosphere compositions can be found on the Internet by searching for "Atmosphere gallery" or related search terms, but often only descriptions of the Atmosphere VR scenes are left. VRML has been superceded by X3D, a new standard for real-time 3D computer graphics on the web, yet no standard has captured widespread adoption.

Immersive VR goes much further than VRML, moving the participant from a third person to a first person point of view. By wearing special goggles, some with small television cameras inside, the user feels that they have entered the scene, instead of just looked at the scene from a distance. This concept provides a first person point of view by design. A fine introduction to one scientific application of virtual reality can be found in Worth Publisher's video collection called Scientific American Frontiers (tape: Segments 27-35) Segment 32 is on "Virtual Fear" and segment 33 is on Arachnophobia, in which virtual reality experiences are used to reduce a variety of different kinds of phobias. For a more futuristic Hollywood treatment with an educational twist, ask your video rental company to find the 1992 movie called Lawnmower Man. Mechdyne bought Fakespace which is just is one company currently specializing in such displays. Display systems that surround the participants in the VR scene on 3 or more sides are also referred to as CAVEs.

Though business (1, 2, 3), science (1, 2, 3)  health ( 1, 2, 3) and emergency workers and defense (1, 2)  institutions have long incorporated 3D and virtual reality features in training and planning, education is only beginning to correlate this form of composition with school curriculum. The most extensive work on this has been done by Dr. Veronica Panelidas, North Carolina State University. Her publication on North Carolina Competency-Based Curriculum Objectives and Virtual Reality provides examples in almost every content area at every grade level of ideas and applications for the integration of virtual reality activities in ways that support the state's curriculum objectives. This document and other related work at their site provides many ideas for those interested in composing 3D and virtual reality designs that would have direct impact on classroom activity.

What has motivated this long period of development toward more refined 3D composition? In short, images and scenes of three dimensional objects provide additional data not available in two dimensions, and make it them highly portable and editable. Their degree of closeness to reality attracts our attention. Their special perspective can put difficult to understand information in a frame of reference that makes it more comprehensible and opens the door to new ways of thinking. Once constructed, the virtual image can be used to test and refine the image or model that is being created. It can also be used to test a user's sense of being in the simulation, a first-person factor called presence. If enough data is sent to the brain, the user takes the heavy streams of sensory information and can feel that the simulated world is real, even if clearly fictional.

The concept of three dimensional perspective is based on visual perception. There is a larger conceptual framework based on the framework of thought and computer technology. As these chapters point out, current composition on a computer has nine dimensions or perspectives, including text, still images, video, audio, animation, 3D, and sensors and remote control and more. A further challenge to composers of the twenty-first century is how to further engage the "reader" through the integration of all media forms discussed in this and previous chapters.

Input

From the morning cartoon shows to architect sketches to Internet web pages, three dimensional digital views of figures and of the world increasingly have become part of our awareness. These images start from a wide variety of sources.

Simple still shots from common digital cameras can be turned into 360 panoramas. Map data which includes digital elevation numbers can be turned into three-dimensional visualizations of terrains. Research data from almost any source can be used to create a 3D rendering that makes it easier to see patterns in the data for further reasoning. Even a series of overlapping pictures can be assembled into a larger virtual reality scene. Computer modeling software using wireframe outlines of objects later covered with textured surfaces are the primary tool used for computer based 3D animation and movies. Many different sizes of 3D scanners are also used to input three-dimensional information (handheld size, desktop units list of systems, search-3d scanner). In its most advanced from, immersive VR allows fast composition of virtual 3D scenes using data gloves body sensor suits, and other tools, somewhat similar to the way a child would build a scene in the physical world with blocks and toys. More sophisticated gloves contain motors and gears that provide push back, or a sense of touch, also called haptic technology.

One of the simpler 3D image techniques has been digital panoramas, sometimes referred to as Quicktime VR or 360 panoramas. A series of photographs are taken in a circle or in a cube (ceiling and floor). Software then seams them together into a single file which might allow rotation horizontally or vertically. More advanced animation software is used to create 3D images from scratch, generally working from 3D objects. Once defined, an object can also be assigned a direction and speed. Hotspots within these images can link panoramas to each other to create large interconnected scenes of real or virtual space. Many 3D design programs, such as Bryce and Maya, enables the composer to create the Quicktime three dimensional images within the program itself, both as still images and as animation.

Attempting to recreate the power of three dimensional sculpture, the digital view is something more and something less. Scaled down to fit the small display space of a computer monitor, 3D can seem but a shadow of the real world.  Wireframe design software is used to put the viewer in the scene. Instead of imagining a world through seeing it on a flat surface, special viewing devices create the perception of being with the scene. However, such digital images create a new world that can seem infinitely large once the user dons VR goggles and joins the world of immersive VR in which one is in the display instead of exterior to the presentation. On a computer or movie screen, the viewer is relegated to the role of observer status, always standing outside the scene. Our imagination enables these digital shadows to become powerful symbols and designs that help express ideas and explore physical space.

Manipulation

Output

Standard output devices include still image formats such as computer screens or printed material. Video and cinema (film) display systems still provide the most exposure to 3D compositions.

Room Output

More sophisticated virtual reality work desk by SGIdisplays can be found on high speed computer workstations as well as special game workstations such as found in flight simulators and video arcades. Larger  systems include desk size, wall size and immersive room sized systems. Click the picture on the left for other examples of visualization facilities. (Image on left courtesy of SGI.)

Multimedia Synthesis:

Integrating Virtual Reality & 3D with Text, Sound, Still, Animation, and Video

Virtual reality on a computer screen becomes just one visual element for which a frame page can be made. Displaying all the media elements at once not only challenges Internet speeds but the the capacity of the users computer workstation. For some time, designers will need to make tradeoffs, such as displaying very small virtual images or directing a small number of media objects to appear at the same time. In the not too distant future, computer and network speeds and new operating system concepts such as those promoted by the Croquet project will make compromises unnecessary.

A major design challenge for fictional work is the creation of a compelling story that the user can follow that includes all the media that is currently web accessible and makes simultaneous use of other applications that are specialized for 3D web use. For essay and curriculum development work, the challenge is simply too learn sufficient composition skills to provide a set of foundation experiences on which to build further.

One of the easiest places to begin is to use the existing proprietary 3D simulation worlds where so much of the needed VR tools are in place. The software provides a split-screen option where links in the 3D world link to browser pages that enable composers to integrate work from the current version of the Web.

Summary

At this point in the digital age, there has been an explosion of work in virtual reality that involves nearly ever form of endeavor as almost everyone finds the reality of 3D of value. Psychologists employ it to help clients deal with phobias. Lawyers want it to recreate the scene of an accident. Medical personnel use VR to model body parts that enable surgeons to practice their surgery of virtual renderings. Pilots train for emergency procedures. Educators build web simulations that invite dozens and hundreds of learners to explore and interact within a 3D world. The selected bibliography below hints at the richness of the state of the art in virtual reality and 3D.

Virtual Reality & 3D Bibliography

Bibliographies: Still - Audio - Video - 2D - 3D - Sensor - interact - MM

Brains (People)

 

Shelves (Libraries)

Hard drives (the Web)

  • 3D & VR Software Products: Tutorials, Reviews, Galleries, Costs

    Selected Virtual Reality and 3D Tools
    Title Access Online Tutorials Reviews Examples Publications Company Costs
    . . . . . . . .
    . . . . . . . .

    3d Publications

  • Virtual Reality & 3D

    QTVR Quicktime Based VR

    QTVR Examples

    QTVR Development Resources

    GIS (Geographic Information Systems)

     



    3D & VR Home Frame | Multimedia Home

    version 4.01 Pub. 4.16.2010. Page author: Houghton- Web Office