Networked Augmented Reality (AR) Future

CopyrightŠ 1997

Virtual Reality Overlayed on Reality. A system allowing you to see virtual world objects overlayed on your physical surroundings.

I confess i thought i was the first person to conceive this, while on holiday in early October 1996. I called it "Virtual Reality Overlayed on Reality". On my return from Holiday i searched all the available internet content for anything on Virtual Reality that might mention simultaneous vision of cyberspace anchored to real space, and for a week or so investigated the patenting potential of the idea. Then on one of my Virtual Reality internet investigations i stumbled across the term "Augmented Reality", and discovered i was not alone with the idea, although AR still doesn't enjoy the attention it deserves.

There is plenty of development required yet, and can there ever be too many ideas or too much imagination in creating the future?

Methods

  1. Video Screens. Live video input is merged with live computer input. This is viewed by the user on a video screen. An example of this is ARGOS: A Display System for Augmenting Reality.
  2. Head Mounted Displays (HMDs) that look like sun glasses. Each user see’s their own perspective of Cyberspace overlayed on Reality through special glasses. Consumer Electronics companies such as Sony will deploy larger devices inside 5 years. AR HMDs the size of today’s sunglasses inside 10 years.
  3. 3D Holographs. Limited ability to control viewing audience. Limited development to date. 4. Direct Brain Stimulation. Inside 100 years. Augmented reality at present is taking two directions, those of the first and second options above.

In my opinion, head mounted displays will ultimately have greater appeal. In spite of the negatives, that:

  1. HMD AR is not yet developed as far as video options have been; and
  2. HMD AR Doesn’t suit currently distributed technology (there are already screens everywhere, but very few HMDs - yet);

the positives win me over. HMD AR is:

  1. Not Dependant on a monitor and video equipment;
  2. Immersive (a monitor is not, unless your nose is pressed against the screen);
  3. Private - if you choose, only you can see what’s in your HMD; and

These combine to afford a wider variety of potential applications. Therefore, the remainder of this discussion will assume the Head Mounted Display will carry AR into the mainstream consumer market, until it is superceded by Direct Brain Stimulation later next century.

How It Works

Put simply, a remote unit with trendy transparent LCD sun glasses, GPS tracking, cellular communications (eg Wireless Internet Connection), Client/Server systems. Convenient, lightweight & COOL!

  • Head Mounted Display (HMD) with glasses at front. Reality seen through glass as with conventional sunnies. VR seen as reflections against inside of glasses. Reflection source projectors mounted at the side of the head. This also minimises the nose heavy problem with many HMDs. Plugs into portable PC / Network Computer / Net PC
  • Special HMD Virtual Worlds location tracked by Global Positioning System (GPS) and orientation tracked by compass and spirit level built into HMD.
  • Wireless Internet connection allows
    • Download of virtual world info from www Virtual Reality Modelling Language (VRML) libraries online. Libraries may be searched by proximity to GPS location as well as by key words.
    • Interaction with other units
  • Virtual Worlds anchored to Real World with six degrees of freedom, earthly scenarios using global latitude, longitude and elevation from Satelite Global Positioning System (GPS), with roll pitch and yaw. Off planet or galactic co-ordinates must be available for non earth experiences.
  • HMD and control glove location and orientation changes and other events may be uploaded to IRC for other users to download realtime to see changes in other units location, orientation and statii.
  • Current realtime events occur in some form of live chat session, such as Internet Relay Chat (IRC), either on a private or public IRC style server. Either way, privacy is controllable using IRC style commands.

The System’s Components, Their Inputs and Outputs

A standard is required to form the basis of all Augmented Reality systems. This standard will be akin to VRML, and may simply be an extension of this. For the purposes of this discussion, I’ll call it ARML, (Augmented Reality Modeling Language - obviously :) ). Inputs and outputs for each component are detailed in figure 1 below.

Figure 1

The Applications Server

The Applications Server is configurable by the application, the user or both. This controls what information is sent to you regarding other users, minimising unnecessary traffic, while enhancing features of particular applications. It’s configurable by:

  • ARML Object ID, including other users Ids
  • ARML Object Proximity.

The PC

The PC may download applications data such as graphics objects, or load them off a local hard disk or CD.

Location and Orientation Information

Your PC is not forced to upload your location and orientation. You may disable this if you don’t want others knowing where you are. This will of course disable operation or features of applications requiring your location. Intermediate security level will also be available on your location information: Trusted applications you’ll nominate can be access to your location and orientation info, whilst it’s restricted to applications and other users. Location and Orientation Information is most commonly by GPS, but ARML will also accomodate other positioning systems. Examples include a Universal Positioning System for off earth experiences, and a Mandlebrot Positioning System.

Total Control

  • Unlimited channels, with various levels of access, similar to the control available in Internet Relay Chat (IRC). This gives options such as
    • Alone on your own channel - you only, seeing only your cyber reality.
    • Sharing your own channel with friends or even strangers as you choose - as with IRC, if they misbehave, you can kick them out! They can gain access by one or a combination of triggers, such as:
      • Physical proximity to you;
      • Common Interests;
      • Common Friends;
      • They make a request of you; and/or
      • You invite them.
    • Web TV and radio channels suspended in space wherever you choose.
    • Having multiple channels enabled. For example, you’re driving to visit your friend Robin, so the channels you have on are:
      • ‘General road signs for the area’ in your language;
      • ‘Cool Stuff in this area’ by a local party animal who markets his local knowledge, and runs his operation like a radio station, financed by local business' advertising; and
      • ‘Directions to Robin Smith’s Party in Cabrawood’, and Robin’s set it only accessible by those invited to the party!
      • Your Favourite Internet Radio channel.
      • Any channel the passengers in your car are browsing automatically (by ‘proximity’ and ‘friendship’ attributes), although, none that may cause danger in driving the car (government controlled).

Total Realism of Appearance

Totally Safe

It may be necessary to see reality through the cyberspace, so cyberspace appears transparent, to avoid being hit by cars, etc. This may be handled by having declared safety zones - areas where it’s unsafe to hide reality behind cyberspace, such as roads, dangerous walking trails in mountainous areas, etc. This will require government involvement.

Applications

Configurable channels that allow communication based on proximity to people or objects, which you can choose to have on or off (of course).

  • There may be a channel that allows you to publicise your interests, or alert you when someone of similar interests comes into your proximety. Either you, or another user or both of you (also configurable) are told something about the person into whose proximity you’ve come. You might prefer that you are not told of the proximity of people with whom you have things in common, preferring the pleasant surprise of such a person just saying hi! Or you may prefer to be told a little about a person before saying hi. You may prefer the opportunity of looking them over first. Or perhaps the mutual friend introduction, by voice audible only to the two (or more) of you, or by a digital Agent. What a great way to meet people with whom you have common interests.
  • Tourist channels that give you a guided tour of the area you're in.
  • Private channels used by companies to show the client what the finished layouts and designs will look like, either on site or somewhere else where the client can literally walk through.
  • Building Developers can allow authorities and investors visualisation of developments on the actual building sites.
  • Discussing ideas in a meeting will no longer be limited to a 2 dimensional medium such as a white board (formal meeting) or a coaster (informal meeting). A 3D design can be made to float in space amongst those discussing it, modifiable by one, some or all, with the full editing capabilies of 3D CAD design. This will allow colaboration from around the world on a single project, as there can be many people in many dispersed teams accesssing the same design, or components of it.
  • Overlay one room on another, or connect rooms wall to wall. Two party rooms full of people can then talk to each other.
    • Virtual Meetings. People from around the world can all appear to be in one room, although they're in 6 different rooms around the world.
    • Virtual Parties.
  • Virtual Agents. Your own cyberspace robots, with your choice of appearance. Cyberghosts. Visible only to yourself or to others. Your robots may look and sound different to others compared with how you see them.
  • Involuntary signs on individuals...
    • Cyberspace signs on the backs of dorks (dependant on their GPS info) that say "Kick Me" or "Insert Brain Here <---" !
    • Flashing neon signs above bad drivers, warning of their bad driving so far today.
    • Operation of this feature would have to be configurable, so that the person carrying the sign could turn off the capability, or at least be made aware of the sign. Otherwise, fear of humiliation would prevent people even getting into AR.
  • Games
    • Marshmallow Monster Scenario: You've just left the office for the day. Heading home along a crowded city street during the afternoon peak hour, you see the Marshmallow Man from Ghost Busters (or Godzilla, or that giant Cat from that BBC cult series "The Goodies" episode, "Kitten Kong") walking down the centre of the street! You're almost surprised to see it. Other commuters appear oblivious to it. You see a flash of lightening come from the other side of the monster, 100 metres up the road from you. Although it was a hit, the monster was facing away from the blast - not the best angle for attack. Without missing a beat you aline the creatures head in the sites mounted in your sunnies, and press the button on the handle of your briefcase... Zap. More Lightening - this time it's yours. You know it didn't blind you, but it sure looked cool from where other players and spectators, comuters also on your channel, looked on... it's a Direct hit! Right between the eyes! You win the credits for the game this afternoon. What a great way to finish the day!
    • A monster sized chess game covers the harbour. Spectators can see from any skyscraper window.
    • Ports for Existing games such as Doom and Battle Chess onto the real world.
  • Personalised Virtual Street Signs.
    • Directions to friend’s Party, and they've set it only visible to those invited to the party.
  • Virtual Gifts. Presents that can be left for another AR enabled individuals to find. You want to surprise your friend. You aquire a piece of digital art, something they would like, say a dynamically unfolding sculpture that expands from pocket size into a like a slow motion jack-in-the-box. You compress it, "wrap" it in flashing animated fluorescent wrapping paper, and leave it somewhere they'll find it. You can either choose that only the recipient can see the gift, or any comination of others can also see it. But only the recipient can open it or move it.
  • Virtual Treasure Hunt. A combination of the Personalised Virtual Street Signs, and the Virtual Gifts above, where there's signs showing where the present is. Or a full treasure hunt - a combination of clues each leading to the next in a sequence until the final prize is uncovered.
  • Sporting Events
    • Commentators draw plays directly onto the field for AR enabled folks at the game.
    • Virtual participants visible to spectators wearing AR gear.
  • Astro Travelling option allowing flight to anywhere anytime (optionally with visor down to shut out real world) allowing conventional VR applications. You can 's fly to another galaxy in in seconds or minutes, without reality's limitations of the speed of light, the effects of flesh crushing acceleration, or lack of oxygen... You're not wearing your space suit!
  • A HMD mounted with a Digital Stereo video recorder can broadcast or store your visual and sound experiences for enjoyment by someone else. You can then enjoy your spouse’s eye view of your child while you’re away.
  • Digital Video will allow particular locations to be constantly visible online, publicly or privately.
    • A digital video camera on a surf beach for anyone to check the surf via the net (this currently being done using digital cameras).
    • A digital video camera mounted at your home allows you to see and hear your family when you’re away.
    • Videoed eye witness accounts by witnesses of driving accidents and roberies.

Social Effect

You can’t be the last one on of your friends to have AR glasses. Everyone’s talking about stuff, looking at stuff, laughing at stuff, and if you can’t see and hear the stuff, you’re out of it!

AR's Mass Market Life Cycle

2008 - AR will inititially burst into the public eye through consumer electronics gaming products from companies like Sony, Sega, Nintendo and Microsoft.

2013 - The growing market will encourage a group of the smaller innovative AR manufacturers and software developers to develop and back a modular AR standard, AR Industry Standard Architure (ARISA). Using ARISA compliant gear you'll then be able to mix and match your system components, similar to today's PC industry.

2014 - The first mainstream games will become available on ARISA systems.

2015 - The big consumer electronics companies, seeing market share going to the smaller standard compliant companies, will make there proprietary products ARISA compliant to maintain their industry positions.

2015 - The first truly cool looking head mounted displays will come on the market, completely opening the way for the scenarios described above.

Current Limitations to Networked AR Deployment

  • Registration - Real and virtual objects must be properly aligned with respect to each other, or the illusion that the two coexist will be compromised. Ron Azuma has done some interesting work recently in resolving the registration issue.
  • There doesn't yet appear to be any reference to global coordinates in the VRML standard. These may be all that's required to allow VRML to operate as ARML.
  • Current lack of Wireless Internet connectivety (at time of writing in 1997 - this is no longer an issue in cities).
  • Current lack of mass marketing.

Related Resources

We will live in a Networked Augmented Reality future.

The ideas expressed here are mine, although i've often later discovered, during searches on the subject, others have thought along the same line!

              Matt

Return to Matt's Home Page

email.gifWhy not send a hello? It'll be Excellent to hear from you!
This page was last modified 20 February 2007.

CopyrightŠ 1997