Reality (AR) Future
Reality Overlayed on Reality. A system allowing
you to see virtual world objects overlayed on your physical surroundings.
I confess i thought i was the first person to conceive this, while on holiday
in early October 1996. I called it "Virtual Reality Overlayed on
Reality". On my return from Holiday i searched all the available internet content for anything
on Virtual Reality that might mention simultaneous vision of cyberspace
anchored to real space, and for a week or so investigated the patenting
potential of the idea. Then on one of my Virtual Reality internet
investigations i stumbled across the term
"Augmented Reality", and discovered i was
not alone with the idea, although AR still doesn't enjoy the attention it
There is plenty of development required yet, and can there
ever be too many ideas or too much imagination in creating the future?
Screens. Live video input is merged with live computer input. This is
viewed by the user on a video screen. An example of this is ARGOS:
A Display System for Augmenting Reality.
- Head Mounted Displays (HMDs) that look like sun glasses. Each user
see’s their own perspective of Cyberspace overlayed on Reality
through special glasses. Consumer Electronics companies such as Sony will
deploy larger devices inside 5 years. AR HMDs
the size of today’s sunglasses inside 10 years.
Holographs. Limited ability to control viewing audience. Limited
development to date. 4. Direct Brain Stimulation. Inside 100 years.
Augmented reality at present is taking two directions, those of the first
and second options above.
In my opinion, head mounted displays will ultimately have
greater appeal. In spite of the negatives, that:
- HMD AR is not yet
developed as far as video options have been; and
- HMD AR
Doesn’t suit currently distributed technology (there are already
screens everywhere, but very few HMDs - yet);
positives win me over. HMD AR is:
Dependant on a monitor and video equipment;
(a monitor is not, unless your nose is pressed against the screen);
- Private -
if you choose, only you can see what’s in your HMD; and
These combine to afford a wider variety of potential
applications. Therefore, the remainder of this discussion will assume the Head
Mounted Display will carry AR into the mainstream consumer market, until it is superceded by Direct Brain Stimulation later next century.
How It Works
Put simply, a remote unit with trendy transparent LCD sun
glasses, GPS tracking, cellular communications (eg Wireless
Internet Connection), Client/Server systems.
Convenient, lightweight & COOL!
Mounted Display (HMD) with glasses at front. Reality seen through glass as
with conventional sunnies. VR seen as reflections against inside of
glasses. Reflection source projectors mounted at the side of the head.
This also minimises the nose heavy problem with many HMDs.
Plugs into portable PC / Network Computer / Net PC
HMD Virtual Worlds location tracked by Global
Positioning System (GPS) and orientation tracked by compass and spirit
level built into HMD.
Internet connection allows
of virtual world info from www Virtual Reality Modelling
Language (VRML) libraries online. Libraries may be searched by
proximity to GPS location as well as by key words.
with other units
Worlds anchored to Real World with six degrees of freedom, earthly
scenarios using global latitude, longitude and elevation from Satelite Global Positioning System (GPS), with roll
pitch and yaw. Off planet or galactic co-ordinates must be available for
non earth experiences.
- HMD and
control glove location and orientation changes and other events may be
uploaded to IRC for other users to download realtime
to see changes in other units location,
orientation and statii.
- Current realtime events occur in some form of live chat
session, such as Internet Relay Chat (IRC), either on a private or public
IRC style server. Either way, privacy is controllable using IRC style
The System’s Components, Their Inputs and Outputs
A standard is required to form the basis of all Augmented
Reality systems. This standard will be akin to VRML, and may simply be an
extension of this. For the purposes of this discussion, I’ll call it
ARML, (Augmented Reality Modeling Language -
obviously :) ). Inputs and outputs for each component
are detailed in figure 1 below.
The Applications Server
The Applications Server is configurable by the application,
the user or both. This controls what information is sent to you regarding other
users, minimising unnecessary traffic, while enhancing features of particular
applications. It’s configurable by:
Object ID, including other users Ids
The PC may download applications data such as graphics
objects, or load them off a local hard disk or CD.
Location and Orientation Information
Your PC is not forced to upload your location and
orientation. You may disable this if you don’t want others knowing where
you are. This will of course disable operation or features of applications
requiring your location. Intermediate security level will also be available on
your location information: Trusted applications you’ll nominate can be
access to your location and orientation info, whilst it’s restricted to
applications and other users. Location and Orientation Information is most
commonly by GPS, but ARML will also accomodate other
positioning systems. Examples include a Universal Positioning System for off
earth experiences, and a Mandlebrot Positioning System.
channels, with various levels of access, similar to the control available
Relay Chat (IRC). This gives options such as
on your own channel - you only, seeing only your cyber reality.
your own channel with friends or even strangers as you choose - as with
IRC, if they misbehave, you can kick them out! They can gain access by
one or a combination of triggers, such as:
proximity to you;
make a request of you; and/or
- Web TV and radio channels suspended
in space wherever you choose.
multiple channels enabled. For example, you’re driving to visit
your friend Robin, so the channels you have on are:
road signs for the area’ in your language;
Stuff in this area’ by a local party animal who markets his local
knowledge, and runs his operation like a radio station, financed by
local business' advertising; and
to Robin Smith’s Party in Cabrawood’,
and Robin’s set it only accessible by those invited to the party!
Favourite Internet Radio channel.
channel the passengers in your car are browsing automatically (by
‘proximity’ and ‘friendship’ attributes),
although, none that may cause danger in driving the car (government
Total Realism of Appearance
It may be necessary to see reality through the cyberspace,
so cyberspace appears transparent, to avoid being hit by cars, etc. This may be
handled by having declared safety zones - areas where it’s unsafe to hide
reality behind cyberspace, such as roads, dangerous walking trails in mountainous
areas, etc. This will require government involvement.
Configurable channels that allow communication based on
proximity to people or objects, which you can choose to have on or off (of
- There may
be a channel that allows you to publicise your interests, or alert you
when someone of similar interests comes into your proximety.
Either you, or another user or both of you (also configurable) are told
something about the person into whose proximity you’ve come. You
might prefer that you are not told of the proximity of people with whom
you have things in common, preferring the pleasant surprise of such a
person just saying hi! Or you may prefer to be told a little about a
person before saying hi. You may prefer the opportunity of looking them
over first. Or perhaps the mutual friend introduction, by voice audible
only to the two (or more) of you, or by a digital Agent. What a great way
to meet people with whom you have common interests.
channels that give you a guided tour of the area you're in.
channels used by companies to show the client what the finished layouts
and designs will look like, either on site or somewhere else where the
client can literally walk through.
Developers can allow authorities and investors visualisation of
developments on the actual building sites.
ideas in a meeting will no longer be limited to a 2 dimensional medium
such as a white board (formal meeting) or a coaster (informal meeting). A
3D design can be made to float in space amongst those discussing it,
modifiable by one, some or all, with the full editing capabilies
of 3D CAD design. This will allow colaboration
from around the world on a single project, as there can be many people in
many dispersed teams accesssing the same design,
or components of it.
one room on another, or connect rooms wall to wall. Two party rooms full
of people can then talk to each other.
Meetings. People from around the world can all appear to be in one room,
although they're in 6 different rooms around the world.
Agents. Your own cyberspace robots, with your choice of appearance. Cyberghosts. Visible only to yourself or to others.
Your robots may look and sound different to others compared with how you
signs on individuals...
signs on the backs of dorks (dependant on their GPS info) that say
"Kick Me" or "Insert Brain Here <---"
neon signs above bad drivers, warning of their bad driving so far today.
of this feature would have to be configurable, so that the person
carrying the sign could turn off the capability, or at least be made
aware of the sign. Otherwise, fear of humiliation would prevent people
even getting into AR.
Monster Scenario: You've just left the office for the day. Heading home
along a crowded city street during the afternoon peak hour, you see the
Marshmallow Man from Ghost Busters (or Godzilla, or that giant Cat from
cult series "The Goodies" episode, "Kitten Kong")
walking down the centre of the street! You're almost surprised to see it.
Other commuters appear oblivious to it. You see a flash of lightening
come from the other side of the monster, 100 metres up the road from you.
Although it was a hit, the monster was facing away from the blast - not
the best angle for attack. Without missing a beat you aline
the creatures head in the sites mounted in your sunnies, and press the
button on the handle of your briefcase... Zap. More Lightening - this
time it's yours. You know it didn't blind you, but it sure looked cool
from where other players and spectators, comuters
also on your channel, looked on... it's a Direct
hit! Right between the eyes! You win the credits for the game this
afternoon. What a great way to finish the day!
monster sized chess game covers the harbour. Spectators can see from any
for Existing games such as Doom and Battle Chess onto the real world.
Virtual Street Signs.
to friend’s Party, and they've set it only visible to those invited
to the party.
Gifts. Presents that can be left for another AR enabled individuals to
find. You want to surprise your friend. You aquire
a piece of digital art, something they would like, say a dynamically
unfolding sculpture that expands from pocket size into a like a slow
motion jack-in-the-box. You compress it, "wrap" it in flashing
animated fluorescent wrapping paper, and leave it somewhere they'll find
it. You can either choose that only the recipient can see the gift, or any
comination of others can also see it. But only
the recipient can open it or move it.
Treasure Hunt. A combination of the Personalised Virtual Street Signs, and
the Virtual Gifts above, where there's signs showing where the present is.
Or a full treasure hunt - a combination of clues each leading to the next
in a sequence until the final prize is uncovered.
draw plays directly onto the field for AR enabled folks at the game.
participants visible to spectators wearing AR gear.
- Astro Travelling option allowing
flight to anywhere anytime (optionally with visor down to shut out real
world) allowing conventional VR applications. You can 's
fly to another galaxy in in seconds or minutes,
without reality's limitations of the speed of light, the effects of flesh
crushing acceleration, or lack of oxygen... You're not wearing your space
- A HMD
mounted with a Digital Stereo video recorder can broadcast or store your
visual and sound experiences for enjoyment by someone else. You can then
enjoy your spouse’s eye view of your child while you’re away.
Video will allow particular locations to be constantly visible online,
publicly or privately.
digital video camera on a surf beach for anyone to check the surf via the
net (this currently being done using digital cameras).
digital video camera mounted at your home allows you to see and hear your
family when you’re away.
eye witness accounts by witnesses of driving accidents and roberies.
You can’t be the last one on of your friends to have
AR glasses. Everyone’s talking about stuff, looking at stuff, laughing at
stuff, and if you can’t see and hear the stuff, you’re out of it!
Mass Market Life Cycle
2008 - AR will inititially burst
into the public eye through consumer electronics gaming products from companies
like Sony, Sega, Nintendo
2013 - The growing market will encourage a group of the
smaller innovative AR manufacturers and software developers to develop and back
a modular AR standard, AR Industry Standard Architure
(ARISA). Using ARISA compliant gear you'll then be able to mix and match your
system components, similar to today's PC industry.
2014 - The first mainstream games will become available on
2015 - The big consumer electronics companies, seeing
market share going to the smaller standard compliant companies, will make there
proprietary products ARISA compliant to maintain their industry positions.
2015 - The first truly cool looking head mounted displays
will come on the market, completely opening the way for the scenarios described
Current Limitations to Networked
- Real and virtual objects must be properly aligned with respect to each
other, or the illusion that the two coexist will be compromised. Ron Azuma has done some interesting work
recently in resolving the registration issue.
doesn't yet appear to be any reference to global coordinates in the VRML standard.
These may be all that's required to allow VRML to operate as ARML.
lack of Wireless Internet connectivety (at time
of writing in 1997 - this is no longer an issue in cities).
lack of mass marketing.
We will live in a Networked Augmented Reality future.
The ideas expressed here are mine, although i've often later discovered, during searches on the
subject, others have thought along the same line!
Return to Matt's Home Page
a hello? It'll be Excellent to hear from you!
This page was last modified 20 February 2007.