Like Reality, But Better

While browsing the technology news recently, I kept coming across the term “Augmented Reality (AR).”  Not being familiar with augmented reality, but being always curious, I decided to investigate exactly what it is.  As it turns out, it’s awesome, so I’m gonna blog about it.

I’m sure you’ve heard of virtual reality.  Virtual reality refers to computer-generated worlds or environments in which a user can explore and interact with virtual objects.  Of course, under that definition, pretty much any video game counts as a virtual reality, but the more popular, science-fiction concept of virtual reality is that of a person completely immersed in a simulated environment, interacting with computer-generated objects, like the holodeck on Star Trek, or the Danger Room of the X-Men, or the world of the Matrix (just to name a few fictional examples).  While modern VR technology isn’t nearly as advanced as it is in sci-fi, this technology is being used and experimented with these days not only for the obvious gaming applications, but for various sorts of military training, and medical applications.

Augmented reality is a little different.  Instead of creating virtual worlds, AR technology superimposes virtual data onto the real world, essentially enhancing the real world with computer-generated graphics.  This actually isn’t such a new or uncommon concept – if you have a smartphone, you have access to augmented reality technology.

Augmented Reality "augments" the real world with computer graphics, as seen here.

At the most basic level, AR tech can be as simple as playing a game with reality as the backdrop.  You may have a game on your phone that has you shooting down enemy planes, but the background of the game is your phone’s camera, so it looks like the planes are flying around your living room.  Or the technology can simply add virtual graphics to images of the real world.  These amusing videos show AR technology being used at the Munich Airport, where astonished passengers were treated to real-time videos of themselves sitting beside virtual people who existed only on the video screen.  

The PlayStation EyePet is
a virtual pet you can interact with.

While those Munich videos are impressive, the really exciting stuff is where AR allows these virtual images to interact with the real world.  The RC vCar is an app for smartphones that allows you to drive a virtual car around real-world environments in the display of your phone camera.   The PlayStation EyePet takes the same concept as the Munich Airport – taking a video of the real world and superimposing a virtual character – but unlike the people at the airport, this game allows you to interact with that character on the screen.  In the real world, you’re just moving your hands around in mid-air, but on the TV screen, your virtual pet is reacting to you movements, jumping around and playing with you.  I can only describe these things so well in words – check out the videos in those links, this is really cool stuff!

Of course, games are the obvious outlet for this type of technology, but there’s more!  There are plenty of useful and interesting phone applicationss that use augmented reality.  StarChart is an iPhone app that allows you to point your phone at the sky and see information on the stars and constellations you’re looking at.  CarFinder is another iPhone app that allows you to tag the position of your car in a parking lot, then when you get out of the store, if you’ve forgotten where you parked, you just hold up your phone and it will point the way back to your car.  Similar technology is currently being explored with the aim of helping police officers zero-in on the location of a dangerous package or suspicious person in a crowd.  Among the most widely-known AR phone apps is Google Goggles.  Developed by the masterminds at Google, this app allows you to take a picture of an object and retrieve information.  You can take a picture of some landmark and the app would bring up information about it; you could photograph a book and the app could bring up book reviews; you can even take a picture of a sign in a foreign language and have the app translate it for you.  There are tons of awesome AR smartphone apps.  Do an internet search for “augmented reality apps” and you’ll find plenty of cool stuff.

Google Goggles are an augmented reality app that allows you to pull up information
from pictures you take with your phone.

Okay, so there’s a lot of neat augmented reality technology out there.  Awesome.  But at the Meniscus, I don’t just talk about what cool stuff we already have, I like to look to the future.  And the future of AR is looking really, really awesome.  Multiple groups of researchers are currently developing technology for eyewear that will superimpose graphic images directly into your field of vision.  Remember how, in the Terminator movies, he would look at stuff, and computer data would pop up in his eyes giving him information about what he was seeing?  Yeah, it’s like that.

Those geniuses at Google are currently working on a project called Google Glasses, which would essentially be glasses that do all the things Google Goggles do, but without the middle step of taking a picture with your phone.  As you walk down the street, you could pull up information on what you’re looking at – landmarks, stores, advertisements, even people (there’s a great ethics discussion there) – this information would be displayed not on your phone, but right there on the glasses in front of your eyes.  These glasses could also allow people to play fully interactive games with the world around them.  The implications of that technology are astounding; just imagine all the things you could use it for.  And that’s not some distant future technology – Google has reported that they plan to begin selling these glasses later this year!  

Terminator vision might soon
be reality (augmented, that is).

The same kind of technology is also being developed by Innovega researchers, who are aiming to create augmented reality contact lenses to put right in your eye.  The ultimate goal of this project is military use – imagine how useful “Terminator vision” would be to a soldier on the battlefield, or a pilot flying through enemy territory – but could also be applied to improving the vision of people with various sight impairments, and of course could be useful for gaming and web-searching just like the Google Glasses.  The benefit of the contact lenses (also the part that makes them more difficult to perfect) is that they are smaller and more discrete than glasses, which is particularly useful in the case of military application.  The researchers working on this technology are even considering the possibility of creating augmented reality lenses that could be implanted directly into the eye, making them permanent installments – true bionic eyes.  These lenses could then be used for all sorts of augmented reality applications, and perhaps even for true virtual reality as well.  Of course, it sounds like this technology is still at least a few years off, but hey, for bad-ass technology like that, I’m willing to wait a little while.

I can’t finish my augmented reality post without mentioning SixthSense.  I learned about this only recently, as I was amazed that I hadn’t heard about it earlier.  Developed by Pattie Maes and Pranav Mistry over at MIT, SixthSense is a “wearable gestural interface” that allows you to impose graphics onto objects around you and interact with them using gestures of your hands and fingers.  You can project a computer interface onto walls or floors and operate the computer screen just by moving your hands; you can play games on the surfaces around you; you can draw a watch on your hand and it will tell you the time; you can take pictures of the world around just by making a frame with your hands.  The is not some far-off technology, or even a later-this-year technology - it already exists!  Its only in its prototype stage at the moment, but it already looks amazing.  There’s no way I can describe this tech properly in words, so watch this video!  Seriously, if you click on no other links on this page, watch this video!  And then check out their website for more information.  This is some crazy-awesome technology.   

SixthSense is an augmented reality technology that can project images and videos onto real-world objects.

Augmented reality technology is going to be something to keep our eyes on in the coming years.  I can only imagine it’s going to get better and better, and the applications of it are limited only by the technology we have available (which is always improving) and the imaginations of engineers (which are seemingly endless).  I’ve said it before – the future is gonna be awesome.

If you know of really cool AR technology or awesome AR smartphone apps that I haven’t mentioned here, let me know in the comments!