Saturday, March 31, 2012

Like Reality, But Better

While browsing the technology news recently, I kept coming across the term “Augmented Reality (AR).”  Not being familiar with augmented reality, but being always curious, I decided to investigate exactly what it is.  As it turns out, it’s awesome, so I’m gonna blog about it.

I’m sure you’ve heard of virtual reality.  Virtual reality refers to computer-generated worlds or environments in which a user can explore and interact with virtual objects.  Of course, under that definition, pretty much any video game counts as a virtual reality, but the more popular, science-fiction concept of virtual reality is that of a person completely immersed in a simulated environment, interacting with computer-generated objects, like the holodeck on Star Trek, or the Danger Room of the X-Men, or the world of the Matrix (just to name a few fictional examples).  While modern VR technology isn’t nearly as advanced as it is in sci-fi, this technology is being used and experimented with these days not only for the obvious gaming applications, but for various sorts of military training, and medical applications.

Augmented reality
is a little different.  Instead of creating virtual worlds, AR technology superimposes virtual data onto the real world, essentially enhancing the real world with computer-generated graphics.  This actually isn’t such a new or uncommon concept – if you have a smartphone, you have access to augmented reality technology.

Augmented Reality "augments" the real world with computer graphics, as seen here.
At the most basic level, AR tech can be as simple as playing a game with reality as the backdrop.  You may have a game on your phone that has you shooting down enemy planes, but the background of the game is your phone’s camera, so it looks like the planes are flying around your living room.  Or the technology can simply add virtual graphics to images of the real world.  These amusing videos show AR technology being used at the Munich Airport, where astonished passengers were treated to real-time videos of themselves sitting beside virtual people who existed only on the video screen.  

The PlayStation EyePet is
a virtual pet you can interact with.
While those Munich videos are impressive, the really exciting stuff is where AR allows these virtual images to interact with the real world.  The RC vCar is an app for smartphones that allows you to drive a virtual car around real-world environments in the display of your phone camera.   The PlayStation EyePet takes the same concept as the Munich Airport – taking a video of the real world and superimposing a virtual character – but unlike the people at the airport, this game allows you to interact with that character on the screen.  In the real world, you’re just moving your hands around in mid-air, but on the TV screen, your virtual pet is reacting to you movements, jumping around and playing with you.  I can only describe these things so well in words – check out the videos in those links, this is really cool stuff!

Of course, games are the obvious outlet for this type of technology, but there’s more!  There are plenty of useful and interesting phone applicationss that use augmented reality.  StarChart is an iPhone app that allows you to point your phone at the sky and see information on the stars and constellations you’re looking at.  CarFinder is another iPhone app that allows you to tag the position of your car in a parking lot, then when you get out of the store, if you’ve forgotten where you parked, you just hold up your phone and it will point the way back to your car.  Similar technology is currently being explored with the aim of helping police officers zero-in on the location of a dangerous package or suspicious person in a crowd.  Among the most widely-known AR phone apps is Google Goggles.  Developed by the masterminds at Google, this app allows you to take a picture of an object and retrieve information.  You can take a picture of some landmark and the app would bring up information about it; you could photograph a book and the app could bring up book reviews; you can even take a picture of a sign in a foreign language and have the app translate it for you.  There are tons of awesome AR smartphone apps.  Do an internet search for “augmented reality apps” and you’ll find plenty of cool stuff.

Google Goggles are an augmented reality app that allows you to pull up information
from pictures you take with your phone.

Okay, so there’s a lot of neat augmented reality technology out there.  Awesome.  But at the Meniscus, I don’t just talk about what cool stuff we already have, I like to look to the future.  And the future of AR is looking really, really awesome.  Multiple groups of researchers are currently developing technology for eyewear that will superimpose graphic images directly into your field of vision.  Remember how, in the Terminator movies, he would look at stuff, and computer data would pop up in his eyes giving him information about what he was seeing?  Yeah, it’s like that.

Those geniuses at Google are currently working on a project called Google Glasses, which would essentially be glasses that do all the things Google Goggles do, but without the middle step of taking a picture with your phone.  As you walk down the street, you could pull up information on what you’re looking at – landmarks, stores, advertisements, even people (there’s a great ethics discussion there) – this information would be displayed not on your phone, but right there on the glasses in front of your eyes.  These glasses could also allow people to play fully interactive games with the world around them.  The implications of that technology are astounding; just imagine all the things you could use it for.  And that’s not some distant future technology – Google has reported that they plan to begin selling these glasses later this year!  

Terminator vision might soon
be reality (augmented, that is).
The same kind of technology is also being developed by Innovega researchers, who are aiming to create augmented reality contact lenses to put right in your eye.  The ultimate goal of this project is military use – imagine how useful “Terminator vision” would be to a soldier on the battlefield, or a pilot flying through enemy territory – but could also be applied to improving the vision of people with various sight impairments, and of course could be useful for gaming and web-searching just like the Google Glasses.  The benefit of the contact lenses (also the part that makes them more difficult to perfect) is that they are smaller and more discrete than glasses, which is particularly useful in the case of military application.  The researchers working on this technology are even considering the possibility of creating augmented reality lenses that could be implanted directly into the eye, making them permanent installments – true bionic eyes.  These lenses could then be used for all sorts of augmented reality applications, and perhaps even for true virtual reality as well.  Of course, it sounds like this technology is still at least a few years off, but hey, for bad-ass technology like that, I’m willing to wait a little while.

I can’t finish my augmented reality post without mentioning SixthSense.  I learned about this only recently, as I was amazed that I hadn’t heard about it earlier.  Developed by Pattie Maes and Pranav Mistry over at MIT, SixthSense is a “wearable gestural interface” that allows you to impose graphics onto objects around you and interact with them using gestures of your hands and fingers.  You can project a computer interface onto walls or floors and operate the computer screen just by moving your hands; you can play games on the surfaces around you; you can draw a watch on your hand and it will tell you the time; you can take pictures of the world around just by making a frame with your hands.  The is not some far-off technology, or even a later-this-year technology - it already exists!  Its only in its prototype stage at the moment, but it already looks amazing.  There’s no way I can describe this tech properly in words, so watch this video!  Seriously, if you click on no other links on this page, watch this video!  And then check out their website for more information.  This is some crazy-awesome technology.   

SixthSense is an augmented reality technology that can project images and videos onto real-world objects.
Augmented reality technology is going to be something to keep our eyes on in the coming years.  I can only imagine it’s going to get better and better, and the applications of it are limited only by the technology we have available (which is always improving) and the imaginations of engineers (which are seemingly endless).  I’ve said it before – the future is gonna be awesome.

If you know of really cool AR technology or awesome AR smartphone apps that I haven’t mentioned here, let me know in the comments! 


  1. This makes me think of the Star Trek hologram games as well as the dr. from Star Trek: Voyager. I remember watching those shows as a kid, wondering if we would ever have that kind of technology. I had no idea how close we (Though of course this "we" is a technologically/economically privileged "we" within the context of the United States) are to having access to these things. Thanks for sharing!

  2. Thanks for reading! I definitely share your sentiment - I'm constantly amazed at how much "future technology" is right around the corner, or already here. When we invent a way to "beam people up" I'll be sure to blog about it!

    A big part of these innovations is also making technology cheaper and easier to produce. That SixthSense device is already amazingly inexpensive even in its prototype stage. It might not be very long before this kind of technology can be made available to more than just the "economically privileged," as you aptly put it - a hopeful thought!

  3. I watched a video about the Google Glasses the other day. I wonder how often people will "use" this kind of technology? I have so many gadgets and apps but I honestly rarely use any of them. They are novelties in most cases. I find I don't even use my Kindle as much as I did when I first bought it and I was convinced that thing was going to change my life forever LOL. I just bought "real" books from that sale they had at the Culp yesterday. I love technology. I love having gadgets and cool stuff but I wish I could actually put the stuff to use more often.

  4. You make a good point. Google states that these glasses are meant to be used like a smartphone or a portable laptop – not all the time, but when needed. The immediate question that comes to my mind is, if I already have a smartphone for that, why do I need the Glasses? On the other hand, when it comes to augmented reality, the Glasses could definitely be beneficial; instead of holding up your smartphone to take a picture or bring up information about something you see, you could just look at something and have the information brought up right there on the Glasses. They might end up being the newest “essential” piece of tech or they might just be another neat gadget that doesn’t see much use. Here’s hoping for the former!

  5. This is a very interesting blog!! I am downloading some of the AR apps on my iphone as I post! lol The glasses and contact lenses are going to be an amazing technology innovation, although I can see how there are some ethical issues to be raised by these technology achievments. I can see how this new technology will benefit military training and will be essential to the military in the near future.

  6. As I was reading, I was very much hoping you would touch on the Google Glasses(And you did!). I had done some research myself on the topic after a friend mentioned the aforementioned glasses to me. As a response to the "when would you actually use them" comments: Heck, I'd wear my AR Glasses all the time! Who wouldn't want a a HUD as they're walking around? The comments that my friend and I had about this particular piece of tech, was that it may even be the beginning of permanent "augments" for people (Think Deus Ex if you have heard of that). Definitely an interesting field of study and hopefully one that will see much advancement in the future!

    1. "Permanent augments" is exactly what I think when I see things like this! One of the ideas surrounding the medical applications of this technology is to enhance the vision of people with impaired eyesight, but of course they could also enhance normal vision, letting people see better than ever before, in addition to the Google Glasses-style HUD.
      On top of that, prosthetic arms and legs are getting better all the time. I think we're well on our way to cyborgs!
      I wrote a short blog post about cyborgs last year: check it out:

  7. Great blog! it should make gaming even better, reminds me of the hologram of tupac at the music festval Coachella. Someday maybe everyone will be wearing a pair of the google glasses, I researched more about that glasses at googles website "project/glass." Really cool stuff.

  8. Wow, I never really thought about stuff like this before. I play video games all the time and I am familiar with virtual reality but not will AR. The Google Glasses concept is wild and its amazing the technology we have. Special effects and video-game technology as well as just picture quality in general is amazing now-a-days. Very informative blog

  9. Incredibly awesome stuff, indeed! Technology certainly has come a long way through the years. The advances in technology occur rapidly as well. That's why someday we will end up like the characters on Wall-E. I do think technology is awesome, but I think it makes us lazy as well. Society really loves to take the easy way out, so we look for ways to make it happen. The solution to making things easier? Technology, of course.

    Not to long ago we used to have to do research by going to the library and checking out books. Nowadays, we use Google (which really isn't research) or we use an online library. As a society, I think we are too heavily dependent on technology. I'm not against technology, but perhaps we should be a little less dependent on it.

    Also, have you heard about the laptop you can control with your eyes by Lenovo? It's called Tobii. There's also the Tobii PCEye which is a device that allows you to control any PC with your eyes. It's all very cool. But, like I said it's making us lazier. I just hope that someday we will not be fully dependent on technology. Considering how fast technology is advancing, I'm sure we're not too far away from that point. Unless, of course, the world ends before we get to that point.

    1. Interesting point, Monica. There certainly does seem to be a trend toward laziness in our technological advances.
      I look up the Tobii technology. That's pretty interesting stuff! Here's a run-through for people who are interested:
      Tobii specializes in eye-tracking technology. Their research allows a computer screen to keep track of where your eyes look most often - what parts of the screen are most interesting to you. This technology is useful for gathering data on how people interact with websites. Pretty cool. What Monica is talking about is the application of that technology that allows the computer to respond to your gaze. By moving your eyes over a part of the screen, the computer will detect your glance and automatically bring up information. That's really amazing! I hadn't heard of that!

      Although, I will say this. In addition to the worry about people getting lazy, using your eyes to operate a computer seems very cumbersome to me. It just doesn't sound nearly as convenient as good old point and click. I think I'll stick to my mouse for now.