I wrote this as a companion piece for an article that may show up in the next issue of iphoneLife, but I thought I would include it here and now since AR is getting more relevant.\u00a0 Also, one of the things I said NEEDS to part of the AR toolkit for iphone (mobile) got a write up today in RRW.<\/p>\n
This is still in draft form, so I’ll edit it again at some point..<\/p>\n
According to WikiPedia – Augmented reality (AR) is a term for a live
\ndirect or indirect view of a physical real-world environment whose
\nelements are merged with-, or augmented by virtual computer-generated
\nimagery – creating a mixed reality. \u00a0The funny thing is, we have had a
\nsimple example of AR on the iphone already. Consider this application
\niHUD. \u00a0In a way, this could be considered an AR application. \u00a0We have
\nvirtual information overlaid on top of reality. \u00a0As our position
\nchanges in the real world it is reflected in the virtual information
\noverlaid. However, for this article, lets refine the definition of AR
\njust a bit more to require that virtual information must contain a
\nspatial component that is dependent upon the view in question. \u00a0In
\nother words, as the reality changes based on what we see on the iphone
\nscreen, the information should change accordingly. Now, this makes AR
\na much harder challenge, but one that opens up even more interesting
\nopportunities.<\/p>\n
A discussion of the enhancements to OS 3.0 and the 3GS that make it possible.<\/p>\n
In order to employ AR, a mobile device must support four key functions.
\n1) A high enough quality video camera that can detect necessary
\ndetails in the scene as well as maintain sufficient refresh to track
\nand measure movement in the scene.
\n2) A compass sensor with 1 deg of resolution accuracy or better.
\n3) Sufficiently high accuracy GPS.
\n4) Access to geo resolved data accurate enough to align with what the
\nusers sees on the display.
\nThe iPhone is the not the first commercial mobile device to display AR
\ncapabilities, that would go the Android phone and the Nokia N97,
\nhowever, both lack the iphone3Gs video capabilities, so even though we
\nhave seen some great apps on these platforms, I’m hopeful for even
\nmore ambitious projects for the iphone. The iphone3G is not as well
\nsuited to AR tasks because of a lack of compass sensor and high enough
\nresolution camera.
\nIt should be noted that the current crop of AR apps for the iphone are
\nnot fully legal. \u00a0The SDK does NOT include the ability to access the
\nvideo feed, so the few AR apps that have snuck through Apple’s review
\nprocess – I’m looking at you Yelp – have jumped the gun a bit. \u00a0But
\nwho can blame them? Its simply too tempting. \u00a0However, by the time you
\nread this article, iphone OS 3.1 should be out which will provide the
\nproper access via the SDK.
\nFor those who have started to dev products, and those of us who has
\ntested them, have been hit square in the jaw with an awful truth. If
\nany of the 4 elements I listed above degrades, so does the experience.
\nSuch that, if the alignment of the virtual content to reality is off
\neven a small degree, that the AR experience could be worthless, or
\nworse, provide false information. \u00a0This is a concern since I have
\nexperienced my fair share of connection problems and GPS errors as I
\ntravel around San Francisco. As for Yelp, the latitude longitude
\ndatabase could use some work.
\nThere is a solution however… and its not an easy one. \u00a0Analyze the
\nvideo that you see through the iphone’s camera for clues as to where
\nyou are. This may
\nseem like an impossible task, but perhaps not as hard as one might
\nthink. For example; If we know a Starbucks is at a certain street
\ncorner (almost every street corner in some cities) we can verify our
\nposition if the
\nusers puts the Starbucks store sign in the
\nvideo camera’s view. OCR (Object Character Recognition) software would
\n‘read’ the sign and confirm or
\ncorrect the known position. \u00a0Using video recognition as
\na geo spatial input could be the 5th element we need to have robust AR
\napplications on mobile devices with less then perfect GPS tracking.<\/p>\n
If we dismiss the limitations of the iPhone in certain locations, and
\nalso assume that some smart programmers begin to employ some forms of
\nvideo geo spatial recognition, we could see a bevy of iphone
\napplication beyond anything we has seen so far.
\nSure, we will have a plethora of direction applications, subway routes
\n(as we have seen), landmark identification, and geo caching will be taken to
\nthe next level. \u00a0But lets imagine a bit further.
\nPerhaps you can play cards with your friend on a table without any
\ncards. Prices and additional information for goods in the supermarket.
\nNot only can we employ a iHUD like trick to get an overlay on the
\nwindshield, but we could calibrate the display to coordinate to the
\nroad such that we would have directional arrows appearing on the road.
\nPerhaps the next time you and your friend are separated in a crowd,
\nyou get a real life position tracker to take \u00a0you right to them! \u00a0Not
\nsure who is who in a room? \u00a0With facial recognition or badge ID
\npatters, you will get a personal ID popup. Seen the latest trick where
\nyou put a pattern card down and then a 3D image pops up?
\n(http:\/\/www.hitl.washington.edu\/artoolkit\/<\/a>) There’s already an app for
\nthat. Walk by a building site and see the proposed new structure as if
\nit was already built. How about a virtual dog that can detect the
\nground and objects, and walks along with you as you walk? As you can
\nsee, only our imaginations will be the limit. And I know that before
\nthe year is over, there will be a number of iPhone apps we just could
\nnot have imagined.<\/p>\n