Mobile Augmented Reality

We had the good fortune to get an interview with the developer behind Pamgoo Software, creator of Aure – Custom Augmented Reality. Aure is a service that enables its users a custom augmented reality view. Every “blob” on the screen is known as an Aure and each one corresponds to a location. Users can add a new Aure from current location or dropping a pin in the map. You can also write the title, description, add a photo , choose the color or even share your Aure with the world adding the country and the city of your new Aure. In My Aures section you can see your private Aures, in Shared Aures section you can see which Aures did you shared with the world, and in Imported Aures section, you can see which one did you imported from the server. The app performs really well and is definitely worth grabbing from the App Store.

App Store Link: Aure – Custom Augmented Reality $1.99

Now for our interview with Daniel Galera from Pamgoo Software.

Sun Seeker: 3D Augmented Reality Viewer is hot application currently available in the App Store and we were lucky enough to snag an interview with the developer. First let’s discuss our hands-on with the application.

The app’s description says that it ‘Provides both a flat view compass and an augmented reality camera 3-D VIEW showing the solar path, its hour intervals, its winter and summer solstice paths, rise and set times and more.’

When you first launch the app you are presented with a top-down compass view showing the sun’s path throughout the hours of the day. In a glance you can see sunrise and sunset, length of day and night, and where in the sky the sun will be.  You can also choose to view the sun’s information for any date you select.

Pressing the camera button brings up the augmented reality mode. This starts the phone’s camera and nicely overlays the sun’s path over a ‘grid-like’ pattern. In addition to the sun’s path you can also see the elevation, azimuth, sun’s position throughout the day, and the direction of the sun’s current position.

We really like this application and felt it offered all of the solar information you would need in a beautiful package.

App Store Link: Sun Seeker: 3D Augmented Reality Viewer $2.99

Now for our interview with the developer of Sun Seeker, Graham Dawson.

1) How did you come to your decision of developing an ‘augmented reality’ application?

Yes. I am working on another idea which I don’t wish to disclose yet, and it will require some trialling to determine whether or not it works well enough in practical situations. Its certainly an exciting area to work in, although a little risky in terms of time and effort spent to prototype your ideas, in an area which is still quite novel and “bleeding edge”.

We thank Graham for his insightful answers and look forward to his future projects.

We recently were able to talk with the head of Aero Visions International Inc, the development company behind the augmented reality app iHUD. Before we get to the interview, let’s discuss their application.

iHUD is an app that turns your iPhone 3G or 3GS into an  aerospace-inspired mobile cockpit. You are presented with a simulated moving horizon based on the vehicle’s acceleration, dynamic moving speed, altitude and vertical velocity ribbons, rotating compass card, slip/skid ball, and separate accelerometer (G-meter). All of this information is elegantly laid on top of the device’s camera view. This data provides optimal situational awareness and is  made possible by the iPhone’s on-board accelerometer and GPS sensor. One of the really cool features of this app is its ability to project the display onto your car’s windshield. This app is currently priced at $5.99 and is definitely worth picking up.

App Store Link:  iHUD $5.99

Now for our interview with Jurg Sommerauer from Aero Visions International Inc.

Since a long time we pondered the idea to add an augmented reality feature (AR) to our aviation inspired head-up display (HUD) showing spatial motion information using the Multi Function Display (MFD) of the existing iHUD while seeing through the camera of the iPhone. Not unless we had a better handle on the API in OS 3.0 we implemented AR.

The provided hardware in iPhone 3GS provides the basic hardware to implement AR quite well, so does the API now, except for video.

There is a lot of possible scenarios wit AR. We have seen already superb apps based on AR, and more will come integrating data base of the real world we are living in making it possible to identify and act upon on-the-fly.

We intend to refine our iHUD spatial motion information for 3D navigation combined with AR. But for this we will need an onboard gyro (which may hopefully be provided in the next hardware upgrade).

Here two screen shots showing iHUD in the normal MFD mode and the AR mode.

Our thanks to Jurg for taking the time to answer our questions.

Over the New Year we were lucky enough to speak with Phil Endecott about his augmented reality application in the App Store.

Panoramascope: Augmented Reality is an app that allows users to compute the skyline as seen from anywhere on the planet. The application draws terrain data from NASA’s worldwide database and is elegantly superimposed over the phone’s camera view. The app uses the phone’s internal compass so that as you turn, correct data is always shown. You can see labels identifying mountain peaks and other features. No mountains in your area? Don’t worry, in flatter areas marker layers are provided for features including tourist attractions and pubs. Another interesting feature of Panoramascope Augmented Reality is it’s ability to interact with a separate application called Topo Maps. Owners of this app will be able to see the view from a point selected on the map.

Overall we felt that the app was an excellent implementation of augmented reality on the iPhone and is definitely worth its current price at $5.99.

App Store Link: Panoramascope: Augmented Reality $5.99

Now for our interview with developer Phil Endecott.

1) How did you come to your decision of developing an ‘augmented reality’ application?

The history goes back a long way.  About 10 years ago I worked for AT&T Labs Research, and some of my colleagues there built an “indoor GPS” location system using ultrasound.  The idea was to track people and objects around buildings, and those guys now have their own company and the hardware is being used in places like huge automotive plants to track vehicles on the production line.  Anyway, a guy called Joe Newman (who’s still working on AR at the University of Cambridge) built an AR headset by hacking sensors onto a cycle helmet, powered by a computer in a backpack.  There’s a paper about this, with some photos, at http://www.cl.cam.ac.uk/Research/DTG/publications/public/files/tr.2001.2.pdf – it was pretty advanced stuff for 2001.  So although I wasn’t directly involved in this, I learnt quite a lot about what was and wasn’t possible and what mathematical techniques were needed.

The next step was in July 2007 when I went to Calgary for a friend’s wedding.  Afterwards I went off to the Rockies for a couple of weeks and did some great hiking, including the famous Skyline Trail just South of Jasper.  There are some great views from there with lots of jagged mountains in the distance, and no-one is satisfied just to look at them anonymously – we all want to know what they’re called!  ”If only there were some gadget…”, I thought.  My initial idea was to add the functionality to a camera: you would take a photo of a view, and the camera would automatically add a layer of mountain names and other information, so that when you looked at the photos in years to come you would know what you were looking at.  This could work for urban tourism as well as mountain views.

So when I got home I made a prototype; it consisted of an Eee 901 laptop, running Linux, with a GPS, webcam and compass module connected by USB.  I tried this out on a few mountain hikes and it worked reasonably well, but getting it into a camera was going to be more of a challenge.  I went to Photokina (a huge photographic trade show) in late 2007 and tried to interest camera people in my idea, but they were all very conservative.  Even now, there are very few cameras that have even a GPS.  So maybe in another 5 years this will be possible, but I needed something that would pay the rent sooner than that!

So then the idea of making an iPhone app came along.  Of course at that time the iPhone didn’t have a compass, so proper AR was not possible, but by that time I had implemented the skyline-drawing algorithm, using the NASA altitude data, that is the most novel aspect of the Panoramascope.  By plotting the mountain names against this computed skyline, the user could work out what they were looking at without needing “live” video and compass tracking.

I launched this original version of the Panoramascope app in January 2009, and although the sales were not spectacular the people who tried it seemed to like it.  Not long after that the first Android phone came out with a compass, and I guessed that Apple would add this to the next iPhone.  Once they had done that, and added the video superposition facility in an iPhone OS update, AR become possible and I launched the current app, “Panoramascope: Augmented Reality” in November 2009.

2) What is your opinion of the capability of the iPhone’s hardware in terms of handling augmented reality applications?

It’s good enough.  The compass accuracy doesn’t seem very good, but I haven’t really investigated this properly yet.  We would all like a better camera, and not only for AR.

My main complaint – this week – about the iPhone hardware is that it is not waterproof, as I learnt from experience while out on the hills trying to get some screenshots for the website…

3) Are you optimistic about the future of augmented reality on mobile devices and can you predict any possible future applications for this technology?

I’m not a natural optimist

Actually I fear that in the same way that email has been ruined by spam and much of the web has been ruined by adverts, someone will find a way to subvert AR.  Free apps that splatter MacDonalds logos over everything will no doubt outsell mine 1000X….

I think the academics who have been thinking about AR for a few decades now have considered most of the useful applications.  I imagine engineers in chemical plants wearing AR goggles so that when they look at a pipe it tells them what is flowing in it, how fast, what temperature etc.  I imagine visitors to large buildings being given a pair of AR goggles at reception to guide them to the person they’ve come to visit.  I imagine museums replacing those audio guides where you have to type in a number from an exhibit with something that knows automatically what you’re looking at.  Games, obviously.  Many of these need things that the iPhone doesn’t have, e.g. accurate indoor positioning and goggles.  I think we’re currently seeing AR on phones because it is the first platform that can deliver it to the mass market, but it’s not the ideal platform in the longer term; for that, custom AR hardware will take over.

4) Do you plan to develop another augmented reality application?

I don’t currently have any plans, partly because I’m selling many more of my other app, Topo Maps, and it’s more productive to spend time working on that.

In the longer term I would very much like to go back to my original idea of augmenting photographs.  If Apple produces an iPhone with a better camera, I’d consider that.

Our thanks to Phil for taking time to answer our questions and we wish him continued success in the App Store.