Mobile Augmented Reality

Over the New Year we were lucky enough to speak with Phil Endecott about his augmented reality application in the App Store.

Panoramascope: Augmented Reality is an app that allows users to compute the skyline as seen from anywhere on the planet. The application draws terrain data from NASA’s worldwide database and is elegantly superimposed over the phone’s camera view. The app uses the phone’s internal compass so that as you turn, correct data is always shown. You can see labels identifying mountain peaks and other features. No mountains in your area? Don’t worry, in flatter areas marker layers are provided for features including tourist attractions and pubs. Another interesting feature of Panoramascope Augmented Reality is it’s ability to interact with a separate application called Topo Maps. Owners of this app will be able to see the view from a point selected on the map.

Overall we felt that the app was an excellent implementation of augmented reality on the iPhone and is definitely worth its current price at $5.99.

App Store Link: Panoramascope: Augmented Reality $5.99

Now for our interview with developer Phil Endecott.

1) How did you come to your decision of developing an ‘augmented reality’ application?

The history goes back a long way.  About 10 years ago I worked for AT&T Labs Research, and some of my colleagues there built an “indoor GPS” location system using ultrasound.  The idea was to track people and objects around buildings, and those guys now have their own company and the hardware is being used in places like huge automotive plants to track vehicles on the production line.  Anyway, a guy called Joe Newman (who’s still working on AR at the University of Cambridge) built an AR headset by hacking sensors onto a cycle helmet, powered by a computer in a backpack.  There’s a paper about this, with some photos, at http://www.cl.cam.ac.uk/Research/DTG/publications/public/files/tr.2001.2.pdf – it was pretty advanced stuff for 2001.  So although I wasn’t directly involved in this, I learnt quite a lot about what was and wasn’t possible and what mathematical techniques were needed.

The next step was in July 2007 when I went to Calgary for a friend’s wedding.  Afterwards I went off to the Rockies for a couple of weeks and did some great hiking, including the famous Skyline Trail just South of Jasper.  There are some great views from there with lots of jagged mountains in the distance, and no-one is satisfied just to look at them anonymously – we all want to know what they’re called!  ”If only there were some gadget…”, I thought.  My initial idea was to add the functionality to a camera: you would take a photo of a view, and the camera would automatically add a layer of mountain names and other information, so that when you looked at the photos in years to come you would know what you were looking at.  This could work for urban tourism as well as mountain views.

So when I got home I made a prototype; it consisted of an Eee 901 laptop, running Linux, with a GPS, webcam and compass module connected by USB.  I tried this out on a few mountain hikes and it worked reasonably well, but getting it into a camera was going to be more of a challenge.  I went to Photokina (a huge photographic trade show) in late 2007 and tried to interest camera people in my idea, but they were all very conservative.  Even now, there are very few cameras that have even a GPS.  So maybe in another 5 years this will be possible, but I needed something that would pay the rent sooner than that!

So then the idea of making an iPhone app came along.  Of course at that time the iPhone didn’t have a compass, so proper AR was not possible, but by that time I had implemented the skyline-drawing algorithm, using the NASA altitude data, that is the most novel aspect of the Panoramascope.  By plotting the mountain names against this computed skyline, the user could work out what they were looking at without needing “live” video and compass tracking.

I launched this original version of the Panoramascope app in January 2009, and although the sales were not spectacular the people who tried it seemed to like it.  Not long after that the first Android phone came out with a compass, and I guessed that Apple would add this to the next iPhone.  Once they had done that, and added the video superposition facility in an iPhone OS update, AR become possible and I launched the current app, “Panoramascope: Augmented Reality” in November 2009.

2) What is your opinion of the capability of the iPhone’s hardware in terms of handling augmented reality applications?

It’s good enough.  The compass accuracy doesn’t seem very good, but I haven’t really investigated this properly yet.  We would all like a better camera, and not only for AR.

My main complaint – this week – about the iPhone hardware is that it is not waterproof, as I learnt from experience while out on the hills trying to get some screenshots for the website…

3) Are you optimistic about the future of augmented reality on mobile devices and can you predict any possible future applications for this technology?

I’m not a natural optimist

Actually I fear that in the same way that email has been ruined by spam and much of the web has been ruined by adverts, someone will find a way to subvert AR.  Free apps that splatter MacDonalds logos over everything will no doubt outsell mine 1000X….

I think the academics who have been thinking about AR for a few decades now have considered most of the useful applications.  I imagine engineers in chemical plants wearing AR goggles so that when they look at a pipe it tells them what is flowing in it, how fast, what temperature etc.  I imagine visitors to large buildings being given a pair of AR goggles at reception to guide them to the person they’ve come to visit.  I imagine museums replacing those audio guides where you have to type in a number from an exhibit with something that knows automatically what you’re looking at.  Games, obviously.  Many of these need things that the iPhone doesn’t have, e.g. accurate indoor positioning and goggles.  I think we’re currently seeing AR on phones because it is the first platform that can deliver it to the mass market, but it’s not the ideal platform in the longer term; for that, custom AR hardware will take over.

4) Do you plan to develop another augmented reality application?

I don’t currently have any plans, partly because I’m selling many more of my other app, Topo Maps, and it’s more productive to spend time working on that.

In the longer term I would very much like to go back to my original idea of augmenting photographs.  If Apple produces an iPhone with a better camera, I’d consider that.

Our thanks to Phil for taking time to answer our questions and we wish him continued success in the App Store.