Natural Feature Tracking for Mobile Phones

Mobile phones are very inexpensive, attractive platforms for Augmented Reality, because they are small, ubiquitous, and generally accepted by customers. The trend to more full featured devices including large screens, cameras and new interface methods, allows for more complex interaction and algorithms to be implemented on phones.

Nevertheless, phones still have severe limitations in both the computational facilities (low throughput, no floating point support) and memory bandwidth (limited storage, slow memory, tiny caches). Therefore, computer vision implementations to support AR are challenging to realize. Most current implementations focus on marker-based approaches, due to their efficiency in searching for markers and matching them.

To avoid the use of artifical visual markers, we have investigated the performance of natural feature tracking algorithms on mobile phones. A good example is the Fern feature classification approach developed at EPFL. We have modified and scaled down its data requirements to make it operate on a Nokia N95 device.


General tracking	Close-up	Rotations and angled views

Together with colleagues from TU Graz we compared this approach with an alternative approach based on feature descriptors on a variety of devices. See the paper from ISMAR 2008 for more details.

Furthermore the localisation algorithm is also used in a prototype of a mobile map augmentation system, developed and studied at TKK, Finland. See the paper from HCI2008 for more details.

Publications

Daniel Wagner, Gerhard Reitmayr, Alessandro Mulloni, Tom Drummond and Dieter Schmalstieg
Pose Tracking from Natural Features on Mobile Phones
In Proc. IEEE ISMAR'08, 2008, Cambridge, UK. [BIBTEX]

Best Paper Award at ISMAR'08
Ann Morrison, Giulio Jacucci, Peter Peltonen, Antti Juustila and Gerhard Reitmayr
Using locative games to evaluate hybrid technology
In Proc. HCI2008, 2008, Liverpool, UK, [BIBTEX]
Ann Morrison, Antii Oulasvirta, Peter Peltonen, Saija Lemmela, Giulio Jacucci, Gerhard Reitmayr, Jaana Nasanen and Antti Juustila
Like bees around the hive: a comparative study of a mobile augmented reality map
In Proc. ACM CHI'09, 2009, New York, USA.
[BIBTEX]

Nominated for Best Paper award

Media

Natural feature tracking on phones using Ferns and SIFT (hosted at TU Graz).
mobiletracking.mp4 example video tracking a colour photograph.

Contact

Dr. Tom Drummond (twd20)
Dr. Gerhard Reitmayr (gr281)
Department of Engineering, University of Cambridge

Acknowledgements

Many thanks to Vincent Lepetit for discussions on Ferns and sample code. This work is supported by the EC project IPCity.