[Univ of Cambridge] [Dept of Engineering]
Thomas Woodley

AIDIA: Adaptive Interface for Display InterAction

Abstract

This paper presents a vision-based system for interaction with a display via hand pointing. An attention mechanism based on face and hand detection allows users in the camera's field of view to take control of the interface. Face recognition is used for identification and customisation. The system allows the user to control the screen pointer by tracking their fist. On-screen items can be selected using one of four activation mechanisms. Current sample applications include browsing image and video collections as well as viewing a gallery of 3D objects. In experiments we demonstrate the performance of the vision components in challenging conditions and compare it to that of other systems.


People

Thomas Woodley
BjŲrn Stenger
Tae-Kyun Kim
Carlos HernŠndez
Roberto Cipolla


Resources

Paper (pdf)

Demo Video (50MB)


Screenshots

gesture                3dmodel

       
   System in Use                                  3d Model Manipulation


Tracking Methodology

We track using Normalized Cross Correlation (NCC) and colour/motion (CM) cues. NCC is used if its confidence remains high enough, otherwise the less precise but more robust CM becomes active. An offline boosted detector is fired locally at regular intervals, and any positive result used to move back up to NCC, and update the NCC template and CM colour model.

tracking_flowchart
Tracking Cascade: Tracking is initialised through global
detection. It proceeds using NCC template matching when
possible, falling back to CM tracking when not. Local detection
is used to move back up the cascade.


tracker switching
Tracker Switching: Position error shown over a test run, with
colour indicating the component in use: blue=NCC,
green=detector, red=CM.

Subsequent Work

We have developed a similar system for the Toshiba stand at the IFA consumer electronics trade fair in Berlin, 2008. This has received some press attention:

Press1
Press2
Press3
Youtube Demo Video