[Univ of Cambridge] [Dept of Engineering]
Thomas Woodley

AIDIA: Adaptive Interface for Display InterAction


This paper presents a vision-based system for interaction with a display via hand pointing. An attention mechanism based on face and hand detection allows users in the camera's field of view to take control of the interface. Face recognition is used for identification and customisation. The system allows the user to control the screen pointer by tracking their fist. On-screen items can be selected using one of four activation mechanisms. Current sample applications include browsing image and video collections as well as viewing a gallery of 3D objects. In experiments we demonstrate the performance of the vision components in challenging conditions and compare it to that of other systems.


Thomas Woodley
BjŲrn Stenger
Tae-Kyun Kim
Carlos HernŠndez
Roberto Cipolla


Paper (pdf)

Demo Video (50MB)


gesture                3dmodel

   System in Use                                  3d Model Manipulation

Tracking Methodology

We track using Normalized Cross Correlation (NCC) and colour/motion (CM) cues. NCC is used if its confidence remains high enough, otherwise the less precise but more robust CM becomes active. An offline boosted detector is fired locally at regular intervals, and any positive result used to move back up to NCC, and update the NCC template and CM colour model.

Tracking Cascade: Tracking is initialised through global
detection. It proceeds using NCC template matching when
possible, falling back to CM tracking when not. Local detection
is used to move back up the cascade.

tracker switching
Tracker Switching: Position error shown over a test run, with
colour indicating the component in use: blue=NCC,
green=detector, red=CM.

Subsequent Work

We have developed a similar system for the Toshiba stand at the IFA consumer electronics trade fair in Berlin, 2008. This has received some press attention:

Youtube Demo Video