Hole Filling Through Photomontage

Marta Wilczkowiak1,2, Gabriel J. Brostow1, Ben Tordoff1,2, and Roberto Cipolla1
(see e-mail addresses at bottom)

University of Cambridge
2MathWorks Ltd., UK


To fill holes in photographs of structured, man made environments, we propose a technique which automatically adjusts and
clones large image patches that have similar structure. These source patches can come from elsewhere in the same image, or from other
images shot from different perspectives. Two significant developments of this work are the ability to automatically detect
and adjust source patches whose macrostructure is compatible with the hole region, and alternately, to interactively specify a
user's desired search regions. In contrast to existing photomontage algorithms which either synthesize microstructure or require careful
user interaction to fill holes, our approach handles macrostructure with an adjustable degree of automation.

Links:    Acrobat PDF version,     Bibtex for BMVC 2005 paper (Oral presentation)

Sample Images

User mask
matching region

Region filling process: (a) Target region extended using graph cuts; (b) The extended target region and its matching part in the image; (c) The final result.

The Giants Causeway, N. Ireland: before and after interactive editing.

Romsey traffic light: Original image and result after filling the user defined regions. Cloned window and brick texture
was compatible due to rectification. The user specified the right-of-center pillar as a hard constraint.

Transformer: Original in top left; Top right shows result of Smart Erase tool in M.S. Digital Image
Pro v.10. Bottom shows our result after filling the user defined region. Bricks replacing the
transformer are brighter than appropriate because no color correction is activated.

St. John's College, Cambridge: Original image; User interface, and result with user interaction.
The user specified the girl as the fill-region. The woman wearing brown and the edges of the sidewalk,
window, and lawn are tagged as hard constraints, as are the cobblestones.


Replace "DOT" and "AT" with '.' and '@' below:

Marta Wilczkowiak:       martaDOTwilczkowiakATgmailDOTcom
Gabriel J. Brostow:         gbrostowATacmDOTorg
Ben Tordoff:                   benDOTtordoffATukonlineDOTcoDOTuk
Roberto Cipolla:             cipollaATengDOTcamDOTacDOTuk

Machine Intelligence Laboratory, Engineering Dept, University of Cambridge