SIGGRAPH Asia' 2014

Slippage-free Background Replacement for Hand-held Video

Fan Zhong1,Song Yang1, Xueying Qin1,Dani Lischinski2,Daniel Cohen-Or3,Baoquan Chen1
1Shandong University, 2The Hebrew University of Jerusalem, 3Tel Aviv University

We introduce a method for replacing the background in a video of a moving foreground subject, when both the source video capturing the subject, and the target video capturing the new background scene, are natural videos, casually captured using a freely moving hand-held camera. We assume that the foreground subject has already been extracted, and focus on the challenging task of generating a video with a new background, such that the new background motion appears compatible with the original one. Failure to match the motion results in disturbing slippage or moonwalk artifacts, where the subject's feet appear to slide or slip over the ground. While matching the motion across the entire frame is impossible for scenes with differing geometry, we aim to match the local motion of the ground in the vicinity of the subject. This is achieved by reordering and warping the available target background frames in a manner that optimizes a suitably designed objective function.

  • Code (.7z 155MB, include source code, executable and one test example)

@article {zhong2014,

author = {Zhong, Fan and Yang, Song and Qin, Xueying and Lischinski, Dani and Cohen-Or, Daniel and Chen, Baoquan},

title = {Slippage-free Background Replacement for Hand-held Video},

journal = {ACM Trans. Graph. (SIGGRAPH Asia 2014)},

year = {2014},

volume = {33},

number = {6},

issn = {0730-0301},

pages = {...},

articleno = {...},

numpages = {11}