Image inpainting is controlled in response to a structure analysis and recovery process for a missing object region. Structure edges are detected about the periphery of the removed object area, and extensions determined in response to color and depth information. Intersections are determined in a structure framework, whose regions are then synthesized using a patch filling process, after which remaining areas of the image similarly synthesized.