A signal processing device for processing three-dimensional image data, having two-dimensional (2D) image data and 2D depth data, includes an input for receiving the 2D image data and the 2D depth data. Further, a first outlier detector is used for establishing a statistical depth deviation of a depth value from other depth values in a first spatial neighborhood of the 2D depth data. A second outlier detector is used for establishing a statistical image deviation of an image value from other image values in a second spatial neighborhood of the 2D image data. The signal processing device also includes a depth value generator for, in dependence on the statistical depth deviation and the statistical image deviation, providing a replacement depth value when the statistical depth deviation exceeds the statistical image deviation.