Embodiments of the present invention provide a method and an apparatus for predicting a residual, which method is applied to three-dimensional video encoding or multi-visual angle video encoding and comprises searching, when performing inter frame image prediction encoding on a prediction unit, a corresponding unit of the prediction unit in an adjacent visual angle encoded at the same time; and predicting a time domain predicted residual of the prediction unit by utilizing a time domain predicted residual of the corresponding unit in the encoded adjacent visual angle. The method is applied to three-dimensional video decoding or multi-visual angle video decoding and comprises judging, when performing inter frame image prediction decoding on a prediction unit, whether the prediction unit adopts a residual prediction; searching, if it is determined the prediction unit adopts the residual prediction, a corresponding unit of the prediction unit in an adjacent visual angle decoded at the same time; and predicting a predicted residual of the prediction unit by utilizing a time domain predicted residual of the corresponding unit in the decoded adjacent visual angle. The present invention improves video compression ratio and enhances encoding/decoding efficiencies.