A predictive video coder performs gradient prediction based on previous blocks of image data. For a new block of image data, the prediction determines a horizontal gradient and a vertical gradient from a block diagonally above the new block (vertically above a previous horizontally adjacent block). Based on these gradients, the encoder predicts image information based on image information of either the horizontally adjacent block or a block vertically adjacent to the new block. The encoder determines a residual that is transmitted in an output bitstream. The decoder performs the identical gradient prediction and predicts image information without need for overhead information. The decoder computes the actual information based on the predicted information and the residual from the bitstream.