A method of selecting reference blocks for intra or inter prediction coding of a current block of a video signal. Each reference blocks is selected by: (i) determining a group of candidate blocks comprising at least some candidate blocks other than immediately adjacent spatial neighbors of the current block; and (ii) selecting one of the candidate blocks as the reference block based on a second metric, different from the first metric, the second metric relating to a number of bits that would be required in the encoded bitstream to encode both the residual block and the side information identifying the respective reference block.