Video encoding or decoding utilizing a spatial transform operating on rows and columns of a block, with a set of transform skip modes including: transform on rows and columns; transform on rows only; transform on columns only; no transform. An indication of the selected mode is provided to the decoder. Coefficients are scaled by a factor dependent upon the norm of the transform vector of the skipped transform to bring the untransformed image values to the same level as transformed coefficients.