A method and apparatus for scaling video images, which applies more particularly to techniques for converting video sequence formats and deinterlacing. The invention comprises a parameterized interpolation (101) which computes interpolations at each new point of the output. An insensitive interval estimation (102) computes an interval where the probability distribution of the output image value is estimated to have its maximum, and an insensitive loss calculation (103) derives for each interpolation value a negative log likelihood measurement from an interval-insensitive probability distribution. After a spatial regularization (104) the invention includes a minimum loss selection (105) which selects the minimum loss interpolation at each new point and provides it as an output image value. The invention also includes a total variation increment module and a directional matching error module that can penalize the insensitive loss, for the minimum loss selection.