Pixel-based method and apparatus of obtaining a downsampled image are provided. The method includes: constructing a virtual image by an image super-resolution technology according to a resolution of the downsampled image; determining relational expressions between pixels of the virtual image and pixels of the downsampled image; calculating a MSE between the pixels of the virtual image and pixels of the original image based on the relational expressions; determining a coefficient matrix between the pixels of the downsampled image and the pixels of the original image under the condition of the MSE being at a minimum value; determining the pixels of the downsampled image according to the pixels of the original image and the coefficient matrix; and outputting the downsampled image. Accordingly, the present invention can obtain sharp image and avoid color aliasing.