Provided are an apparatus and method of obtaining a high-quality image by efficiently processing an image obtained at low-light levels. The image obtaining apparatus comprises a sensor which detects a pixel array value by using a color filter including color pixels for obtaining a color image and a reference pixel for obtaining a reference image used for hand-trembling function estimation; an exposure controller which controls exposure times of the color pixels and the reference pixel; and an image generator which generates a long-exposure color image signal and a short-exposure reference image signal from the detected pixel array value, the long-exposure color image signal and the short-exposure reference image signal being aligned with each other. Accordingly, accurate estimation of the hand-trembling function is possible, thereby enabling the high-quality color image restoration.