An optical pointing device includes: an image obtaining part for continuously obtaining an image on a work surface as a sample image; a movement value calculator for comparing the sample image with the previously obtained image to calculate and output a movement value; a button selection signal generator for outputting a button selection signal corresponding to a specific reference pattern, when the sample image is the same as the specific reference pattern of a plurality of reference patterns; and a controller for providing the button selection signal and the movement value to a computing device when the button selection signal is transmitted from the button selection signal generator, and providing the movement value only to the computing device when the button selection signal is not transmitted from the button selection signal generator. Therefore, it is possible to recognize a plurality of specific patterns to generate button selection signals and replace mechanical selection buttons, thereby minimizing manufacturing cost and size of the optical pointing device. In addition, it is possible to readily change the number of button selection signals provided from the optical pointing device by changing the number of specific patterns.