A pill identification system identifies a pill type for a pharmaceutical composition from images of the pharmaceutical composition. The system extracts features from images taken of the pill. The features extracted from the pill image include color, size, shape, and surface features of the pill. In particular, the features include rotation-independent surface features of the pill that enable the pill to be identified from a variety of orientations when the images are taken. The feature vectors are applied to a classifier that determines a pill identification for each image. The pill identification for each image is scored to determine identification for the pharmaceutical composition.