A method for compressing a bi-level document image containing text is disclosed. The document image is segmented into symbol images each representing a letter, numeral, etc. in the document. The symbol images are classified into a plurality of classes, each class being associated with a template image and a class index. Classification is done by comparing each symbol to be classified with template of existing classes, using a number of image features including zoning profiles, side profiles, topology statistics, and low-order image moments. These image features are compared using a tolerance based method to determine whether the symbol matches the template. After classification, certain classes that have few symbols classified into them may be merged with other classes. In addition, the template images of the classes are down-sampled, where the final sizes of the template images are dependent on the likelihood of confusion of the template with other templates.