One or more techniques and/or systems are disclosed for compensating for affine distortions in handwriting recognition. Orientation estimation is performed on a handwriting sample to generate a set of likely characters for the sample. An estimated affine transform is determined for the sample by applying hidden Markov model (HMM) based minimax testing to the sample using the set of likely characters. The estimated affine transform is applied to the sample to compensate for the affine distortions of the sample, yielding an affine distortion compensated sample.