A nonlinearity calibration system and method for a frequency modulation (FM) transmitter. A nonlinearity calibration system for a FM transmitter includes a digitally controlled oscillator (DCO) with a variable capacitor array. The DCO receives a calibrated fine code for tuning the variable capacitor array to modulate a digitally encoded audio signal transmitted by the FM transmitter to a modulation frequency. The nonlinearity calibration system also includes a nonlinearity estimator for generating an approximation of an integral nonlinearity associated with processing of a fine code to tune the variable capacitor array. The nonlinearity calibration system further includes a subtractor for subtracting the approximation of the integral nonlinearity from the fine code to generate the calibrated fine code.