A method performed at a transmitter comprises generating a set of digital signals representing a constellation point, each dimension of the point being uniquely represented by m bits, where mϵ{1, 2, 3}; applying a linear polyphase filter to a digital signal of the set, the linear polyphase filter pre-compensating for a linear transfer function of an electro-optic path of the digital signal and generating a first filtered signal having a roll-off factor α, where α is a positive real number satisfying α≤1; applying a nonlinear polyphase filter to the digital signal, in parallel to applying the linear polyphase filter to the digital signal, the nonlinear polyphase filter generating a second filtered signal representing nonlinear noise in the electro-optic path; calculating a pre-compensated digital signal from a difference between the first and second filtered signals; and transmitting an optical signal based on the pre-compensated digital signal.