Certain aspects of the present disclosure provide a method for compressing an activation function, comprising: determining a plurality of difference values based on a difference between a target activation function and a reference activation function over a range of input values; determining a difference function based on the plurality of difference values; and performing an activation on input data using the reference activation function and a difference value based on the difference function.