A hybrid delta modulator that can be used as a variable threshold neuron in a neural network is described. The hybrid delta modulator exhibits a memory of the prior state of the modulator, similar to a delta modulator, and receives a sum-of-products signal from a weighting circuit and generates a quantized output stream that represents the sum-of-products signal, potentially including an activation function and offset. With appropriately selected components, the hybrid delta modulator separates the integral function of the feedback from the gain function. Further, the gain can be selected, and the characteristic of the output pattern can be tailored to include an arbitrary combination of the input and the rate of change of the input. The use of a hybrid delta modulator of the present approach provides a simpler solution and better performance than many prior art neurons.