Systems and methods are provided for receiving input data to be processed by an encrypted neural network (NN) model, and encrypting the input data using a fully homomorphic encryption (FHE) public key associated with the encrypted NN model to generate encrypted input data. The systems and methods further provided for processing the encrypted input data to generate an encrypted inference output, using the encrypted NN model by, for each layer of a plurality of layers of the encrypted NN model, computing an encrypted weighted sum using encrypted parameters and a previous encrypted layer, the encrypted parameters comprising at least an encrypted weight and an encrypted bias, approximating an activation function for the level into a polynomial, and computing the approximated activation function on the encrypted weighted sum to generate an encrypted layer. The generated encrypted inference output is sent to a server system for decryption.