A method for protecting a machine learning model is provided. In the method, a first machine learning model is trained, and a plurality of machine learning models derived from the first machine learning model is trained. Each of the plurality of machine learning models may be different from the first machine learning model. During inference operation, a first input sample is provided to the first machine learning model and to each of the plurality of machine learning models. The first machine learning model generates a first output and the plurality of machine learning models generates a plurality of second outputs. The plurality of second outputs are aggregated to determine a final output. The final output and the first output are classified to determine if the first input sample is an adversarial input. If it is adversarial input, a randomly generated output is provided instead of the first output.