The present disclosure is directed to a novel system for performing online reconfiguration of a neural network. Once a neural network has been implemented into a production environment, the system may use underlying construction logic to perform an in-situ reconfiguration of neural network elements while the neural network is live. The system may accomplish the reconfiguration by modifying the architecture of the neural network and/or performing adversarial training and/or retraining. In this way, the system may provide a way increase the performance of the neural network over time along one or more performance parameters or metrics.