A computer-implemented method is provided for refreshing cells in a Non-Volatile Memory (NVM)-based neuromorphic circuit wherein synapses are each composed of a respective cell pair formed from a respective Gp cell and a respective Gm cell of the cells. The method includes randomly selecting multiple neurons and reading a conductance of any of the synapses connected to the multiple neurons. The method further includes selecting any of the synapses connected to the selected multiple neurons for which the Gm conductance has reached a maximum conductance. The method also includes resetting the Gp cell and Gm cell of the selected synapses, and setting, at most, one of the Gp cell and Gm cell of each of the selected synapses to recover an effective total weight of each of the selected synapses.