A neural network unit is disclosed. This neural network unit includes: a main synapse array having a plurality of main synapses in rows and columns of the array, each connected to one or more bit lines associated with the network unit; a main synapse driver for applying an input to one or more of the main synapses in a row; a reference synapse for generating a particular output to a selected one or more associated bit lines; and, a sensing circuit for controlling the reference synapse in generating the output in a selected column of the array, wherein the sensing circuit is adapted to infer an output from one or more main synapses in a selected column by: defining a reference range of the output to be generated by reference synapse; setting a reference level from the reference range; activating a selected reference synapse to implement the reference level that is set or adjusted; and comparing an output from the main synapse summed with the output from the activated reference synapse in the selected column with a threshold value predetermined.