Weight normalization in hardware or software without a division operator is described, using only right bit shift, addition and subtraction operations. A right bit shift is performed on an expected sum to effectively divide the expected sum by two to provide a first updated value for the expected sum. An iteration is performed which includes: incrementing with a first adder a first variable by the first updated value of the expected sum to provide an updated value for the first variable; subtracting with a first subtractor a second weight from a first weight to provide a first updated value for the first weight; and performing a left bit shift on the second weight to effectively multiply the second weight by two to provide a first updated value for the second weight.