Data stored in non-volatile storage is decoded using iterative probabilistic decoding. An error correcting code such as a low density parity check code may be used. In one approach, initial reliability metrics, such as logarithmic likelihood ratios, are used in decoding sensed states of a set of non-volatile storage element. The decoding attempts to converge by adjusting the reliability metrics for bits in code words which represent the sensed state. Soft data bits are read from the memory if the decoding fails to converge. Initial reliability metric values are provided after receiving the hard read results and at each phase of the soft bit operation(s). In one embodiment, a second soft bit is read from the memory using multiple subsets of soft bit compare levels. While reading at the second subset of compare levels, decoding can be performed based on the first subset data.