A memory apparatus and method of operation is provided. The apparatus has blocks each including non-volatile storage elements. Each of the non-volatile storage elements stores a threshold voltage representative of an element data. The apparatus also includes one or more managing circuits configured to erase at least one of the blocks in an erase operation and program the element data in a program operation. The one or more managing circuits are also configured to proactively identify ones of the blocks as potential bad blocks and selectively apply stress to the ones of the blocks identified as the potential bad blocks and determine whether the potential bad blocks should be retired from the erase and program operations and put in a grown bad block pool or released to a normal block pool used for the erase and program operations based on a judgment after selectively applying the stress.