The present application discloses systems and methods for using probabilistic prioritization to assign human reviewers to review data stored in or indexed by an information system. Some embodiments include accessing an index of data items, where individual data items have a corresponding probability f of having a problem, a cost to review the data item, a penalty if a problem associated with the data item is not remedied, and a gain if a problem associated with the data item is remedied; identifying a subset of data items having a corresponding f that is greater than or equal to a decision threshold based on the data item's corresponding cost, penalty, and gain; and ranking at least a portion of the subset of data items based at least in part on their corresponding cost, f, and gain.