Performance-based digital content delivery in a digital medium environment is described. Initially, different items of a collection of digital content are delivered to a substantially equal number of users. The collection is then iteratively tested to identify which content item achieves a desired action (e.g., conversion) at a highest rate. During the iterative test, data describing user interaction with the delivered content is collected. Based on the collected data, measures of achievement are determined for the different content items. Measures of statistical guarantee are also computed that indicate an estimated accuracy of the achievement measures. Responsive to determining that a condition for ending the test has not yet occurred, an optimized allocation is computed for delivery of the content by applying one of multiple allocation optimization techniques. The particular technique applied is based on the condition for ending the test and a type of statistical guarantee associated with the test.