Sequential hypothesis testing techniques are described, which involve testing sequences of increasingly larger number of samples until a winner is determined. In particular, sequential hypothesis testing techniques is based on whether a result of a statistic has reached statistical significance that defines a confidence level in the accuracy of the results. Sequential hypothesis testing also permits the user to “peek” into the test through use of a user interface (e.g., dashboard) to monitor the test in real time as it is being run. Real time output of this information in a user interface as a part of sequential hypothesis testing may be leveraged in a variety of ways. In a first example, a user may make changes as the test is run. In another example, flexible execution is also made possible in that the test may continue to run even if initial accuracy guarantees have been met.