Embodiments provide systems, methods, and computer storage media for generating stroke predictions based on prior strokes and a reference image. An interactive drawing interface can allow a user to sketch over, or with respect to, a reference image. A UI tool such as an autocomplete or workflow clone tool can access or identify a set of prior strokes and a target region, and stroke predictions can be generated using an iterative algorithm that minimizes an energy function considering stroke-to-stroke and image-patch-to-image-patch comparisons. For any particular future stroke, one or more stroke predictions may be initialized based on the set of prior strokes. Each initialized prediction can be improved by iteratively executing search and assignment steps to incrementally improve the prediction, and the best prediction can be selected and presented as a stroke prediction for the future stroke. The process can be repeated to predict any number of future strokes.