A method and apparatus is disclosed herein for associating strokes with a document image. In one embodiment, the method comprises capturing strokes written on a screen over a first document image while the document image is being displayed, associating captured stroke data of the captured strokes with underlying image patches of the document image being displayed, determining that a second document image is being displayed on the screen, determining whether one or more image patches of the second document image, or parts thereof, had previously been associated with captured stroke data, and drawing one or more previously captured strokes, or portions thereof, on image patches of the second document image based on results of determining whether one or more image patches of the second document image, or parts thereof, had previously been associated with captured stroke data.