Speech-based interaction with a display window is disclosed, including: receiving speech input with respect to a display window that is presented at a device; converting the speech input into an operation instruction; determining an interface element of the display window that corresponds to the operation instructions, including by querying interface element information corresponding to the display window; and activating the interface element of the display window based at least in part on simulating a manual selection event with respect to the interface element of the display window.