Described is a technology by which a gesture made with a source device (e.g., a cellular telephone), such as a throwing or pointing motion, is used to automatically set up a connection with another device to which the gesture is directed. Audio signals output during the gesture, e.g., such as at the start and end of the gesture, are detected by candidate (listening) devices. The device having the least time difference between detection of the start and end signals is the device that is generally best aligned with the throwing or pointing motion, and thus may be selected as the target device. Once selected, a connection such as a network connection may be set up between the source device and the target device for further communication, such as for application data exchange.