This is directed to controlling media playback based on particular touch gestures detected by a touch-sensing interface. The electronic device can identify particular touch inputs, such as combinations of tapping and holding a touch sensitive surface, or circular motions. In response to detecting a particular touch gesture, the electronic device can perform a playback operation specifically associated with the detected touch gesture. To provide a consistent user interface with the device, some of the particular touch gestures can match other inputs provided using a button, for example a button integrated on a wired headset. In such an embodiment, the same combination of tapping and holding a touch input and pressing and holding a button can control the same electronic device operations.