Embodiments of the invention provide a system for reconfiguring touch controls on a device, such as a cell phone, in reaction to a user's changing mode of operation. For example, a standard display of icons that works well in two-handed operation may not work as well for one-handed operation where the user tries to operate the controls with a thumb of the holding hand. In such a case, a device can detect that the user is trying to operate the controls with a thumb of the holding hand and the controls can be reconfigured to be more suitable for thumb operation. Reconfiguration can also occur in other modes of operation, such as when the user is using a stylus, multiple fingers of a second (non-holding hand), fingers on both hands, etc. In some embodiments, detection of the mode of operation can be automatic such as where a camera takes an image to determine the mode.