A multi-directional remote control system and method is adapted for use with an entertainment system of a type including a display such as a monitor or TV and having display functions employing a mouse type control. The remote controller may be conveniently held in one hand of a user and still provides full mouse type functionality. The remote control system and method images the screen of the TV or other display to detect relative motion between the controller and screen. This position information is transmitted to the entertainment system for control of a cursor or other GUI interface.