The present invention discloses a remote controllable video display system and a controller therefor, wherein positional information is generated by a sensor according to light from a light source. The controller includes a button with a predetermined function, wherein when the button is pressed or released, the status of the button (pressed or released) is outputted but (i) the positional information is not outputted or not processed within a predetermined period of time; or (ii) the positional information is not outputted or not processed when the remote controller has an acceleration larger than a predetermined value. Or, when the button is pressed once, the controller outputs double-click or multiple-click action information.