A sports video game having a new and improved user interface that substantially increases the potential realism of the game uses a handheld controller that is able to sense its own position, orientation and/or motion and is affixed to a ball. The ball can be a model of a football that is provided with a depression that snugly accepts and retains the handheld controller. The handheld controller may sense motion, position and/or orientation of the ball and provide wireless inputs to a video game system. The video game system may process such inputs to provide a simulated sports action game play on a video display.