Methods, systems and computer programs for determining the location in a field of play of a game controller are provided. A ball section is attached to the game controller to locate the controller using visual information. The method obtains an image of the field of play where the game controller is present, and then finds pixels in the image associated with the ball section. The method further establishes an area encompassing the found pixels and determines a geometric shape based on the area associated with the ball. The location of the controller is calculated based on the geometric shape, with the center of the geometric shape indicating the horizontal and vertical location of the controller, and the size of the geometric shape determining the depth of the controller within the field of play. The location is stored in memory, which is used to drive an action by the computer.