Natural three-dimensional (xw, yw, zw, tw) gesture player interaction with a two-dimensional game application rendered on a two or three dimensional display includes mapping acquired (xw, yw, zw, tw) gesture data to virtual game-world (xv, yv, zv, tv) coordinates or vice versa, and scaling if needed. The game application is caused to render at least one image on the display responsive to the mapped and scaled (xw, yw, zw) data, where the display and game interaction is rendered from the player's perception viewpoint. The (xw, yw, zw) data preferably is acquired using spaced-apart two-dimensional cameras coupled to software to reduce the acquired images to a relatively small number of landmark points, from which player gestures may be recognized. The invention may be implemented in a handheld device such as a smart phone or tablet, which device may include a gyroscope and/or accelerometer.