Disclosed are a device for user interaction with a combined projector and camera and a method and a device for user interaction for recognizing an actual object to augment relevant information on a surface or a periphery of the actual object. The device for user interaction, includes: at least one projector-camera pair in which a projector and a camera are paired; a motor mounted in the projector-camera pair and configured to control a location and a direction of the projector-camera pair; and a body including a computer capable of including a wireless network and configured to provide connection with an external device, and a projection space and a photographing space of the projector-camera pair overlap each other.