A system capable of recognizing position, a shape, a posture and the like of an object present in a marginal environment of a device such as a robot in order to make the device perform operations on the object as a subject. In an environment recognition system, 3D information and physical information (color information and the like) of a subject are associated by using camera parameters of each of a 3D image sensor and a 2D image sensor. Thereby, the position, the posture and the shape related to the subject and the physical information of the subject present in the environment of a robot are obtained.