A system and a method of multi-user coaching are introduced herein. Motion-sensing cameras are applied to capture images, and a depth image stitching module is applied to perform a depth image stitching process on the captured images to expand the motion-sensing range, so as to establish a virtual environment for multi-user coaching. Each user can be coached individually by a one-to-multiple approach to improve his or her motions. By using the system and the method of multi-user coaching, that is, the system can only calculates on motion similarities of the users, and instructions are fed back to each user. Therefore, the system and the method described herein can be extensively applied to various products, such as a virtual gymnasium, a virtual aerobics classroom, a virtual Budokan, and so on.