To generate 3D representation of a scene volume, the present invention combines the 3D skeleton approach and the shape from silhouette approach. The present invention efficiently works on complex scenes like sport events with multiple players in a stadium, with an ability to detect a wide number of interoperating 3D objects like multiple players.