Methods, systems, and computer readable media for acoustic classification and optimization for multi-modal rendering of real-world scenes are disclosed. According to one method for determining acoustic material properties associated with a real-world scene, the method comprises obtaining an acoustic response in a real-world scene. The method also includes generating a three-dimensional (3D) virtual model of the real-world scene. The method further includes determining acoustic material properties of surfaces in the 3D virtual model using a visual material classification algorithm to identify materials in the real-world scene that make up the surfaces and known acoustic material properties of the materials. The method also includes using the acoustic response in the real-world scene to adjust the acoustic material properties.