An overlay network platform facilitates a multi-party conference. End users participate in the conference using client-based web browser software, and using a protocol such as WebRTC. According to this disclosure, an enhanced “audio” experience for the conference is providing by collecting and correlating microphone data from multiple co-located clients, and then constructing (at the platform) a three-dimensional (3D) sound profile of the room in which the clients are co-located. By processing in the platform (as opposed to locally at each client), the approach enables platform-side creation of an ad-hoc, high quality microphone array that identifies the relative positions and orientations of the microphones that are being used by the clients. Individual audio streams received from the microphones are combined, and the relative position information (of the individual microphones) is used to render a single audio stream that represents a high quality recording of the audio in the common physical space. Other clients in the conference request, receive and play back this high quality stream to obtain a high-fidelity 3D representation of the audio as if they are physically present in the room.