Facilities of a shared environment are automatically optimized by inferring sentiment from unstructured conversational data towards various environmental entities such as heat, light, service levels, etc. Conversational audio streams from different areas are analyzed to identify an entity and associated sentiment, and a heatmap is created representing the sentiment across the different areas. The conversational audio streams are captured by directional microphones and are assigned metadata such as a location tag indicating a position of a microphone within the shared environment. Heatmap creation can be supplemented by other sensory data. A cognitive system is used to generate actions for control of the facilities based on the heatmap. A suggested action may still be subject to operational policies for the facility. In some scenarios a first suggested facility action compensates for an effect of a second suggested facility action.