A conference system provides metadata about the display of a multimedia stream to allow communication endpoints to control the format of a video conference. A multipoint control unit (MCU) can provide metadata that describes the display layout and other information about the multimedia stream. After setting up the conference, the MCU can generate a layout description in eXstensible Markup Language (XML) that can be sent to and understood by the communication endpoints. The communication endpoints can read and interpret the XML metadata to determine how the display layout is configured. If desired by the user, the communication endpoint can change the display layout or other multimedia stream function based on the received metadata.