A smart display allows a user to build custom layouts of user interface blocks on the smart display independent of the software on the computer creating the user interface. A customization mechanism in the smart display allows a user to select portions of a user interface and move them to different positions on the display. The customization mechanism creates custom layout metadata that defines a screen offset for portions of a user interface moved by the user. The smart display monitors the incoming display data and re-assigns pixel rendering data to the new location in the moved user interface blocks as the data coming from the computer application changes.