Remote desktop sharing allows a user to access applications and data on a remote computer over a network. An intermediary computer can communicate with a host computer and one or more client computers to facilitate remote desktop sharing. The host can send the intermediary computer an encoded key frame and encoded updates. The intermediary computer can decode the host screen data, and apply subsequent updates to the decoded image so that the intermediary computer has a near live host screen image in a first buffer. A second buffer on the intermediary computer can store an instance of a screen encoder for each client computer that has joined the session and the most current host image that the client computer has. The encoder compares the near live screen image and the stored client computer host image to create an encoded differential screen image update to pass to the client computer.