A display device, a remote control device for the display device, a method of controlling the display device, a method of controlling the remote control device, and a method of controlling a server includes collecting at least one piece of situation information or receiving at least one piece of situation information from a remote control device, determining a current state of a user based on the collected or received at least one piece of situation information, instructing the remote control device to output a notification signal corresponding to the current state of the user, and generating the control command according to the current state of the user and providing a sense-based image service to the user according to the control command.