A system is configured to implement a graphical overlay in a streaming game based on a current game state. Game data generated by a video game is received including game video in the form of a video stream containing game video frames. The game video is displayed on a display screen of a computing device to represent the video game to a user playing the video game at the computing device. At least one feature of the video game is identified at least in the game data. A user interface (UI) control configuration associated with the identified at least one feature is selected from among a plurality of UI control configurations for the video game and a graphical overlay corresponding to the selected UI control configuration is implemented on the video game in the display screen.