A control device of a continuously variable transmission has at least an automatic shift mode and a temporary manual mode. The control device includes an insufficient drive force determination unit, a first insufficient acceleration determination unit, and a downshift control unit. The insufficient drive force determination unit determines whether drive force insufficiency occurs if acceleration is performed in the current transmission gear. The first insufficient acceleration determination unit determines whether there is an acceleration request made by the driver of the vehicle. The downshift control unit downshifts from the current transmission gear to a transmission gear at a lower speed side in a case where, in the temporary manual mode, the insufficient drive force determination unit determines that a drive force insufficiency occurs and the first acceleration request determination unit determines that there is an acceleration request.