A downhole tool includes a main body, and a setting member configured to press the main body radially outwards so as to set the main body with the surrounding tubular, made at least partially from a dissolvable material configured to dissolve in a well fluid, and defining a bore therein. The tool also includes an acid pill positioned in the bore of the setting member. The acid pill contains an acid therein, and is made at least partially from a dissolvable material configured to dissolve in the well fluid such that the acid mixes with the well fluid upon the acid pill at least partially dissolving. The acid mixed in the well fluid increases a rate at which the dissolvable material of the setting member dissolves in the well fluid.