Provided is an image forming apparatus having a charge brush for charging a transfer residual toner remaining on an intermediate transfer member. A toner is adhered to the charge brush. While a charge voltage is applied to the charge brush, the toner adhered to the charge brush is attracted by an electric field onto the charge brush. However, when the charge voltage is not applied to the charge brush, the toner adhered to the charge brush is moved to the intermediate transfer member. When a subsequent toner image is formed on the toner adhered to the intermediate transfer member, the quality of the subsequent image is deteriorated. Accordingly, the charge voltage is applied to the charge brush when an intermediate transfer area to which the toner image is transferred onto the intermediate transfer member immediately afterward faces the charge brush.