The present invention relates to a rust removing device for an elevator guide rail and aims at solving a problem of great time and labor taken and insufficient rust removal due to that rust of existing guide rail is removed manually. The rust removing device comprises: a support; a rust removal rack, mounted on the support and covering guide rail; and a rust removal material, arranged on a U-shaped inner wall of the rust removal rack and fitted with a surface of the guide rail. The rust removing device of the present invention has advantages that the rust of the guide rail is removed automatically instead of being removed manually, featured by a high rust removing speed, energy saving, simple maintenance, and less influence on users due to long-time service stop because of rust removal.