An electric submersible well pump assembly includes a pump and an electrical motor filled with a dielectric lubricant. A capsule extends around a portion of the motor, defining a chamber between the capsule and the motor. A port through a side wall of the motor communicates the lubricant within the motor to the chamber. Thermoelectric devices are located within the chamber. Voltage supplied to the thermoelectric devices cools the lubricant.