Multi-piece golf balls containing a multi-layered core structure having layers with different hardness gradients and specific gravities are provided. The core structure includes a small, heavy inner core (center) having a relatively high specific gravity with metal material preferably dispersed in a first thermoset composition. The intermediate core layer is preferably formed from a thermoplastic composition such as an ethylene acid copolymer ionomer resin; and the outer core layer is preferably formed from a second thermoset composition. The resulting ball has high resiliency and good spin control.