In a insert-type continuous casting nozzle comprising a highly functional layer formed to have a high corrosion resistance, a high anti-attachment capability, etc., and provided to define an inner bore thereof, the present invention is directed to providing a refractory material (mortar) for an intermediate layer of the continuous casting nozzle, which has a property capable of fixing an inner bore-side layer to an outer periphery-side layer (a nozzle body) of the continuous casting nozzle, while preventing the occurrence of expansion splitting in the outer periphery-side layer due to a difference in thermal expansion between the inner bore-side and outer periphery-side layers, and a continuous casting nozzle using the refractory material for the intermediate layer. The refractory material for the intermediate layer contains a hollow refractory aggregate in an amount of 10 to 75 volume %, wherein a ratio of an average radius R of each particle of the aggregate to an average wall thickness t of the particle satisfies the following relation: R/t≧10. This refractory material is disposed between an inner bore-side layer (2) and an outer periphery-side layer (3, 4) of a continuous casting nozzle.