A method to upgrade heavy oil feedstock using an ebullated bed reactor and a novel catalyst system is provided. The ebullated bed reactor system includes two different catalyst with different characteristics: an expanded catalyst zone containing particulate catalyst having a particle size of greater than 0.65 mm; and a slurry catalyst having an average particle size ranging from 1 to 300 μm. The slurry catalyst is provided to the ebullated bed system containing the heavy oil feedstock, and entrained in the upflowing hydrocarbon liquid passing through the ebullated bed reaction zone. The slurry catalyst reduces the formation of sediment and coke precursors in the ebullating bed reactor system. The slurry catalyst is prepared from rework materials, which form a slurry catalyst in-situ upon mixing with the heavy oil feedstock.