A process to upgrade heavy oil and convert the heavy oil into lower boiling hydrocarbon products is provided. The process employs a catalyst slurry comprising catalyst particles with an average particle size ranging from 1 to 20 microns. In the upgrade process, spent slurry catalyst in heavy oil is generated as an effluent stream, which is subsequently recovered/separated from the heavy oil via a filtration assembly. The filtration assembly has a least a filtration unit employing at least a membrane for separating heavy oil from the catalyst particles. Valuable metals can be recovered from catalyst particles for subsequent re-use in a catalyst synthesis unit, generating a fresh slurry catalyst.