A multi-modality system (10) includes a nuclear imaging system (12) and a computed tomography (CT) scanner (14). The nuclear system (12) includes a PET scanner (28) which acquires electronic data that is reconstructed into a PET blob image by a PET reconstruction processor (50). The CT scanner (14) acquires the scanned data which is reconstructed into a 3D CT voxel image by a CT reconstruction processor (56). An interpolation processor (62) interpolates the PET blob image directly into the CT voxel space. Once the PET and CT images are in the same space, they are combined by a combining means (110). A video processor (66) processes the received composite PET-CT data for a display on a monitor (68).