A passive 3D image system includes a passive 3D image display apparatus, a scaling module and a luminance adjusting module. The 3D passive image display apparatus includes N rows of display units. After receiving an original image including N rows of original data, the scaling module merges the N rows of original data to generate a scaled image including N/2 rows of scaled data. The original image is either a left-eye image or a right-eye image. The luminance adjusting module adjusts the N/2 rows of scaled data to increase the luminance of the scaled image.