A fisheye image transformation algorithm based on virtual spheres is provided, comprising: defining a real camera coordinate system O-XYZ and a virtual camera coordinate system O2-X2Y2Z2, wherein a small virtual sphere centered at O with a radius of 1 and a large virtual sphere centered at O2 with a radius of n are defined, the small sphere is internally tangent to the large sphere; transforming a fisheye image to 3D unit vectors in the real camera coordinate system, transforming the 3D unit vectors between the real camera coordinate system and the virtual camera coordinate system according to a relative position between two spheres, and conducting conventional projection of the 3D unit vectors in the virtual camera coordinate system to generate the transformed image; and reducing the line curvature of the transformed image to a certain degree by adjusting the HFOV angle or the VFOV angle of the virtual camera.