The present application provides a method and a device of encoding and decoding based on free viewpoint, and relates to the technical field of video encoding. The method includes: generating a planar splicing image and splice information based on multiple single-viewpoint videos at a server side; generating a planar splicing video based on the planar splicing image; generating camera side information of the planar splicing video based on camera side information existing in the multiple single-viewpoint videos; and encoding the planar splicing video, the splice information and the camera side information of the planar splicing video to generate a planar splicing video bit stream, and decoding a planar splicing video bit stream to acquire a virtual viewpoint according to viewpoint information of a viewer at client side.