A conference system includes at least one server and at least one mobile terminal connectable to the server. The mobile terminal includes an orientation estimation unit configured to detect orientation information of the mobile terminal, an operation unit configured to detect operation information of an operation performed; and an event transmission unit configured to transmit the orientation information and the operation information. The server includes an image information generation unit configured to generate image information of a conference material and image information of a pointer drawn over the image information of the conference material based on the drawing event, and an action information generation unit configured to generate action information corresponding to the conference material based on drawing history information of the drawn pointer.