Implementations generally relate to localizing and mapping. In one implementation, a method includes determining one or more map points in a point cloud space, where the point cloud space corresponds to a real physical environment. The method further includes determining movement information of one or more of the map points, where the determining of the movement information is performed by a neural network. The method further includes determining a self-position in the point cloud space based on the determined movement information.