A system and method for three-dimensional (3D) object classification are disclosed. A computing system receives point cloud data from an input source. The point cloud data may include first and second points in 3D space. The first point may represent a feature of an object. The computing system invokes a neural network for classifying the first point with a first label, and regresses a bounding box based on classifying the first point with the first label. Regressing the bounding box includes predicting at least a location of the bounding box. The computing system may control an autonomous vehicle based on regressing the bounding box.