Generating a grasp pose for grasping of an object by an end effector of a robot. An image that captures at least a portion of the object is provided to a user via a user interface output device of a computing device. The user may select one or more pixels in the image via a user interface input device of the computing device. The selected pixel(s) are utilized to select one or more particular 3D points that correspond to a surface of the object in the robot's environment. A grasp pose is determined based on the particular 3D points. For example, a local plane may be fit based on the particular 3D point(s) and a grasp pose determined based on a normal of the local plane. Control commands can be provided to cause the grasping end effector to be adjusted to the grasp pose, after which a grasp is attempted.