A vehicular vision system includes an in-cabin camera having a field of view that encompasses at least a hand of a driver of the vehicle. A control includes an image processor operable to process image data captured by the in-cabin camera. The control, responsive to image processing by the image processor of image data captured by the in-cabin camera, is operable to determine a movement of a hand of the vehicle driver. Responsive to determination of the movement of the driver's hand, the control is operable to determine if the movement is indicative of a gesture for controlling an accessory of the vehicle. Responsive to the determined gesture, the control may control pitch of another camera, yaw of another camera, roll of another camera, and/or a virtual viewing angle of a virtual camera of the bird's eye view system.