A safe driving support apparatus may include: an emotion index detection unit configured to detect an emotion index of a driver or passenger, using one or more of vision, voice and behavior of the driver or passenger; a driver risk index detection unit configured to detect a driver risk index using ego vehicle driving information and surrounding environment information of an ego vehicle; a driving difficulty level detection unit configured to detect a driving difficult level depending on a driving position of the ego vehicle; a vehicle driving unit configured to control autonomous driving of the ego vehicle; and a control unit configured to control the vehicle driving unit to drive the ego vehicle to a safe area, according to whether the emotion index, the driver risk index and the driving difficulty level are included in a preset safety area.