This disclosure relates to method and system for providing personalized driving or navigation assistance. The method may include receiving sensory data with respect to a vehicle from a plurality of sensors and multi-channel input data with respect to one or more passengers inside the vehicle from a plurality of onboard monitoring devices, performing fusion of the sensory data and the multi-channel input data to generate multimodal fusion data, determining one or more contextual events based on the multi-modal fusion data using a machine learning model, wherein the machine learning model is trained using an incremental learning process and comprises a supervised machine learning model and an unsupervised machine learning model, analysing the one or more contextual events to generate a personalized driving recommendation, and providing the personalized driving recommendation to a driver passenger or a navigation device.