The present invention discloses a method and a system for assisting user with product handling in a retail shopping. The method comprising identifying one or more products from a list of products that user intends to purchase, neighbouring products and orientations of the one or more products and the neighbouring products from a plurality of image frames and rack information, comparing the identified orientations with historic orientations of products and neighbouring products associated with the rack information, identifying at least one product from the one or more products and the neighbouring products that is improperly oriented based on comparison, determining product safety handling instructions of the identified at least one product and assisting the user in real-time with handling of the identified at least one product based on the product safety handling instructions.