A system and method for script and orientation detection of images are disclosed. In one example, textual content in the image is extracted. Further, a vertical component run (VCR) and horizontal component run (HCR) are obtained by vectorizing each connected component in the extracted textual content. Furthermore, a concatenated vertical document vectors (VDV) and a horizontal document vector (HDV) are computed. In addition, a substantially matching script and orientation is obtained by comparing the computed concatenated VDV and HDV of the image with reference VDV and HDV associated with each script and orientation, respectively. Also, the substantially matching script and orientation are declared as the script and orientation of the image, if the computed concatenated VDV and HDV of the image substantially match with the reference VDV and HDV of the matching script and orientation, respectively.