The method to determine tampering of a security label (102) comprises, associating at least a portion of a first pattern to an external reference (110), wherein a first layer (202) of the security label (102) comprises the first pattern. Further, a second pattern (206) defined in a second layer (204) is used to change the contour of the portion of the first pattern, when the security label (102) is at least partially disengaged from a surface. Subsequently, when there is change in contour of the portion of the first pattern, the portion of the first pattern is disassociated from an external reference (110). Further, the portion of the first pattern and the external reference (110) are scanned and finally tampering of the security label (102) is determined based on the association between the portion of the first pattern and the external reference (110).