A semiconductor device has a carrier with a die attach area. A semiconductor die is mounted to the die attach area with a back surface opposite the carrier. A modular interconnect unit is mounted over the carrier and around or in a peripheral region around the semiconductor die such that the modular interconnect unit is offset from the back surface of the semiconductor die. An encapsulant is deposited over the carrier, semiconductor die, and modular interconnect unit. A first portion of the encapsulant is removed to expose the semiconductor die and a second portion is removed to expose the modular interconnect unit. The carrier is removed. An interconnect structure is formed over the semiconductor die and modular interconnect unit. The modular interconnect unit includes a vertical interconnect structures or bumps through the semiconductor device. The modular interconnect unit forms part of an interlocking pattern around the semiconductor die.