A modular building system uses a perimeter bond beam and prefabricated wall panels to form an exterior curtain wall. The wall panels are captured within, and may move vertically inside a channel formed in the bond beam. The wall panels are quickly erected by inserting an upper edge of each panel into the channel and swinging the panel onto the foundation where it is claimed in place on the foundation.