A method of simulating physics in a virtual worlds system includes selecting at least one of the client devices participating in an instance of a scene as a physics host, the physics host determining subsequent states of objects and sending the subsequent states to one or more processors of a server, the subsequent states of objects comprising one or more of: subsequent locations, orientations, velocities and accelerations determined based on characteristics of the objects and constraints for simulating physics consistent with the new instance of the scene of the virtual worlds system.