Described is a virtual environment built by drawing stacks of three-dimensional objects (e.g., discrete blocks) as manipulated by a user. A user manipulates one or more objects, resulting in stack heights being changed, e.g., by adding, removing or moving objects to/from stacks. The stack heights are maintained as sample points, e.g., each point indexed by its associated horizontal location. A graphics processor expands height-related information into visible objects or stacks of objects by computing the vertices for each stack to draw that stack's top surface, front surface and/or side surface based upon the height-related information for that stack. Height information for neighboring stacks may be associated with the sample point, whereby a stack is only drawn to where it is occluded by a neighboring stack, that is, by computing the lower vertices for a surface according to the height of a neighboring stack where appropriate.