The present invention is directed to a system and method for populating a database with a set of image sequences of an object. The database is used to detect a tubular structure in the object. A set of images of objects are received in which each image is annotated to show a tubular structure. For each given image, a Probabilistic Boosting Tree (PBT) is used to detect three dimensional (3D) circles. Short tubes are constructed from pairs of approximately aligned 3D circles. A discriminative joint shape and appearance model is used to classify each short tube. A long flexible tube is formed by connecting all of the short tubes. A tubular structure model that comprises a start point, end point and the long flexible tube is identified. The tubular structure model is stored in the database.