A building texture extracting apparatus and a method thereof are provided, wherein the building texture extracting apparatus comprises a storage unit and a processor. The storage unit is configured to store an aerial image and a panoramic image of a building. There is a coordinate correlation between the aerial image and the panoramic image. The processor defines an edge segment of a building in the aerial image, calculates an edge function according to the edge segment, projects the edge function onto the panoramic image according to the coordinate correlation to derive an edge curve function, decides an edge curve segment according to the edge curve function, captures an image area under the edge curve segment to be a building texture of the building, and stores the building texture in the storage unit.