In various implementations, specific attributes found in images can be used in a visual-based search. Utilizing machine learning, deep neural networks, and other computer vision techniques, attributes of images, such as color, composition, font, style, and texture can be extracted from a given image. A user can then select a specific attribute from a sample image the user is searching for and the search can be refined to focus on that specific attribute from the sample image. In some embodiments, the search includes specific attributes from more than one image.