A semiconductor device includes a processor, a memory, a plurality of tags, a plurality of ways each of which can store a plurality of data of consecutive addresses of the memory in which a tag value stored in each tag of the plurality of tags is taken as a reference address, and a cache controller configured to determine whether a second way has an address change direction flag matching an address change direction flag of the first way and has a tag value continuous with a tag value of the first way in a direction opposite to a direction that the address change direction flag of the first way indicates and prefetch data indicated by a tag value continuous with the tag value of the first way in the direction that the address change direction flag indicates to the second way based on the result of the determination.