A neural network computing device and a cache management method thereof are provided. The neural network computing device includes a computing circuit, a cache circuit and a main memory. The computing circuit performs a neural network calculation including a first layer calculation and a second layer calculation. After the computing circuit completes the first layer calculation and generates a first calculation result required for the second layer calculation, the cache circuit retains the first calculation result in the cache circuit until the second layer calculation is completed. After the second layer calculation is completed, the cache circuit invalidates the first calculation result retained in the cache circuit to prevent the first calculation result from being written into the main memory.