In one aspect, a system for monitoring soil composition within a field using an agricultural machine may include a ground-engaging tool configured to rotate relative to soil within a field as the agricultural machine is moved across the field. The ground-engaging tool may, in turn, include a tooth defining a cavity therein, with the cavity including an opening. Furthermore, the system may include a sensor positioned within the cavity, with the sensor configured emit an output signal through the opening for reflection off of the soil within the field. The sensor may also be configured to detect the reflected output signal as a return signal, with a parameter of the return signal being indicative of a soil composition of the soil within the field.