A system for monitoring field conditions during the performance of an agricultural operation by an agricultural machine may generally include a support arm configured to be coupled to and extend from an agricultural machine such that, when the agricultural machine makes a pass across a field along a given swath, a portion of the support arm extends across or is positioned over at least a portion of an adjacent swath within the field. The system may also include a sensor provided in association with the support arm, with the sensor being configured to detect a parameter indicative of a field condition associated with the adjacent swath. In addition, the system may include a controller communicatively coupled to the sensor, with the controller being configured to monitor the field condition based on sensor data received from the sensor.