A refrigerator appliance is provided including a cabinet defining a chilled chamber, a door rotatably hinged to the cabinet to provide selective access to the chilled chamber, and an audiovisual input assembly for monitoring the chilled chamber. A controller obtains a video stream and an audio stream using the audiovisual input assembly and analyzes these streams using a standard object weighting to generate a video-based inventory change and an audio-based inventory change. An inventory list is updated if the video-based inventory change and the audio-based inventory change match. Otherwise, the controller can reanalyze the streams using an updated object weighting to see if the audio and video streams generate matching proposed inventory changes.