Usually this error shows up when a field that is referenced in the Display Fields web part property is not properly installed (custom field) or the user doesn't have proper permissions to access the field (cross site lookup field).
Since you mention that this error is intermittent and it goes away after clearing the cache and restarting the browser the above may not apply. It'd be good to take a look to the ULS log entry for this error and see more details.
Also, please attach the list and web part template and let me know your QWP, SP and browser version so I can try it.
Thanks,