Before 1.14, villagers would stare at an iron golem randomly, similar to how they randomly look at other mobs. However, they do not randomly stare that iron golems.

To reproduce, spawn a villager near iron golem, and wait:
Minecraft 1.16.2 - Singleplayer 2020-09-03 07-52-52.mp4