It's mostly in the title -  when a villager refuses, and shakes their head, the direction their head shakes is based on what direction the villager's body is facing, not the head - so if the villager is looking to the side, they may look like they're nodding!

Attached is a short video of this in action - although the villager was constantly moving, the effect was visible.