Villagers don't look directly at you. They look at your feet, and also look down at each other. This bug is only with villagers, so it must be something to do with them, and not with the player hitbox. Version that I'm in is 20w15a but it started before that. Definitely in 1.16 though.