The zombie and drowned only do tracking the villager if there is a hole at height the head of the villager. If the villager is completely covered (even when they are transparent blocks) the zombie and the drowned do not track it. In the case of opening a hole at the height of the villager's feet it is also not tracked.