I have built a zombie spawner which takes zombies to a single point after a 23-blocks fall (to kill them easily).
Near this point, I have some villagers.

When there are a lot of zombies (left of the screenshot), if I leave and join the game, zombies can go through the walls.
I think they are attracted by villagers and they start moving before the chunk is fully loaded.