I have repopulated and expanded a naturally generated village after all of its villagers died. I have master level villagers, but sometimes when a new villager grows up, they take a job block that is already claimed by one of the master villagers, leaving them to wander aimlessly around town. The blocks aren't being moved or destroyed before being reclaimed.