There seems to be a bug in which a villager will become linked to the same job block as another villager (for example I'll have 3 looms in my village and there will be four shepherds). Destroying the job block temporarily fixes the issue, but then usually the villager will just transition to another taken job block. It seems to only be affecting largely populated villages as I didn't have the issue until I had a pop. of like 10 or so villagers. It also only seems to be affecting baby villagers that grow up and take a job. It's more frustrating than game-breaking but hopefully, this helps resolve the issue.