Villagers would despawn if I leave the nether or end dimension after having brought villagers to those dimensions.

Also villagers appear to be randomly despawning in general even though they aren't supposed to despawn at all.