I play on a realm and I recently created a villager breeder. The breeding villagers don't despawn, possibly because they were zombie villagers before I converted them, but the other villagers do. 

 Each room looks like this, and the hallways are fully lit with torches. I did make sure to trade with them, haven't tried name tags on them yet. Some villagers stay however, and they only stay for a few days before despawning. This bug is frustrating and I really need a fix or this is just a waste of my time. (Usually happens when I leave the realm and join back at a later time)