I've entered multiple new villages in multiple worlds. If I stay in the village, I always notice the population dropping, despite no events occurring that would kill the villagers. There is no specific rate of decline, but after several in-game days, the village always winds up empty. Villagers won't breed in times of low population, causing the entire village to become deserted. 

Edit: The villagers are, in fact, dying. They leave their doors open and get killed by monsters in their beds.