I started a world, found a village, took some hay then went down into a cave. i popped back up a couple days later and all villagers but 2 were gone, the Iron golem was still there.