Villagers can climb up ladders or vines when wandering around, fall from there and take some fall damage. This kills them if they do this enough. This happened in me and my brother's village and will continue to happen unless we remove all the ladders and vines from there.