The pathfinding of walking villagers sometimes lets them hit a cactus and die.

Sometimes they walk too close past a cactus, so they touch it with the shoulder and get damage. Sometimes they even walk directly into the cactus and die.

Attached screenshot is a villager moments before walking into a cactus. (I didnt get the exact moment he gets damage)