Sometimes, when they are near walls / colliding with walls, phantoms despawn without any discernible reason. So far, I experienced this only with south-facing walls, in Creative mode, below Y=16.

…in broad daylight. Never mind.