I've been exploring a few worlds for a few hours now, and every single cave I come across is completely filled with water. Caves have been far and few between for me anyway, but I can't find one that is explorable without a water breathing potion or turtle shell.