Caves that generate underneath frozen oceans will have icebergs generating to replace air blocks. It appears that they don't replace stone blocks. These icebergs generate directly underneath the corresponding icebergs found on the ocean surface.

This issue appears to be related to MCPE-125156.
I've only tested worlds with "Caves and Cliffs Experimental Features" enabled so far. Will update this report with screenshots and further testing.

Edit: merge with MCPE-125128?