Sometimes, kelp or seagrass can spawn one block under ice and it creates a little bug. You can see through ice and any update will break the kelp / seagrass, also, when they will break, the water will flow.

Attachment: https://www.youtube.com/watch?v=a2JClcGhgA4&feature=youtu.be