Water will never start flowing if blocks are broken around it (either by hand or player).

After reloading a world, the water will start flowing correctly, but blocking off the source blocks, doesn't make the flowing water dissapear either.

UPDATE: this only seems to happen at random.