Simple enough, with Haste (I've also tried resistance, it does the same) you clearly break blocks very quickly, but when you leave the world and rejoin (ignore the fact that i went into video settings) it resets the level of haste.

Edit: Uploaded correct video file