While diggin, you will not be able to break stone when there is 2-3Blocks of Dirt, Sand, Gravel in front of it. It seems that the Efficiency V Pickaxe digs too fast and then the stone can not be broken. (Same effect as braking stuff at Spawn)

Reproduce:
-Place 2-3Blocks of Dirt with a block of stone at the End.
-Digg it with Efficiency V Diamond Pickaxe