A piston powered diagonally creates a BUD. If the redstone on a block diagonal to the piston has a signal strenght of one, and so a redstone on top of a block above the piston isn't powered, placing said "level one" redstone will power the piston. It will unpower it if you break it.

That's the only way to toggle the piston, any other way to turn off the level 1 redstone will cause a BUD, as if the redstone above the piston wasn't there and so acting normal (emptying the dispenser hooked up to a comparator to create the signal of the appropriate strenght, preventing the comparator to emit by powering its B input, breaking the source or any redstone piece on the line that isn't the level 1 one).

It's not directionnal, and happens in both gamemodes, and will not happen with glowstone or half slab as the block above the piston.

I think it comes from the level 1 redstone placing / breaking causing an update to the adjacent redstone piece which update the block below, because it's redstone, which in turn update the piston. Although i don't know the inner workings of redstone, so take it only as the assumption that it is (and correct me if i'm wrong, i would appreciate to know more about redstone).

(Edit) Mentionned the glowstone and half slab thing.