Pistons aren't unpowered if they are powered diagonally. It seems to me that they can not be powered diagonally, but when they are updated, they mistakenly think they are powered and thus don't retract. See screenshot #1. I achieved that by flipping the lever on the left and then pressing the button on the right.

Further proof for this behavior is screenshot #2: I achieved that by pressing the button on the left, then immediately pressing the button on the right. What (I think) happens is:
1) The piston is powered and extends.
2) The block diagonally next to the piston is powered.
3) The piston is unpowered, but it believes it's still powered by the block diagonally above it and thus doesn't retract.
4) The redstone on that block turns off, but doesn't update the piston, which is why it stays extended without any redstone current at all.

What did I expect to happen? Well, I expected the piston to completely ignore anything that's not directly next to it.