When you push one block with an non Sticky Piston against the Worldborder the Piston will push through the block you are wanting to push. You can still walk on it and see it, but you can not destroy it.
Maybe the hitbox of the block is pushed thro the Worldborder and the texture and collision-box not?

For the Sticky Piston it is the same, but only then you retract it again, so say pulling it, the block will update and is behind the Worldborder.

After an reload and/or restart of the Client the blocks pushed by the Non Sticky Piston are gone. Only at the place where they where pushed from is a bugged spot, with you can't enter, without to be pushed away.