jeb resolved this report as "Works As Intended". He also commented roughly one year later as response to the following:
I always viewed the breaking of the sticky piston connection as inertia, and taking away the block teleportation part, I don't think this behaviour is buggy at all.
The reason why the piston behavior "technically a bug" is because the block in front of the sticky piston is supposed to act as if fixated to the piston. Your interpretation as "it's inertia" actually makes sense.
When a sticky piston is powered with a very short pulse ("0-tick pulse"), it will push the block, but not retract it.
Build the contraption in this screenshot:
The redstone block will not be pulled back.
After 1.14, it is also possible to reproduce this by placing a button on the side of a sticky piston, and press the button to move a block.