Experience orbs have a client/server position desync. They can move on the client side despite being in place on the server.

How to reproduce

1. Create a superflat world
2. Place a repeating, active command block with the command /tp @e[type=xp_orb] 0 5 0
3. Throw an experience bottle
4. As you approach the orbs, they will appear to move to you, despite being actually in the same place. You will be unable to pick them up until you approach where they actually are (invisible).

Video

xp bug.mp4