XP orbs behave strangely since the 1.20 update. When fishing it is most noticeable as not all of them come in towards the player and are gathered. Sometimes they go away, and they go out of gathering range and just sit there. One has to then go and collect them. When shooting a phantom with an arrow, the xp orbs will almost do a little dance in the sky, before coming down to the ground, moving very erratically.