Sometimes you can get fake XP orbs that are flying around you, but you can't collect. Reconnecting fixes it.

Video (by DocM77): http://www.youtube.com/watch?feature=player_detailpage&v=XQNtBD1jMQE#t=1375s (until about 24:00)

How to reproduce:
1) Download DocM77's world: http://www.mediafire.com/?sbnqqj7481onfdc
2) Go to the endermen farm.
3) Kill some endermen.

Here is one of the comments on that video, that might explain why this happens:

i had that glitch with xp orbs and i resolved it putting non-transparent block under my mob farm. I think the glowstone is holding the orbs underneath, blocking only the real orbs, but not the animation that will keep go until you get the real orbs or relog