To reproduce:

• Place furnace and smelt at least one item in it
• Break the furnace with a pickaxe or explode it or use `/fill … destroy` or whatever

Result: Contained items drop, but not the stored XP.

Anecdote: I have smelted cobblestone in a furnace for about a month of in-game time and a creeper just blew up the furnace, destroying all the XP inside.