When the game rule "doTileDrops" is enabled, blocks still drop xp. I do not know if this is intended, but considering you can turn off block drops, you should be able to turn of xp drops as well.