Holding arrows in your off-hand can result in them becoming stuck there, and you are unable to remove them from that part of your inventory. Further more, you cannot collect dropped arrows into this slot which means you cannot pick up arrow drops from mobs.