When attempting to swap an item to a different slot in your inventory by simply pressing the key (here my key happens to be 'f', the default 'swap offhand and main hand' key), the item doesn't go to the correct slot, it instead automatically goes to the offhand.

This is easily reproducable:

1. Swap a hotkey for any inventory slot (say for slot 9) to the 'f' key.
2. Unbind the key to swap an item from the offhand to the current inventory slot using the 'escape' key.
3. Place an item in slot 1.
4. Open your inventory, and press the 'f' key. The item will go to the offhand, instead of the newly designated slot.