There is a delay(lag) when opening the inventory. When trying to view the inventory, it takes a few seconds to load.

I have however found a temporay solution

Carry a chest with you and keep it in your hotbar. When you need to access your inventory, place the chest down and open it. Now you will be able to switch items around without any LAG.

When you are done, just break the chest. Just make sure you always keep a spot open in your hotbar for the chest