Rebinding Open/Close Inventory won't close inventory

I've been playing games that use TAB as the inventory button lately and now I'm use to it. So I swapped 'e' and 'TAB' from their default functions. Doing so TAB will open the inventory but not close it.

1. Rebind Open/Close Inventory to TAB. (change TAB's default function to avoid keybind conflict)
2. Head in-game and press TAB to open inventory.
3. Press TAB again to close, but it won't close.