If you press "E" on the keyboard, by default the inventory will open. You can also press "E" a second time to close. Closing the menu by pressing a key for a second time does not work once changed.

Edit: only breaks when set to a mouse button.