Every time I enter my inventory or use a chest, the FPS drops massively, going from 60FPS down to 30 or 25FPS.

I'm pretty sure this has to do with the enchantment effect on items, or even the amount of items the inventory/chest/barrel has, but the weird thing is, the inventory does NOT lag at all IN CREATIVE.