Entities, even those not visible to a player, cause massive drops in FPS.

On the example picture, the character is standing at the same place, just looking in different directions. The "museum" contains some Item Frames and Paintings. Even though not visible to the player, the fps drop is massive (from over 100fps to 12fps, in this case).

I would suggest some kind of hidden entity detection to not render those entities.