Steps to reproduce:

• use default 1GB of memory allocated
• Create new creative world with customized world type with mega taiga hills biome (this is to ensure known, constant average memory usage per chunk)
• Set render distance to 32
• Press F3 and wait until MultiplayerChunkCache value no longer increases (should get to about 3800-4000)
• Spawn a sheep and hit it
• Save and exit
• Without restarting Minecraft, load that world again
• Move a few hundred blocks away from spawn
• After a while you should notice the game freezing very frequently for a fraction of a second, caused by being close to running out of allocated memory (depending on exactly how far away you move, you may run out of memory and crash Minecraft). Sometimes hitting an entity again makes the issue go away, sometimes it doesn't.

Why this proves there is an issue?

Because being close to full memory usage means the JVM will try to do full GC, removing all stale objects. So if even after that, memory usage is still high enough to frequently trigger full GC, there muct be more memory actually being used by something. Heap dump can confirm that there actually are 2 WorldServer objects ("oo" class in 1.12.2 obfuscated Minecraft).

Most likely cause:

I initially found this issue with my mod, and I couldn't find anything in heap dump pointing to my code. The only GC roots seemed to be related to a class MCP calls EnchantmentHelper and it's inner Iterator classes.

After testing it in vanilla, I found that it's actually vanilla issue. The issue seems to be that the inner Iterator classes hold references to Entities, potentially long after the world the entities are in has been unloaded (until they are replaced by entities from a new world, which most of the time happens when the player hits an entity, I wasn't able to figure out why it fixes the issue only sometimes).