Since I was asked in MC-121190 to make a seperate issue.
So I found During Porting of Chunk Pregen to 1.14.4 i found Memory (Leaks / issues) with Minecrafts Worldgen/ChunkManager
And during some testing into it I found the root causes for the leaks and some strong hints why issues appear.
(May also result in better performance)
- Cause 1: PointsOfInterests get "NEVER" ever unloaded. In a Small playthrough thats fine, but for servers who are generating larger areas thats a big problem.
Its the "Structure" memory leak in 1.12 or older all over again that you actually fixed. A 40k Generation (10 mins of pregen) will cause around a 50-100MB RAMLeak Until the server restarts. Depends on the POI.
- Fix for Cause 1: Unload the PointsOfInterests when the chunk unloads. That saves a lot of ram and the problem too.
- Cause 2: Every Chunk that gets LightEngine checked that is not loaded by a player does not get its data unloaded until a player visits that Chunk.
This can be easily replicated because its an issue with the TicketSystem. ForceLoad a chunk via a command and you get around 121 chunks stuck in the LightEngine.
- Fix for Cause 2: Mark Any chunk that unloads not to raintain the lightengine data.
Both Fixes were implemented in ChunkPregenerator and were extensivly tested.
Now lets get to the Memory Issues:
First of all a tiny Rant about the ServerChunkProvider
Why the hell do you have to implement such a optimization?
Isnt it intendet that the chunkmanager provides all chunks the way you want them?
I get why its there, but the issue is Minecraft isnt really about "Accesing the same chunk thousands of times per game tick and then the next chunk"
Ok but now to the real problem with MCs worldgen memory issues.
MCs ChunkManager and TicketManager a full of Lambda Expressions not even cached function expressions, nope its full of newly created allocations.
So for every chunk and for every new lambda you are creating a lot of new Memory just to process things async.
Its to the point where you create enough garbage through this system that you are having a performance hit.
Same with gamelogic.
So the issue i found was: Every Chunk in worldgen but also in loading allocates so much Ram because of the wasteful overhead that it is having.
Solution: Stop this Methodbased programming. Yes it looks nicer i give you that but you are slowing everything down. I mean technical players already know that the game is half as fast as it was before, this is 1 reason because of that.
If you "HAVE" to use a Lambda. Cache it and Reuse it, that way you have 1 allocation and a smaller performance hit, because a lambda is slower then or equal calling everything via reflection.
So Memory leaks when loading a lot of chunks really quickly can be explained simply by that.
So to give some good feedback and dont turn this into a rant only:
I love the new Ticket System and also Chunk Request System.
Its a good idea, it works smoother and allows me to reduce my memory footprint massivly.
I pregeneration of 1000 Chunk radius before was eating 1.5GB do to how chunk loading worked,
Now I get less then 10MB of ram usage thanks to this system. The Cap is now a radius of 25k Chunks.
Also thanks to the new system I am able to multithread chunkgeneration quite easily, without changing the System.
I am only changing how I request chunks. So that works fine.
PS: Since The bugtracker does not allow me to select anything else then 1.15.2 i am selecting that.