Server is running mostly default survival settings except for simulation distance of 6 and chunk render distance of 32. Has been stable for the past year without issue until around 1 month ago. At first we believed the crashes to be caused by entering portals too quickly (going from nether to overworld, then overworld to end portal fast) since some of the crashes were occurring under those conditions, but we have also been getting crashes throughout normal gameplay, when performing normal actions such as placing blocks or even walking around. The crashes occur in multiple areas, overworld, nether, and end. There doesn't seem to be a specific chunk or groups of chunks causing this since the same areas may be fine one moment, then crash at another. As of now the server will crash at least 4-5 times a day if people are active. Checking memory usage showed a steady usage of around 700MB our of the 3GB allocated, so there doesn't seem to be any issue there.
Made the bug public and removed the world file since I confirmed the memory leak occurs with a fresh world. Also adding some extra info from my comments below:
After further testing this seems to be a memory leak related to chunk loading. If you check the memory while standing still, not loading chunks, it should be stable. When you load chunks, whether it's via crossing portals, flying, etc. the memory use increases and not come back down, which it should not be doing. Running farms while standing still did not affect the memory usage for us. Passive mob farms, redstone mechanisms, etc. did not contribute to the memory leak. Here's a log of our memory use up until the server crashed again.
Crash reports are fairly limited in information, but here are some of them attached below.