[BDS-3103] Major lag spikes, and low resource utlisation, and other issues Created: 07/Feb/20  Updated: 09/Aug/20  Resolved: 09/Aug/20

Status: Resolved
Project: Bedrock Dedicated Server
Affects Version/s: 1.14.0.9
Fix Version/s: None

Type: Bug
Reporter: Edward Dolezal Assignee: Unassigned
Resolution: Invalid Votes: 4
Labels: None
Environment:

Windows 10 Pro


Attachments: PNG File 11.png     JPEG File mobs.jpg    
Confirmation Status: Unconfirmed

 Description   

Im getting major lag spikes in certain areas on my server especially near spawn and my house,

the issue seem to occur after friends bred quite a lot of chickens making the game barely playable when I went  to check the resource usage in task manger it appears the dedicated server tool uses barely and of the available CPU or ram, I have an i7 3770 with hyper threading enabled and 32 gb of 1600mhz ddr 3 ram.

the server hardly uses any more than 200mb of the 32 gb ram and doesn't even scale correctly across the 4 cores or 4 threads, this leads to extreme lag and I was under the impression that bedrock was multi threaded.

its acting like there is a resource cap.

the other issues I find are the lack of support for upnp, which if it existed would make my life easier especially when I reset my router and forget to re enable port forwarding, since its a painful headache, due to the servers lan ip changing after the reset.

also a native method of joining the server from consoles since I'm getting tired of having to use work around methods and tools just to join from my switch and I cant join friends servers on my Xbox due to lack of native connection methods which I fund extremally frustrating



 Comments   
Comment by IonicEcko [ 09/Aug/20 ]

Thank you everyone for your participation on this report.

Based off the description and the discussion following I'm going to close this one off as it currently doesn't appear to be related to a bug but rather just the general performance of BDS. Please also know BDS-2574 is tracking significant performance issues on Linux.

On the issue of lag, its important to note Bedrock Dedicated Server, while multi-threaded, only spins off additional threads for some workloads. The main game loop is still single threaded meaning that on a server with a low single-core performance your server is going to be able to handle far less players than something with a higher single-core capabilities.

Additionally, because of this you will never see a CPU at 100% when lag begins to set in.

Its important to note there is a significant difference between client lag (600 chickens in a 1x1 square causing FPS to drop in the area - server is fine) and server lag (slow/jittery entity movement, action lag - client is fine).

Issues caused by server lag because of high resource utilisation do not warrant a bug report on their own, as the issues are purely resource constraint based and general performance improvements, while important, are not tracked as a bug.

However if we're able to produce a scenario in which we can cause unreasonable server lag (for example standing a cow on a trapdoor causes 100% CPU utilization) please feel free to raise a new report with steps to reproduce.

Quick Links:
📓 Issue Guidelines – 💬 Community Support – 📧 Customer Support – ✍️ Feedback and Suggestions – 📖 Game Wiki

Comment by Jakob Knigga [ 28/Apr/20 ]

So, I wonder now if this is related to mobs not despawning. Currently when my server starts stuttering, I kill all hostile mobs. That usually helps, although there have been a couple instances where it server still lags.

I was pretty amazed when I saw how many mobs were being built up.

Comment by Otocin [ 11/Apr/20 ]

What is the problem?
Only the "CPU 0" is using nearly 100 percent.
I certainly let the server setting use all 16 threads.

Comment by Edward Dolezal [ 09/Apr/20 ]

If you read the CPU he said he's using is the generation before mine a 2000 series CPU so the upgrade would make logical sense in that case, a ryzen is no viable than an Intel CPU due to the amount of mitigations in place due to all the vulnerable of there architecture, all I made is a suggestion, if the other user decided to follow it it's up to them, I've been reading here about another user with similar issue who went that route and it resolved not for them I've personally been planning to do it myself, since I've already helped a friend upgrade form an FX series CPU to ryzen 6 core, I'm just waiting for a sale again on the CPU motherboard and ram bundles, it was actually quite reasonable in price the last few times.

Actually considering that the moden consoles are all using amd based hardware single core preformace becomes less relevant since most computing is shifting into he direction of multi core scaling, and better utilisation of multi core CPUs, single core preformace might be relevant in the current state of the software but as time progresses it will become even less relevant since Intel is shifting towards a multi core CPU approach.

The fact that you keep stating that the software is infact Multi threaded which I have seen upon further analysis proves that the issue is less about single core preformace and more to do with IPC or instructions per clock especially when it comes to mobs and other ai such as villagers, even down to chunk loading.

Whilst I do understand the need to strong single core preformace, IPC is more important.

I also so wish there was a better system requirements guide to help people decide on budget for a server and to be able to better understand what they need to get hardware wise to run the amount of players that they wish to host for like a recommended hardware guide since when I was reading about Java the recommendation for CPUs was a I7 4770 or better.

Comment by IonicEcko [ 09/Apr/20 ]

Single threaded performance is just as relevant to Bedrock Dedicated Server given the main game thread limitations. Please refrain from giving potentially expensive advice if you're not confident on the logic behind it.

Of course a Ryzen would help in your scenario because it would be replacing a roughly 9 year old CPU... Almost anything new will beat that.

A Ryzen will help anyone upgrading from an old device, but single core performance is king, so a Ryzen may not be the best choice for their personal scenario. It needs to be weighed up based on budget and preference.

The Ryzen ranges first entry in single core performance comes in at about #19 on the list and that's one of the top (desktop not HEDT) offerings. The 6-core processors rank 37th.

Ionic

Comment by Edward Dolezal [ 09/Apr/20 ]

@IonicEcko I was what I read fixed the issue for someone else, and in my own personal case it would apply, from the CPU utilisation I've seen seems like this PC I'm using to host is too old and under powered to host keeping in mind it is and old office pc.

Also I've seen the core usage as of recently it seems to spike in sequence across cores, I'm convinced that in my case a 6 core ryzen will fix my issue probably due to the PC I'm currently using to host might be having power supply issue therefore causing issue.

Single threaded preformace is mainly relevant to java Minecraft.

The PC I'm using is a HP elite 8200, so as you can see it's probably not ideal for the use.

Comment by IonicEcko [ 09/Apr/20 ]

Please don't anyone upgrade to a "6-core Ryzen" per Vexira's advice. Its not going to solve your issue and if you have money to spare that's not the right way to go about it. You want to look at single core benchmarks and Ryzens lose out on all of them.

BDS is multi-threaded, that's not disputable. You can prove this to yourself by running a server with a decent player base and monitoring the CPU usage. However only certain workloads spin off to a different thread. So far from my testing one big one appears to me mob AI. If you stick a bunch of cows on lower half slabs you should see a higher CPU load on a core separate from the main thread. Villager AI however is on the main thread (and hits it quite heavily at the moment, there is a bug over in the MCPE project for that).

But its not as simple as "just make it multi-threaded" and that doesn't do anything to help progress the report.

If you're having issues please do some analysis on your resource utilization and how its affected in different areas of your world and post back. For example, "My BDS server is only at 6%" is misleading, one core might be maxed out while the remainder are underutilized.

Comment by Otocin [ 09/Apr/20 ]

I'm using AMD Ryzen 7 3700x 8-core processor and BDS seems to use only one thread.
Of course, the maximum thread use in server settings is set to 0.

Comment by Jakob Knigga [ 08/Apr/20 ]

Yep, switched to Ubuntu (same server hardware) and had the same performance.

Comment by Edward Dolezal [ 08/Apr/20 ]

I tired it not dice it's about the same, results in preformace, I'm waiting for the major optimisation update to happen, and I agree it's server side not client, it seems that the the server might be incompatible with hyper threading, or not working correctly with it.

Comment by Jakob Knigga [ 08/Apr/20 ]

I am going to switch to the Ubuntu server to see if that performs any better. Having to buy a new processor to run Minecraft server seems insane.

Comment by Edward Dolezal [ 08/Apr/20 ]

@Jakob Knigga it's an issue with the code, and the server software, it seem that there is most likely and from what I've read the most immediate fix is to upgrade to a 6 core ryzen processor, inorder for it to be stable performance wise.

Comment by Jakob Knigga [ 08/Apr/20 ]

I too have been experiencing something very similar to what @Edward Dolezal is describing. It's definitely not client side lag. The game stutters like the server can't keep up. When I look at the resource utilization though, the server is hardly using CPU. I have attached a video showing what is going on. In the upper left you can see that my FPS stays high, but the animals start stuttering. During this time, the bedrock_server.exe CPU utilization never goes over 6%.

 

I am running BDS 1.14.32.1

Server - i7-2600K with 16GB RAM

Edit: I was having trouble uploading my video to the ticket. Here it is.

Comment by Edward Dolezal [ 17/Feb/20 ]

IonicEcko i can upload the world to media fire if you want or if you have preference i can upload to google drive, after ive been testing im starting to believe that the issue is with the cpu not being capable of hosting a world with that may mobs, from what i can see is that hyper threading is most likely not helping preformance that much.

Comment by Edward Dolezal [ 13/Feb/20 ]

@Pablo im running vanilla no mods because bedrock has none available, only custom servers do.

Comment by Pablo [ 12/Feb/20 ]

I had this problem with an Addon, yFarmcraft, many chickens spawning in the same zone and a terrible lag. I fixed it deactivating the addon.

Comment by Edward Dolezal [ 12/Feb/20 ]

I found the issue the client might be multi threaded, and the server should be the one to handle the majority of the leg work to reduce client performance issues, at in the server should handle the bulk of calculations for ai etc, the server is not exactly multithreaded I now can confirm its more  single threaded or its possibly bugged which would make sens+e.+

I was watching an osd tool of sorts and noticed that one core at a time will be pegged at 50% utilisation then it will cycle and alternate cores the core that is spiking, from my understanding of  multithreading the load would distribute across all cores and not peg one at 50% at a time, across 4 cores.

unless what im seeing is a bug I can upload the picture I took and the world if need be, also the other issue I noticed is that when someone joins the world the cpu utilization will spike upwards of 60-90%, then it will settle shortly after ive been watching task manger.

This has me leaning towards a possible problem with hyperthreading and the server tool, which has me thinking that a cpu with more physical cores would be ideal especially under load.

Aslo what is with the secondary ports do I need to forward them as well.

Im hoping we will have a gui based tool soon with upnp support since the client has it and the option to set an automatic reboot and back up, since I've found my self rebooting the server to fix the bug where players get suck in walls and drown, and also that future updates will let the server handle the majority of processing allowing lower end clients such as nintendo switch and mobile phones to be able to connect with less performance issues.

Comment by Edward Dolezal [ 11/Feb/20 ]

I was under the assumption, that the server should be taking the brunt of the loading of chunk generation, and handling the AI, therefore alleviating the load on client devices, such as Nintendo switch, and some older android and apple phones, which would struggle greatly, im hoping that in future updates they an offload most of the processing to the server rather than to the clients, and that it will be able to take advantage of newer processors.

im just wondering if the issue is my current CPU and hyper threading causing the lower prefomance under pressure.

Comment by Edward Dolezal [ 10/Feb/20 ]

Another question what exactly is the secondary set of ports In the command line for,  aside form the default ports, because with DNS redirector and using a port other than default I got my switch to see the server as a LAN server I'm hoping for upnp support in the future like the client has to make set up easier, especially due to the secondary ports changing when the default port is used, and when a non default port is used the default ports will become the secondary. I'm wondering if opening them would help with clients joining?

Comment by Edward Dolezal [ 10/Feb/20 ]

Strangely i was talking to someone on the discord in technical feedback, about the server who was telling me the sever only sits on one core, of the CPU, that aid its still single threaded and the AI is hitting that one thread but from my understanding is the client software is multi threaded, but not so much the server which i understand is till in alpha, so my other question is how does the server scale across multiple cores as in more than 4 of them, im considering hardware upgrade maybe sometime later this year, and im wondering it it might help  especially when i have more friends join the server.

also i will upload a copy soon as i have the chance to.

ive found that my switch suffers from im assuming is a frame drop, but is more like stutter thought, i did see a spike in disk usage on the SSD, the CPU utilization remains low as well as the ram utilization, all i can see its using 400-500 mb of  ram and barely any CPU.

i was assuming that most of the heavy lifting should be done by the server itself, therefore any issues should be more server side vs client,  unless the issue im facing is related to hyper threading.

 

ill test host it on my main pc, my friend who has the worst latency or issues on pc is running the same CPU i7 3770  as the server the other friend is using an iPhone 6, so im guessing the cpu is under the requirement for a server and a ryzen 6 core possibly would have better results especially with more  AI in a farm and red stone machines in future?

But my other most question how does the server render distance work and ticks, in regards to cpu and memory usage since after increasing server render distance to 96, my friend noted better prefomance overall when he joined from pc.

Comment by IonicEcko [ 10/Feb/20 ]

Hey Edward,

Sorry I've been working through as many BDS tickets as I can trying to help get this section cleaned up a little so didn't get time to read your last message (the long one) until now.

FPS drops are 100% client side. If you're seeing frame drops thats because your client PC can't keep up. This doesn't necessarily correlate to 100% CPU usage or GPU usage as not everything can be multithreaded but Bedrock is absolutely multithreaded.

I, for example, have a guardian farm on my server that shuffles guardians off to the nether to bypass the mob cap. When going into the nether there can be 4000+ guardians there which kills my FPS like crazy, but the server itself is fine. I can have 10 others on the server who have no idea I'm destroying a certain part of the Nether.

If you're able to upload a copy of your world I can chuck it on my server and see if I have the same issues but at the end of the day, if a billion chickens is killing a part of your world your friends need to stop making client-unfriendly farms.

Java doesn't suffer from this exact issue but thats only because they prevent entity cramming and kill mobs once theres more than 30 (ish) on one block.

Ionic

Comment by Edward Dolezal [ 10/Feb/20 ]

Extremely low CPU usage, especially in the most areas where the performance drops the most, i would most expect CPU usage to spike across all cores, in general even the ram utilization in resource manger is quite low lower than i would expect which doesn't make logical sense.

unless its an issue with the hardware as in the power supply  is not enough or i need a better motherboard which i would accept as a possible issue considering that im using a hp Compaq elite 8300 SFF

Comment by IonicEcko [ 10/Feb/20 ]

Hi Edward,

What makes you think the server software isnt multi-threaded? From my experience (currently have 4 cores assigned to my main BDS VM) it utilises them all fairly well.

Comment by Edward Dolezal [ 10/Feb/20 ]

I'm starting to believe that the server soft were needs multi core multi threading support, in order to resolve, performance issues like the one im facing especially with mobs, form my understanding and research its like the mobs, ticks and chunk generation should be spread across multiple cores and threads, that way it would eventually be possible to have an infinite loop railway system where the cart could circle a track with out a player present.

im currently looking at a 6 core Ryzen CPU to see if that solves the latency issue but im considering that it might be a waste if the server is unbale to utilize the available and threads.

Comment by Edward Dolezal [ 08/Feb/20 ]

what im seeing is major slow downs as in the frame rate drops to im going to estmate form 10-3 frames making it  a near slideshow, and the item pick up for example there is a delay when picking up eggs, or opening chests or menus like inventory.

from what my understanding is the game seems to only run off of one core just like java, which to me is quite archaic I hope they mange to get the server to scale across multiple cores considering that multi core cpus are becoming the standard,

in regards to console joining, they could make custom servers function like realms where as there is an invite code issued and used to allow people to join, that could bypass platform restrictions.

 

the other thing that bugs me is the lack of a gui based tool, where id be able to set up automatic updates and back ups and reboots much like the Linux based script I used when I tested the ubuntu version of server as part of troubleshooting, id love to have upnp support just like the game client has for the server it would simply everything for me and some others especially when using Linux, or windows, I have the tendency to reset my router  to factory every few updates, to keep it running well, and I have a tendency to forget to re ad port forwarding every time.

 

I found the same issue with performance issues on Linux also ive tested a few distros and had the same results

performance wise in the same areas, ive been see low resource usage on ram and cpu, so it seems to me that there is more of a restriction in the server software it self on how much memory it can use.

im hoping the do update it to  utilize available CPU cores in future updates, since im planning to upgrade the system to either a 6 or 8 core system in future,

 

the big issue for me is that the issue is that sever some of my friends have problems joining especially because the sapwn is in the worst affected area, I know from a friend on iphone6 she cant join, one of my friends on pc gets hit hard when he goes to the farm or the basement of my mountain house, the house is near spawn so is the farm my gf cant even move because of how sever it is.

 

 

Comment by IonicEcko [ 07/Feb/20 ]

Hi Edward,

When you talk about lag spikes what exactly are you seeing on your server in these areas? While Bedrock is indeed multi-threaded it isn't a silver bullet so to speak. Some thread just have to complete before others can continue. The areas where you are experiencing lag, are they where the chickens are?

I have a massive guardian farm that, even when I accidentally leave it running too long, it seems to cause FPS drops for me, but others in the area don't seem too affected.

Joining servers from Console has been something Mojang has addressed in the past, unfortunately it is a requirement of the different platform owners (such as Sony and Nintendo) that the custom server options be stripped out of the code before it can go through their approval process. As such there is nothing Mojang can do in that space.

Ionic

Generated at Sat Jan 11 07:24:08 UTC 2025 using Jira 9.12.2#9120002-sha1:301bf498dd45d800842af0b84230f1bb58606c13.