Server experiencing extreme lags when Pillagers are loaded.

Game goes from running smoothly to crash like state. Nothing loads and players start dropping into the void through the world.

Took us a while to figure out Pillagers / Illagers were the problem, yet confirmed by loading a Pillager Fort (this caused what described above) We then killed all Pillagers there with console, which stabilised us to be able to play again. (Although short lived as they respawned.)

Appears to only be happening since 1.14.4. this was NOT an issue on our server for 1.14.3.