I went to a pillager outpost and then noticed how there wasn't any pillager's spawning around it (it was day time). Then once night fall's that's when the pillager's start spawning around the outpost.