After joining a world in spectator mode with the gamerule spectatorsGenerateChunks set to false, chunks around the immediate spawning place will load, but chunks farther out than a few chunks either have a hard time loading / refuse to load all together.

Chunks that are affected by this appear to have some random sub chunks loaded, and a few more sub chunks will load after stepping into the chunk, but the chunk will refuse to load completely.

After setting spectatorsGenerateChunks to true or going into another gamemode will cause chunks to continue loading as normal, but setting the gamerule back to false and going into spectator mode, after flying around to the edge of your render distance the problem resumes.

 I can confirm the chunks that are not loading have been generated before.