I was playing in my survival world solo, then I got bad omen from killing a pillager captain. I was going to set the game to hard mode to get more waves, but I forgot on first try. Luckily I had a backup (copy) of my world in case something went wrong. Every time a wave failed to start, I would copy my backup, then delete the failed wave world and play the backup, essentially allowing me to retry if things didn't go as planned.

Normal Mode, Wave 5 failed (to start)
Hard Mode from here on, Wave 2 failed
Wave 3 failed
Wave 2 failed
Wave 1 failed (and thus the raid failed to start after the bar filled up)
Wave 2 failed
Wave 1 failed
Wave 1 failed

My villagers were housed on an island and there was a tiny island separated by a one-block wide gap where water ran through, and a much bigger island ~20 blocks away. Pillagers spawned only on the coast of the big island or the center of the small island. I had 3 iron golems on the island the villagers were on, who were on the outside of the buildings to fight.

This link leads to a mostly irrelevant video but it shows you what the "village" and islands look like, if it helps to see.
https://xboxclips.co/onepowerhour/503536d1-9c67-4a19-8262-ed3a843d563a

I have seen a post that had the same issue on Java Edition, but for an earlier version (1.14.2), meanwhile this occurred in 1.16.220..... I hope this resolved soon. Thank you!