During a raid, pillagers will spawn underground & in caves. I fear these spawning conditions can also lead to pillagers spawning inside of villagers' houses.

As mentioned in other tickets, pillager patrol spawns are also uncontrolled, so this might be a spawn condition error that's specific to pillagers (they should only spawn in dimension 0).