Alerted pillagers from an outpost, of a player in survival, are still  to track a player that switched  in spectator mode and even shoot at the spectator player. Even pillagers in a sealed of cave can follow the movement no matter what. The continued line of sight does not reapply in spectator when you go out side of their range. Also I thought pillagers could only spawn on the upper most block, not in a sealed cave