When I saw patrols and try to attack them, it sometimes invisible (It actually 'invisible' because they shot arrows to me!), so I can't attack them.

In that situation, I have to save&reload game. After reloading their invisible is over.