When enderman spawn, they appear to have been "agroed" or looked at. They display the angry, open mouth and the hostile noises, but even when standing right next to them, they don't attack unless truly looked at.