The endermen can´t spawn far fom the main island in the end, even if there aren´t solid blocks except the area that you want in the charge chunks.

You can see it in the video.