Whenever a friend joins my Minecraft world  (over invite with Xbox account) my skin will change to theirs, and if I change my skin back their skin changes to mine.  If they leave the world then come back the issue goes away.  This can also affect multiple people in the world.  Like when me and another friend are already on the world, we've sorted the issue then another friend joins and all of our skins change to match the new player.