Even though the minecraft launcher shows that the custom skin is currently being used, the player will wear a default skin when playing the game. There are no error messages related to this in the launcher.

Edit: does not seem to happen when "use beta version of launcher" is checked off