[Update] No longer an issue apparently? Just retried for the 8th time and its fine? So not a consistently reproducible issue.

A world that has used the SphaxPure BD Craft Resource Pack and HD UI and forced other players to download was uploaded to a Minecraft Realm. After re-downloading it and removing these, all Players inventories (XP, Items, Armor, etc...) was lost but map changes such as buildings and changes to the environment stay in effect.

Effects present when doing this on both Xbox One and PC - Windows 10.