When you go back into the overworld via nether portal, the game takes slightly longer to load in. When it finally loads in, the nether portal is in a completely different spot, around 5 to 50 blocks away from the portal you went into the nether with. This happened in a infinite survival multiplayer world with usually 5 people on it.