My Wife and I have a separate servers and a Realm together and this has happened on all of them. We make our main portal and enter it, to return in a different location other than the original place. It worries me on nether traveling to completely new locations. 

First pic is Secondary and Second pic is the Main portal. We are not returning back to Main site