Some times when I (the player) move far enough that the game no longer renders the villager which is in the minecart, it seems they get separated from each other, which is not noticeable until I move the minecart. When I move it the villager stays in the same position as before, when I broke the minecart the villager gets teleported to the correct position where it's supposed to be in the minecart before I broke it. The "teleported" villager always wants to go back where he thinks he was before I broked the minecart, like the normal villager (in a village) wants to go inside a house at night. This issue gets fixed when I relog to the world.