This is happening on our brand new 1.17 bedrock realm. We have a villager breeder that is over 200 blocks away from our spawn village. In between the breeder and the village we have a place where we can place a workstation and trade with the villager to set it's profession. We break the workstation and move the villager to our spawn village, where they very quickly switch profession to whatever workstation is nearby, and then will not change back. Have had this happen 3 times just this morning.