After this happened multiple times I decided to test this out.

Spawn in a new villager right next to work station.  Villager acquires workstations.  Inspect villager data using data get entity command.  Villager confirmed to have memory of workstation.  Trade with the villager until locked out.  Run day night cycle.  Re-inspect villager and now memory is gone.  Work station has not been touched or moved.  This is a fletcher and is one of the villager that has been affected the most.  It seems to randomly happen to some of the other villagers.  I have found no way of resetting the fletcher other than killing him!