I've been villager trading for years and I've never had this problem. 

I just got a Fortune III villager. I traded with it multiple times. I even bought one of the Fortune 3 books from it. I got the villager to level 2. Just for extra details, I remember the second level was a Loyalty book. Since the villager was so good I wanted to trap it in a house to keep it safe. So I put it in a boat and trapped it in a house. I then broke its profession block so I could move it into the villager's house.

When broke the block the villager CHANGED into an armorer. That villager was a level 2 librarian and when I broke its profession block it turned into a level 2 armorer. I have not traded with a single armorer in this world. This must be a bug.