Very rarely two villagers would switch workplaces with each other even if they are of two different professions. Note that both villagers would attend work there consistently as if it was normal and will be able to restock even though the jobsite block doesn't match their profession.

I only started noticing this behaviour after zombie curing, so it may possibly be something related to that.