This happens for example to the Armorer or to the butcher. In certain villages there is a block on top of the job site block , i.e. Bricks on top of Blast Furnace, or stone fence on top of smoker) then the villagers will not find a direct way to the block, and the wont claim it. See example video.