Having large amount of free work sites (e.g. 256+) can cause a second of lag, However increases with the more work sites there are where a couple of thousand can cause each tick to become 20 seconds each. Whilst this may not be feasible to achieve in survival mode, this could cause major issues on creative mode(or build) servers.

Reproduction:

1. Fill a large area with a work site block

   /fill ~ ~ ~ ~32 ~ ~32 minecraft:fletching_table

1. Spawn a villager nearby(preferably in the center)

   /summon minecraft:villager