First of all, I only tested this with 21w44a, this might affect versions between 1.14.1 - 21w44a.

(Farmer) villagers prefer looking at players over working.
Farmers will start to wander off towards a grown crop for a short moment and quickly turn back around to continue staring at the player.
According to Minecraft Java Edition 1.14.3 Villagers, especially Farmers should prefer working during work hours.

Steps to reproduce:

• Make a crop farm using a Farmer villager.
• Stand somewhere in the farmable area.

Result:

• Villager will look at the player and not be able to work.

Suggestion:

• Make Villagers prefer working over looking at Players.
  or
• Make Villagers move to another task after a couple seconds of idle time while looking at Players.

Video by frikinJay: (they are fine with me using the video)
https://youtu.be/2jJAETygbzk
Videos by me:
45 seconds of the villager looking - https://youtu.be/_ICYIMEV9NI
40 seconds switching - https://youtu.be/b5fIXgtKWQI