What's left as of 1.20 Snapshots:

This bug was marked as fixed as of 22w42a, and although this is true for the throwing item AI, Allays can still be delayed for a while before deciding to follow the player. Even a few seconds can count when a mob needs to keep up with an actively moving player, and I know mobs are capable of doing it as tamed wolves are able to instantly start following as soon as they're out of range.

Although not consistent, this clip is a good example of how they can delay following for a few seconds:
AllayDelayedFollowing120.mp4

This clip also showcases at the end an example where the Allay decided to not follow for several seconds, and only ended up "wandering" close to me:
AllayDelayedFollowing120_2.mp4

And finally, here's a clip from Bedrock edition to showcase how instantaneous the mob is able to switch to follow mode as soon as I'm out of range:
AllayFollowingBedrock.mp4

This seems to be all that's remaining of the bug as of 22w44a.

-----------------------------------------------------

In Java edition, Allay's AI freeze for a second and hesitate before flying over to drop items at players or noteblocks. This makes them seem slow and unresponsive in comparison to the Bedrock edition, which throw their collected items at their target very quickly. Sometimes almost immediately.

The videos below show this difference between the two versions; it's very easy to see how much slower the Java version is at throwing.

-------------------------------------------------------------

Edit: I have noticed that they seem to also sometimes hesitate before doing ANY action, such as following when a player gets out of range, or noticing items. Throwing items, however, is where it occurs most consistently. 

This is particularly obvious with following. As noted in this video, AllayFollowDelay.mp4, It's possible for the Allay to "hesitate" or delay before deciding to follow the player for so long, that a fast moving player may get out of range for the Allay's following mechanic, causing it to be left behind.