Crossing any chunk boundry causes a very noticable lag (like the game dropped half a second's worth of frames). I was using F3+G to confirm where the chunk boundries were, but then testing it with F3+G turned back off.

The fact that this happens every 16 blocks is what upgrades it from an annoyance to a major problem.