Sometimes I get off my horse and 2 minutes later I can't find it anymore, until after some searching I find it 100-200 blocks away, and it seems to wander exclusively in one particular direction.

Horses should remember the location where you dismount it and not wander further than a certain limit. This would be inappropriate for other mobs, but since horses can be quite valuable friends they shouldn't just disappear from where you left them.

I'm aware of leads, but I think this is necessary in addition to leads. I don't really want to tie my horse to a post every time I get off of it for 1-2 minutes.

In any case this looks like bug because no other mobs wander that far in such a short time.