What I thought would happen was that I could leash my Horses to a fence post outside my house and go work on another project.

What actually happened is, as soon as I unloaded the chunk, The lead broke and my horse started wandering around.

To reproduce, Leash a horse onto a fence away from the spawn chunks and then move away from it until the chunk unloads. Then, come back and the lead will have broken.