While holding the sprint button and riding a horse, everything is as usual, until after sprinting on the horse for a few seconds and then dismounting. When you dismount, you are still sprinting, and will continue to sprint until you hold down your sprint button again and move forward. Apparently, you can even sprint backward. Again, you do not have to be holding the sprint button after dismounting to continue sprinting.