When you try tame horse and ride horse first time (Horse isn't tamed) and open inventory and add saddle then you can ride horse without taming.
You can't lead horse and If you take saddle off then horse will thrown off.