Your horse can be ridden to travel faster than you can on foot. You automatically obtain a horse when you buy a Stable from Robin (10,000 gold, 100 hardwood*, 5 iron bars).
Horses in Stardew require no upkeep. They need no food, water, medicine, or petting. If you leave your horse
elsewhere, it will return to its stable safely overnight.
Right clicking your horse while dismounted and having a hat equipped will place that hat on your horse.
* Hardwood is easiest found in the Secret Forest where you can harvest 12 pieces a day.