Device attached to the side of saddle to support rider's foot